Study design and other considerations

This page briefly describes three commonly used study designs, and the advantages and disadvantages of each:

1. Cross-sectional study
2. Cohort study
3. Case-control study

1. Cross-sectional study

References: Oxford Centre for Evidence-Based Medicine; Lix, 2006; and Mann 2003

Cross-sectional study designs are used when studying one or more variables within a given population at one point in time. Such studies are useful for establishing associations rather than causality and for determining prevalence, rather than incidence.

Examples of MCHP studies using a cross-sectional design:

• Kozyrskyj 2001

2. Cohort study

References: Oxford Centre for Evidence-Based Medicine; and Lix, 2006

In cohort studies, a group of people within a population is followed over a specified period of time to track who experiences or develops the same significant life event or treatment. This type of design can be used "to study incidence, causes, and prognosis. Because they measure events in chronological order they can be used to distinguish between cause and effect." (Mann 2003) Cohort studies can be done prospectively, retrospectively, or using cross-sectional methods. As well, two groups may be followed: one containing the agent of interest and the other acting as a control group.

A sequential cohort study is an example of a cohort study which, instead of following a single age-homogeneous cohort, uses two or more distinct age cohorts and tracks each for a shorter period of time than in a regular cohort study. This convergence model combines cross-sectional and longitudinal data: there is a simultaneous model fitting of between- and within-individual trajectories over a wider span of time than observed longitudinal trends. This type of cohort study is efficient, potentially more representative (less longitudinal attrition), and reduces cumulative testing effects. However an (age x cohort) interaction may threaten validity of inferences (Lix, 2006).

Examples of MCHP studies using a cohort design:

• Menec 2004
• Morgan 2003

3. Case-control study

References: Oxford Centre for Evidence-Based Medicine ; Mann 2003

Case-control studies are used to compare cases who have a certain condition with a control group known not to have developed the outcome of interest. The control group is usually not only taken from the same population base, but also matched for age and gender. Such studies "seek to identify possible predictors of outcome and are useful for studying rare disease or outcomes. They are often used to generate hypotheses that can then be studied via prospective cohort or other studies" (Mann 2003).

Family-based design is a specific type of retrospective case-control design. Related persons are used as the study control group, which "confers robustness against the potentially biasing effects of genetic admixture." Designs using sibling or cousins are "useful for diseases with early or later onset and can be analyzed by using conditional logistic regression, with fine stratification on family." Designs using parents are useful for studies involving birth defects or diseases with early onsets. (Weinberg 2000).

Examples of MCHP studies using a case-control design:

• Kozyrskyj 2005
• Cohen 1993

References

• Brownell M, Lix L, Ekuma O, Derksen S, De Haney S, Bond R, Fransoo R, MacWilliam L, & Bodnarchuk J (2003). Why is the health status of some Manitobans not improving? The widening gap in the health status of Manitobans. [Full Report - Why is the health status of some Manitobans not improving? The widening gap in the health status of Manitobans (PDF)] [Summary - The most healthy vs. the least: the widening gap in Manitoba. (PDF)]
• Cohen MM. (1993) Using administrative data for case-control studies: The case of the papanicolaou smear. Ann Epidemiol 3(1):93-98
• Institute for Work and Health. What researchers mean by ... retrospective vs prospective studies. Atwork newsletter. Issue 59, Winter 2010. [accessed July 6, 2011].
• Jayawant N. Mandrekar, Sumithra J. Mandrekar.  An introduction to matching and its application using the SAS® system SUGI 29 Proceedings. Montreal, Canada: May 9-12 2004.
• Kozyrskyj A, Lix L, Dahl M, Soodeen R ( 2005). High-cost users of pharmaceuticals: Who are they? [Full Report - High-cost users of pharmaceuticals: Who are they? (PDF)] [Summary - The high cost of high-cost drug users in Manitoba. (PDF)]
• Kozyrskyj A, Mustard C, Derksen S (2001). Studying health and health care use in at-risk goups.
• Lix L. Observational epidemiology: design and analysis of longitudinal data. Lecture notes. University of Manitoba. February 2006.
• Lori S. Parsons. Performing a 1:n case-control match on propensity score SUGI 29 Proceedings. Montreal, Canada: May 9-12 2004.
• Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J 2003; 20:54-60. [Full text - Observational research methods. Research design II: cohort, cross sectional, and case-control studies (PDF)] 
• Mayo Foundation for medical education and research. Locally written SAS macros.  [accessed February 6, 2006 - no longer available on-line.]
• Menec V, Lix L, Steinbach C, Okechukwu E, Sirski M, Dahl M & Soodeen RA (2004). Patterns of health care use and cost at the end of life. [Full Report - Patterns of health care use and cost at the end of life. (PDF)] [Summary - On death and dying in Manitoba. (PDF)]
• Metge C, Kozyrskyj A, Roos N (2003). Pharmaceuticals: focusing on appropriate utilization. [Full Report - Pharmaceuticals: focusing on appropriate utilization. (PDF)] [Summary - Drug prescribing in Manitoba: how appropriate is it? (PDF)]
• Morgan S, Kozyrskyj A, Metge C, Roos N, Dahl M (2003). Pharmaceuticals: therapeutic interchange and pricing policies. [Full Report - Pharmaceuticals: therapeutic interchange and pricing policies. (PDF)] [Summary - Controlling prescription drug costs in Manitoba. (PDF)]
• Oxford Centre for Evidence-Based Medicine. Accessed February 22, 2006.
• Young K. Population Health: Concepts and methods. 2nd ed. New York: Oxford University Press, 2005.
• Weinberg CR, Umback DM. Choosing a retrospective design to assess joint genetic and environmental contributions to risk. Am J Epidemiol 2000; 152(3):197-203.

There are other considerations that may help to determine the scope of your study. For example, the demographic characteristics of subjects, treatment and provider characteristics, geographic considerations, and changes that might occur during the timeframe for your study.

In addition, the order in which inclusions/exclusions are defined is important!  Although analyses may exclude certain records, information on these records may be necessary for methodological definitions. For example, if hospital transfers are part of the analyses, they should be defined before exclusions are applied. In the case of transfers to an out-of-province hospital, these would not be captured if out-of-province hospital exclusions had been done prior to defining transfers. It is also advisable to "over-include" when creating study subsets if some of the eligibility criteria are not yet firm. And it is always a good practice to always run frequencies to determine how many records are affected.

Examples of methodological considerations for the following key characteristics are described below:

1. Demographic characteristics of health care user

Age

Variables available for age calculations can differ among databases, and it may be necessary to go the Registry data for this information (for example, when defining a cohort). In addition, analyses may require different age definitions. An individual may be eligible for a study based on age as of service date of a particular claim while for rates analyses, age might be calculated as of December 31.

Selecting eligible age groups for study depends upon the research question and the available data. For example, for certain disease categories, the event may be rare in younger age groups and a decision may be made to exclude such cases to avoid inflating the denominator.

MCHP concepts and resources related to Age:

• Age - provides examples of different methods for defining age.
• Age discrepancies - illustrates potential problems in age calculations over time.
• Population denominator - important age-related issues in preparing data to be used as a denominator in analyses

Residence

Like age, residence definitions will depend upon the variables available in the databases and upon the nature of the analyses. For example, changes in residence over time need to be taken into account. For cohort and rates analyses, ideally Registry residence as of December 31 should be used. However, this can be time-consuming to set up so an alternative is to use residence as of the first-occurring utilization record (or as of the most recent utilization record in the fiscal year, or the most frequently-occurring residence for an individual). Analyses involving a numerator and denominator will need to ensure that location is defined the same way.

Typical residence exclusions include:

• Out-of-province residents: individuals showing a residence outside Manitoba on their records.
• Non-residential postal codes: depending upon the type of analysis, postal codes associated with the physical location of departments such as the Office of the Public Trustee and Family Services are excluded. Records with these postal codes represent multiple individuals for whom actual residence is not known.
• Records with missing PHINs (primarily newborns) may all show the same residence, and may thus also need to be excluded.

MCHP concepts and resources related to residence:

• Residents of Manitoba - definitions using postal code and/or municipal code.
• Residential mobility - ways of defining and operationalizing mobility.
• Follow-up issues - issues surrounding loss of longitudinal information, such as moves into and out of the province.
• Postal code - description of how postal code is entered in Manitoba Health data and implications related to changes in postal code over time.
  • Public Trustee Office
  • Child and Family Services

2. Treatment and provider characteristics

Diagnosis and treatment

Analyses of overall utilization might group all services provided together, regardless of type of treatment.  For example, all inpatient hospitalizations or all physician visits that took place in Manitoba for a given fiscal year. Or information about specific diagnoses, surgical procedures, and/or drug prescriptions, may be obtained using common classification schemes typically found in administrative health data:

• Diagnoses and procedures - the International Classification of Diseases (ICD) system is used, for example, in the hospital discharge abstracts database and the physician services database. These ICD codes can in turn be grouped into broader categories (e.g., DRGs, ACGs, and CMGs). Procedures are classified in the medical claims using tariff codes.
• Prescription drugs - are normally classified using ATC (Anatomical Therapeutic Chemical) codes.

Because classification systems generally are subject to change annually (with implications for grouper software), it is important to ensure that the codes being used to define an event are appropriate for the study years. For example, the ICD-9-CM code for total knee replacement was "81.41" prior to 90/91, changing the following year to "81.54". Crosswalks are sometimes available to help deal with not only historical changes in systems but also with the possibility of different classification systems across databases.

MCHP concepts and resources related to diagnosis and treatment

• ICD - MCHP concepts
  • ICD changes in MCHP data over time
  • ICD-9-CM/ICD-10-CA conversions
• Classification of diseases, functioning and disability - National Center for Health Statistics website
• Groupers - MCHP concepts
  • CMG overview  - Case Mix Groups
  • DRG overview - Diagnosis Related Groups
  • CMG vs DRG - a comparison of CMGs with DRGs
• Drug classification systems - MCHP concept: Drug classification systems: an overview

Provider characteristics

As with other key variables, provider labels and/or numbers can change over time. Another type of change is a change in designation; for example, a facility may change in purpose from acute care to a chronic care hospital.

Typical provider exclusions: out-of-province providers/facilities.

MCHP concepts and resources related to provider location:

• Hospital types - categorized by size and location
• Hospital regions {internal only} - to format numeric hospital numbers to the hospital postal code and municipality code
• Physician service areas - also see service areas/RHAs

3. Geographic area definitions

The choice of small area definitions will vary with the purpose of the study. For example, if a study is done as a collaboration with the Winnipeg Regional Health Authority, Winnipeg will likely need to be defined using an RHA definition. If the study is intended for comparison with Statistics Canada's reports, then St. Paul, for example, should not be included as part of Winnipeg because Statistics Canada does not consider St. Paul part of Winnipeg.

Conversion tools are maintained at MCHP to permit cross-walking between different methodological definitions of areas. It is important to note that areas can shift boundaries over time. For example, Whitehead RM moved into Brandon RHA (from South Westman) sometime during 1998/99 and Headingly moved out of Winnipeg in 1995/96. Normally the current definitions of areas are used and applied retrospectively, not the definition of the area at the time of the data (unless the study dictates otherwise).

MCHP concepts related to area definition:

General

• Small area analysis

Manitoba

• Small areas of Manitoba - a listing of various ways to group areas in Manitoba, including approximate population sizes of the small areas (Manitoba Health and Statistics Canada). Detailed information is available within the following MCHP concepts:
  • RHAs in Manitoba - Regional Health Authority (largest level of Manitoba Health area)
  • RHA districts and zones in Manitoba - (next largest Manitoba Health areas, as of 2004)
• Macros for uniformly assigning records to RHAs {available internally}

Winnipeg

• Winnipeg area definitions - including community areas and neighbourhood clusters
• Winnipeg Sub-Regions (N=9)
• Winnipeg areas and RHAs
• Measures of neighbourhoods
• Community centre areas (N=72/75)

Conversion / crosswalk files (available internal only):

• PCCF (Postal Code Conversion File) - to crosswalk between postal code and Statistic Canada census areas (including material added by MCHP). For some small area analyses, it may be necessary to access earlier year-specific versions of the POSTMUN file.
• WRHA conversion table - to convert combinations of municipal code and postal code to Winnipeg neighbourhood areas (CCA, NC, neighbourhood).
• ICD conversion files - to convert different versions of ICD codes into a usable component over time.

4. Time frame of study period

The choice of study years may be limited by the years for which data are available. (Note that most data bases at the Centre are organized by the fiscal year of April 1 to March 31.) Several issues are relevant with regard to defining boundaries for study years:

Fiscal year boundaries

Explicitly defining the years of study helps ensure that only records for the year of interest are present. For hospital discharge abstracts, for example, date of separation is typically used to define the fiscal year boundaries. Note that when comparing utilization by year, dates outside the fiscal year are normally removed for each year to ensure comparability. When years are combined for analyses, only the dates outside of the multi-year period are removed.

Completeness of data

Depending on the data base, fiscal year data may be incomplete, and will require going to a subsequent year to pick up late submissions: for fiscal 97/98 hospital abstracts, for example, both 97/98 and 98/99 hospital abstracts should be accessed to obtain all separation dates falling within the April 1, 1997 to March 31, 1998 period.

Records overlapping fiscal years

Hospital stays can span two fiscal years of data; for example, a stay of March 30/98 (admitted in FY 97/98) to April 3/98 (discharged in FY 98/99) will be picked up in the FY 98/99 hospital abstracts, not in 97/98. This has implications for length of stay.