Study design and other considerations

A research study design depends a lot on the research question being asked. This  page briefly describes the advantages and disadvantages for three types of study designs: cross-sectional, cohort and case-control, and provides information on other considerations for your research.

Types of study designs

This page briefly describes three commonly used study designs, and the advantages and disadvantages of each:

  1. Cross-sectional study
  2. Cohort study
  3. Case-control study

1. Cross-sectional study

References: Oxford Centre for Evidence-Based Medicine; Lix, 2006; and Mann 2003

Cross-sectional study designs are used when studying one or more variables within a given population at one point in time. Such studies are useful for establishing associations rather than causality and for determining prevalence, rather than incidence.

Advantages and disadvantages of cross-sectional studies
Advantages Disadvantages
  • Simple and inexpensive
  • Ethically safe
  • Quick data collection
  • Attrition is not an issue
  • Holds time constant
  • Does not permit distinction between cause and effect
  • Recall bias susceptibility
  • Confounders may be unequally distributed between groups
  • Does not economize on subjects
  • Differences may be due to age/time effects or cohort effects
  • Inter-subject variability exists, making it harder to detect a difference

Examples of MCHP studies using a cross-sectional design:

  • Kozyrskyj 2001

2. Cohort study

References: Oxford Centre for Evidence-Based Medicine; and Lix, 2006

In cohort studies, a group of people within a population is followed over a specified period of time to track who experiences or develops the same significant life event or treatment. This type of design can be used "to study incidence, causes, and prognosis. Because they measure events in chronological order they can be used to distinguish between cause and effect." (Mann 2003) Cohort studies can be done prospectively, retrospectively, or using cross-sectional methods. As well, two groups may be followed: one containing the agent of interest and the other acting as a control group.

A sequential cohort study is an example of a cohort study which, instead of following a single age-homogeneous cohort, uses two or more distinct age cohorts and tracks each for a shorter period of time than in a regular cohort study. This convergence model combines cross-sectional and longitudinal data: there is a simultaneous model fitting of between- and within-individual trajectories over a wider span of time than observed longitudinal trends. This type of cohort study is efficient, potentially more representative (less longitudinal attrition), and reduces cumulative testing effects. However an (age x cohort) interaction may threaten validity of inferences (Lix, 2006).

Advantages and disadvantages of cohort studies
Advantages Disadvantages
  • Ethically safe
  • Subjects can be matched
  • Can establish timing and directionality of events
  • Eligibility criteria and outcome assessments can be standardized
  • Administratively easier and cheaper than Randomized Controlled Trial
  • Possible to examine multiple outcome variables
  • The controls may be difficult to identify
  • Exposure may be linked to a hidden confounder
  • Randomization not present
  • For rare diseases, large sample sizes or long follow-up are necessary

Examples of MCHP studies using a cohort design:

  • Menec 2004
  • Morgan 2003

3. Case-control study

References: Oxford Centre for Evidence-Based Medicine ; Mann 2003

Case-control studies are used to compare cases who have a certain condition with a control group known not to have developed the outcome of interest. The control group is usually not only taken from the same population base, but also matched for age and gender. Such studies "seek to identify possible predictors of outcome and are useful for studying rare disease or outcomes. They are often used to generate hypotheses that can then be studied via prospective cohort or other studies" (Mann 2003).

Family-based design is a specific type of retrospective case-control design. Related persons are used as the study control group, which "confers robustness against the potentially biasing effects of genetic admixture." Designs using sibling or cousins are "useful for diseases with early or later onset and can be analyzed by using conditional logistic regression, with fine stratification on family." Designs using parents are useful for studies involving birth defects or diseases with early onsets. (Weinberg 2000).

Advantages and disadvantages of case-control studies
Advantages Disadvantages
  • Quick and cheap
  • Only feasible method for studying very rare disorders or those with long lag between exposure and outcome
  • Fewer subjects needed than cross-sectional studies
  • Reliance on recall or records to determine exposure status
  • Confounders
  • Selection of control groups is difficult
  • Potential bias: recall, selection
  • cannot calculate the relative risk

Examples of MCHP studies using a case-control design:

  • Kozyrskyj 2005
  • Cohen 1993

References

Other considerations

There are other considerations that may help to determine the scope of your study. For example, the demographic characteristics of subjects, treatment and provider characteristics, geographic considerations, and changes that might occur during the timeframe for your study.

In addition, the order in which inclusions/exclusions are defined is important!  Although analyses may exclude certain records, information on these records may be necessary for methodological definitions. For example, if hospital transfers are part of the analyses, they should be defined before exclusions are applied. In the case of transfers to an out-of-province hospital, these would not be captured if out-of-province hospital exclusions had been done prior to defining transfers. It is also advisable to "over-include" when creating study subsets if some of the eligibility criteria are not yet firm. And it is always a good practice to always run frequencies to determine how many records are affected.

Examples of methodological considerations for the following key characteristics are described below:

1. Demographic characteristics of health care user

Age

Variables available for age calculations can differ among databases, and it may be necessary to go the Registry data for this information (for example, when defining a cohort). In addition, analyses may require different age definitions. An individual may be eligible for a study based on age as of service date of a particular claim while for rates analyses, age might be calculated as of December 31.

Selecting eligible age groups for study depends upon the research question and the available data. For example, for certain disease categories, the event may be rare in younger age groups and a decision may be made to exclude such cases to avoid inflating the denominator.

MCHP concepts and resources related to Age:

  • Age - provides examples of different methods for defining age.
  • Age discrepancies - illustrates potential problems in age calculations over time.
  • Population denominator - important age-related issues in preparing data to be used as a denominator in analyses

Residence

Like age, residence definitions will depend upon the variables available in the databases and upon the nature of the analyses. For example, changes in residence over time need to be taken into account. For cohort and rates analyses, ideally Registry residence as of December 31 should be used. However, this can be time-consuming to set up so an alternative is to use residence as of the first-occurring utilization record (or as of the most recent utilization record in the fiscal year, or the most frequently-occurring residence for an individual). Analyses involving a numerator and denominator will need to ensure that location is defined the same way.

Typical residence exclusions include:

  • Out-of-province residents: individuals showing a residence outside Manitoba on their records.
  • Non-residential postal codes: depending upon the type of analysis, postal codes associated with the physical location of departments such as the Office of the Public Trustee and Family Services are excluded. Records with these postal codes represent multiple individuals for whom actual residence is not known.
  • Records with missing PHINs (primarily newborns) may all show the same residence, and may thus also need to be excluded.

MCHP concepts and resources related to residence:

2. Treatment and provider characteristics

Diagnosis and treatment

Analyses of overall utilization might group all services provided together, regardless of type of treatment.  For example, all inpatient hospitalizations or all physician visits that took place in Manitoba for a given fiscal year. Or information about specific diagnoses, surgical procedures, and/or drug prescriptions, may be obtained using common classification schemes typically found in administrative health data:

  • Diagnoses and procedures - the International Classification of Diseases (ICD) system is used, for example, in the hospital discharge abstracts database and the physician services database. These ICD codes can in turn be grouped into broader categories (e.g., DRGs, ACGs, and CMGs). Procedures are classified in the medical claims using tariff codes.
  • Prescription drugs - are normally classified using ATC (Anatomical Therapeutic Chemical) codes.

Because classification systems generally are subject to change annually (with implications for grouper software), it is important to ensure that the codes being used to define an event are appropriate for the study years. For example, the ICD-9-CM code for total knee replacement was "81.41" prior to 90/91, changing the following year to "81.54". Crosswalks are sometimes available to help deal with not only historical changes in systems but also with the possibility of different classification systems across databases.

MCHP concepts and resources related to diagnosis and treatment

Provider characteristics

As with other key variables, provider labels and/or numbers can change over time. Another type of change is a change in designation; for example, a facility may change in purpose from acute care to a chronic care hospital.

Typical provider exclusions: out-of-province providers/facilities.

MCHP concepts and resources related to provider location:

  • Hospital types - categorized by size and location
  • Hospital regions {internal only} - to format numeric hospital numbers to the hospital postal code and municipality code
  • Physician service areas - also see service areas/RHAs

3. Geographic area definitions

The choice of small area definitions will vary with the purpose of the study. For example, if a study is done as a collaboration with the Winnipeg Regional Health Authority, Winnipeg will likely need to be defined using an RHA definition. If the study is intended for comparison with Statistics Canada's reports, then St. Paul, for example, should not be included as part of Winnipeg because Statistics Canada does not consider St. Paul part of Winnipeg.

Conversion tools are maintained at MCHP to permit cross-walking between different methodological definitions of areas. It is important to note that areas can shift boundaries over time. For example, Whitehead RM moved into Brandon RHA (from South Westman) sometime during 1998/99 and Headingly moved out of Winnipeg in 1995/96. Normally the current definitions of areas are used and applied retrospectively, not the definition of the area at the time of the data (unless the study dictates otherwise).

MCHP concepts related to area definition:

General

Manitoba

  • Small areas of Manitoba - a listing of various ways to group areas in Manitoba, including approximate population sizes of the small areas (Manitoba Health and Statistics Canada). Detailed information is available within the following MCHP concepts:
  • Macros for uniformly assigning records to RHAs {available internally}

Winnipeg

Conversion / crosswalk files (available internal only):

  • PCCF (Postal Code Conversion File) - to crosswalk between postal code and Statistic Canada census areas (including material added by MCHP). For some small area analyses, it may be necessary to access earlier year-specific versions of the POSTMUN file.
  • WRHA conversion table - to convert combinations of municipal code and postal code to Winnipeg neighbourhood areas (CCA, NC, neighbourhood).
  • ICD conversion files - to convert different versions of ICD codes into a usable component over time.

4. Time frame of study period

The choice of study years may be limited by the years for which data are available. (Note that most data bases at the Centre are organized by the fiscal year of April 1 to March 31.) Several issues are relevant with regard to defining boundaries for study years:

Fiscal year boundaries

Explicitly defining the years of study helps ensure that only records for the year of interest are present. For hospital discharge abstracts, for example, date of separation is typically used to define the fiscal year boundaries. Note that when comparing utilization by year, dates outside the fiscal year are normally removed for each year to ensure comparability. When years are combined for analyses, only the dates outside of the multi-year period are removed.

Completeness of data

Depending on the data base, fiscal year data may be incomplete, and will require going to a subsequent year to pick up late submissions: for fiscal 97/98 hospital abstracts, for example, both 97/98 and 98/99 hospital abstracts should be accessed to obtain all separation dates falling within the April 1, 1997 to March 31, 1998 period.

Records overlapping fiscal years

Hospital stays can span two fiscal years of data; for example, a stay of March 30/98 (admitted in FY 97/98) to April 3/98 (discharged in FY 98/99) will be picked up in the FY 98/99 hospital abstracts, not in 97/98. This has implications for length of stay.

Request information in an accessible format

If you require access to our resources in a different format, please contact us:

We strive to provide accommodations upon request in a reasonable timeframe.

Contact us

Manitoba Centre for Health Policy

Community Health Sciences, Max Rady College of Medicine
Rady Faculty of Health Sciences, 
Room 408-727 McDermot Ave.
University of Manitoba
Winnipeg, MB  R3E 3P5 Canada

info@cpe.umanitoba.ca
204-789-3819