Observational and Experimental Studies

Elementary Statistics

MTH-161D | Spring 2025 | University of Portland

January 31, 2025

Objectives

These slides are derived from Diez et al. (2012).

Previously… (1/3)

Types of Variables

Types of Variables

Previously… (2/3)

Relationship Between Variables

\[\text{explanatory variable} \xrightarrow{\text{might affect}} \text{response variable}\]

Associated vs Independent Variables

Types of sampling

Previously… (3/3)

The guiding principle of statistics is statistical thinking.

Statistical Thinking in the Data Science Life Cycle

Statistical Thinking in the Data Science Life Cycle

Types of Studies

Observational Experimental
Researchers observe subjects without interference. Researchers intervene by applying treatments to subjects.
No treatment or manipulation is imposed. Includes a control and treatment groups with random assignments (ideally).
Used to find associations, not causation. Can determine causal relationships.

\(\star\) Key Difference: Observational studies find patterns, while experimental studies test cause-and-effect.

Case Study 1

Is there a relationship between smoking and lung cancer?

Study Design:

Findings:

\(\star\) Since this is observational, it cannot prove smoking causes lung cancer –other factors (e.g., genetics, pollution) may also contribute. However, strong correlations from multiple studies can strengthen this conclusion.

Types of Observational Studies

Aspect Case-Control Cohort (Longitudinal) Cross-Sectional
Study Design Compares individuals with a condition (cases) to those without (controls). Follows groups of individuals over time, observing exposures and outcomes. Measures a population at a single point in time, observing various variables.
Main Focus Identifying exposures or risk factors associated with an outcome. Observing how exposures lead to outcomes over time. Examining the prevalence of variables or conditions at a given time.
Temporal Sequence Retrospective –looks back in time to find past exposures. Prospective –follows participants forward in time. No temporal sequence – snapshot of a population at a single time point.
Data Collection Collects past data (often using medical records or interviews). Collects data over time, often requiring repeated observations or surveys. Collects data at one point in time.

\(\star\) Key Differences: Case-Control looks at data in the past, Cohort follows the data, and Cross-Sectional looks at data at one point in time.

Strengths and Limitations of Observational Studies

Aspect Case-Control Cohort (Longitudinal) Cross-Sectional
Strengths Good for studying rare diseases, cost-effective, relatively quick. Can establish temporal relationships, good for studying causes and effects. Quick, inexpensive, good for identifying associations.
Limitations Cannot establish causality, relies on recall bias. Expensive, time-consuming, and prone to participant attrition. Cannot determine causality, only associations.

\(\star\) Key Similarities: The limitation of observational studies is that it can not determine causality, only associations.

Prospective vs Retrospective Observational Studies

Study Type Description Strengths Limitations
Prospective Study Researchers follow subjects forward in time, starting with an exposure and observing future outcomes. Can establish a temporal relationship between exposure and outcome, reduces recall bias. Expensive, time-consuming, potential participant dropout.
Retrospective Study Researchers analyze past data, identifying subjects with an outcome and looking back to determine exposure. Quick, cost-effective, useful for rare diseases or long-term effects. Prone to recall bias, missing or incomplete data, cannot establish causality.

\(\star\) Key Differences: Prospective means present and future data and retrospective means the past data.

Case Study II

Is there a relationship between hypertension and stroke incidence in an older population?

Study Design:

Findings:

\(\star\) This is an example of a retrospective cohort Study because the data is in the past and the design involves groups.

Case Study III

Does energy gels make a person run faster?

Study Design:

Findings:

\(\star\) This is an example of an experimental study because the design involves an intervention, which is the treatment group (with intervention) and compared it to the control group (without intervention).

Case Study III: Blocking

Does energy gels make a person run faster? Since it is suspected that energy gels might affect pro and amateur athletes differently, we block for pro status.

Study Design:

Findings:

\(\dagger\) Why is is blocking important? Can you think of other variables to block for?

\(\star\) Since this is an experimental study, we can conclude a causal relationship between use of energy gels and faster running.

Principles of Experimental Design

Principle Description
Control Compare treatment of interest to a control group.
Randomize Randomly assign subjects to treatments, and randomly sample from the population whenever possible.
Replicate Within a study, replicate by collecting a sufficiently large sample. Or replicate the entire study.
Block If there are variables that are known or suspected to affect the response variable, first group subjects into blocks based on these variables, and then randomize cases within each block to treatment groups.

\(\star\) Key Idea: Experimental studies establish a cause-and-effect relationship by manipulating independent variables and observing their impact on dependent variables while controlling for confounding factors.

Blocking vs Explanatory Variables

Aspect Blocking Explanatory
Definition Characteristics that experimental units come with and that we want to control for. Variables that we manipulate or observe to explain the outcome of the experiment.
Purpose Used to reduce variability by grouping experimental units with similar traits. Used to explore or test the effect of a treatment or intervention on outcomes.
Role in Experiment Serve as a way to control for potential confounders and reduce bias. Act as the independent variable(s) whose effect on the dependent variable is tested.
Timing in Experiment Applied before random assignment to ensure balanced groups. Manipulated or measured during the experiment to observe their effect.

\(\star\) Key Idea: Explanatory variables are factors tested for their impact, while blocking groups subjects to reduce confounding effects.

More Experimental Design Terminology

Random Assignment vs Random Sampling

Activity: Identify the Type of Study

  1. Make sure you have a copy of the F 1/31 Worksheet. This will be handed out physically and it is also digitally available on Moodle.
  2. Work on your worksheet by yourself for 10 minutes. Please read the instructions carefully. Ask questions if anything need clarifications.
  3. Get together with another student.
  4. Discuss your results.
  5. Submit your worksheet on Moodle as a .pdf file.

References

Diez, D. M., Barr, C. D., & Çetinkaya-Rundel, M. (2012). OpenIntro statistics (4th ed.). OpenIntro. https://www.openintro.org/book/os/