Oversized headlines obscure tiny findings

Not all research studies are created equal and that’s on purpose. But the study type is a key component that should be used when assessing the importance and potential impact of research.

A study of two people is going to yield less widely applicable insights than a study of 2,000. So when you read the latest headline about a diet that can prevent cancer or a dye that can cause it, take a look at the kind of study it was to help gauge the importance of its findings. 

• Prospective studies observe a group of people — or compare different groups, or cohorts, over time — to look for relationships between lifestyle factors or environmental exposures and the development of conditions or diseases. Exposure information is collected at baseline, or the beginning of the study, before disease is diagnosed, and at various points over time to document changes in exposure that may correlate with the development of disease.
• Retrospective studies rely on study subjects who have already been diagnosed with a condition or disease, called cases, as well as a comparison group of subjects who do not have the condition or disease, called controls. Cases and controls are then asked about past exposures to tease out any associations between exposure and disease.

Here are several types of observational study designs:

• Cross-sectional studies examine the relationship between an exposure and a health outcome within a specific population at a single point in time. Such studies provide a snapshot of activity but often cannot determine which came first: the exposure or the disease.
• Case-only studies look for interactions between the effects of genetic and environmental factors on a disease using data collected solely from patients with the disease, who are known as cases.
• Case-control studies compare two groups: those with a disease or condition of interest (cases) and those who are similar in age and other demographic characteristics who do not have the condition or disease (controls). Researchers then compare the frequency of exposure prior to the disease diagnosis to one or more risk factors in both groups to help determine whether there’s a link between the past exposure and the development of the disease or condition being studied.
• Cohort studies typically look at large groups of people, or cohorts, to understand the causes of a disease or condition. They can be prospective or retrospective. Most often, information on exposures is collected at baseline and then individuals are followed up over time to determine who develops the disease or condition of interest. A well-known example of a prospective cohort study is the Framingham Heart Study, which was launched in 1948 with an aim to identify factors that contribute to cardiovascular disease. The study, which has involved more than 14,000 participants from three generations, found that high blood pressure and cholesterol are major risk factors for cardiovascular disease, among other key results.

Experimental studies

Experimental studies, also called randomized, controlled trials, or RCTs, assign groups of people at random, like the toss of a coin, to either receive (or not receive) a preventive or therapeutic intervention or screening test, such as a drug, device or procedure. Researchers then assess the impact of the intervention on those who received it as compared to those who did not.

Well-known examples of RCTs include those conducted by the Women’s Health Initiative, a massive nationwide undertaking coordinated by Fred Hutchinson Cancer Research Center. Launched in 1991 with a $625 million grant from the National Heart, Lung, and Blood Institute, the WHI conducted four randomized, controlled Phase 3 trials (see below re: trial phases) that were designed to measure the effects of various interventions, from hormone therapy to dietary supplements to a low-fat diet, on preventing the most common health issues in postmenopausal women, including heart disease, breast and colorectal cancer, and osteoporosis-related fractures. The initiative, which enrolled more than 161,000 older women across the U.S., had — and continues to have — a wide-ranging impact on medical practice and women’s health.

RCTs are designed to evaluate the safety and effectiveness of experimental drugs or tests for diseases such as cancer. They are conducted incrementally, in a series of steps that are called phases. Below are brief descriptions of the study phases.

• Phase 1 trials are first-in-human studies that are conducted in small groups of people to evaluate a new drug, test or procedure. This can include determining the safe dosage of a drug, the most effective way to administer it and identifying any serious side effects.
• Phase 2 trials involve slightly larger groups of participants to test efficacy and further test safety.
• Phase 3 trials are conducted in a large study population to provide a complete picture of safety and efficacy. These trials aim to determine whether the drug, test or procedure is as good as the current standard of care for a disease or condition. Such studies are typically designed as randomized, controlled trials.
• Phase 4 trials are conducted in large populations after a drug, test or procedure has been introduced on the market to further monitor safety and potential long-term adverse outcomes. Such post-marketing trials are typically designed as observational, long-term cohort studies.

Causation vs. correlation

When reading about study results, it is important to not confuse causation with correlation or association. While prospective, observational studies that follow large groups of people over time can suggest relationships between certain exposures and diseases, they cannot on their own prove cause and effect. Only randomized, controlled trials comparing intervention and control groups — the research gold standard — can definitively confirm causation.

The bottom line: Always interpret health headlines — particularly nutrition claims — with a proverbial grain of salt, because if they seem too good to be true, they probably are.