Research 101: Generalizability
In the Hydrocephalus Research 101 blog series, we have discussed how clinical trials are designed and the strengths and weaknesses of these designs. One factor that can affect the usefulness of a study, regardless of the strength of the design, is its generalizability.
What is Generalizability?
Very simply, generalizability is a measure of how useful the results of a study are for a broader group of people or situations.
If the results of a study are broadly applicable to many different types of people or situations, the study is said to have good generalizability. If the results can only be applied to a very narrow population or in a very specific situation, the results have poor generalizability.
This concept is especially important for individuals living with hydrocephalus because there are so many different causes of hydrocephalus, comorbidities, and secondary complications. In many studies, the study participants represent a very specific group of hydrocephalus patients.
Examples are studies that include only X-linked hydrocephalus patients or patients with post-hemorrhagic hydrocephalus. These populations are unique in many ways and therefore, depending on the specifics of the study, the results may not apply to other patient groups. Likewise, a study looking at long-term cognitive outcomes in shunted patients may not be applicable to patients with an endoscopic third ventriculostomy (ETV).
What limits the generalizability of a research project and how can you tell if the results of a study apply to you?
Let’s work through a study to find out.
What are we studying?
We want to know if brisk walking lower blood sugar levels.
What is the study design?
It is easy to get male and female college students to participate in research studies. So, the researchers randomly assign students at the local college to Group A or Group B and take a baseline blood sugar measurement. Group A will briskly walk for 30 minutes every day for five weeks. Group B will be asked to continue their normal activities.
At the end of the experiment, researchers will measure blood sugar levels again. This is a well-designed prospective, randomized control trial.
The Results
At the end of the five weeks, researchers conclude that brisk walking significantly lowers blood sugar levels.
How Generalizable are these Results?
The generalizability of this study is limited by the characteristics of the study participants. Remember our study population was made up of male and female college students. Most college students are between the ages of 18-22 years old, so it may be reasonable to generalize the results to other young adults assuming a large sample size and diversity among participants.
Things get trickier, however, if one would like to generalize the results to senior citizens. There are a lot of factors that make college students different from seniors including differences in baseline activity, blood pressure, baseline blood sugar levels, and what each may consider a ‘brisk’ walking pace. These differences may limit the generalizability of the study to older populations, or, in other words, the results may not apply to older adults.
The specific intervention being tested can also limit how generalizable a study is. In this example, study participants were asked to walk ‘briskly.’ Would the results have been different if the participants had been asked to run, or swim, or lift weights?
The Take Away
When you read a research paper, pay close attention to:
- who was included in the study group
- who was excluded from the study group
- the details and timing of any interventions
Then decide if the characteristics of the group are similar to your specific experience. If your case is reflected in the study group characteristics, the study results are likely applicable to you. But remember, every case is unique and group averages may not mirror your experience.
Other entries in our Hydrocephalus Research 101 Blog Series with Dr. Koschnitzky:
Research 101: An Explanation of Clinical Trials Design
Research 101: Randomization
Research 101: The Importance of Sample Size
Research 101: Junction, Junction, What’s Your Function?