Research 101: Levels of Evidence in Hydrocephalus Clinical Research Studies

February 19, 2015

Clinical studies are often reported in the news and through other media outlets, but how do you know what to believe and what to take with a grain of salt? To help, a system has been developed to classify studies based on the strength of the evidence each presents. Studies classified as Level 1 have the highest degree of experimental validity while studies classified as Level 5 have the lowest degree of experimental validity. Experimental validity is a measure of how strong and useful the results of the experiment are.

Experimental validity can be divided into two parts. The first part is Internal Validity. If an experiment has good internal validity, it means that the experimental factor, say a drug, is very likely the cause of any differences seen between the control and experimental group. Internal validity increases when the experimental factor is randomly assigned to participants, when an appropriate control group is present, and when the number of participants is large (a large sample size). The second part is External Validity. If an experiment has good external validity, it means that the results of the experiment can be generalized to a larger population.

The strength of the experimental design is therefore largely reliant on 4 factors: Randomization, Control Groups, Sample Size, and Generalizability. The highest level study, a Randomized Control Trial (RCT), uses randomization to determine which treatment arm a participant enters. This helps ensure that the control group and experimental group(s) start off on an equal footing (i.e. there are no initial differences between the groups). It’s kind of like in science class, where there is only one variable in your experiments you can change (your independent variable) and everything else must be the same. In most cases RCT researchers will perform a power analysis to determine how many participants are needed in each arm of the study before the study begins. If a power analysis is not included in the research paper, it is difficult to know if the sample size obtained is large enough to show significant differences between groups. This is now a standard part of all grant submissions. Generalizability can differ between studies depending on the goal of the study. In some cases, it is important for the results to be generally applicable to the entire population (i.e. risk factors for heart disease). In other cases, such as in hydrocephalus, it may be enough for the study to only be generalized to other patients with the condition or a specific subpopulation within a condition (i.e. post-hemorrhagic hydrocephalus).

One of the lowest level of clinical studies is the Case Report. Case Reports contain an in-depth analysis or description of one patient. Therefore, there is no randomization, no control group, a small sample size, and limited generalizability. It is impossible to make solid conclusions about the effect of a drug or intervention or about a specific condition from a Case Report. However, Case Reports are not all bad. Case Reports often generate ideas that lead to hypothesis-driven research, and in the case of rare diseases, it may be extremely difficult to conduct a higher-level study.