What Makes a Good Yoga Study? Some of the Basics

People often ask me what makes a good yoga study. The short answer is, it’s complicated. If good yoga research were simple, we’d all be doing it. Here are the fundamentals that all yoga teachers, students and yoga therapists should know.

The Golden Rule – Correlation Does Not Imply Causation

There is one golden rule in research, and that includes yoga studies: Correlation does not imply causation. No matter how much we would like to think that the programs we develop cause someone or something to change, we can’t make that assumption even in the most carefully controlled of studies.

Why? Because humans are complex. We simply can’t statistically rule out the myriad contextual, biologic, psychosocial and other factors that may explain the effects we detect. That’s not to say that you should curb your enthusiasm. It’s a suggestion to take research results for what they are – data from one study at one point in time. And don’t assume that one thing ‘caused’ another, even if it looks to be the case. Period.

In addition to the Golden Rule, there are a number of fundamental terms and concepts that are important to understand, particularly in regards to the basic research design of studies and what that implies about the conclusions we can draw.

Research Designs: The Basic Blueprint of Data Collection

A research design is essentially a systematic blueprint that guides the collection and analysis of data. There are two primary types of designs – descriptive and experimental. Descriptive research focuses on how something works, whereas experimental research emphasizes what works.

Descriptive Research Designs. Descriptive research includes ethnographic studies and case examples among others. Studies often focus on understanding how a particular program is implemented and how participants perceive their experiences and outcomes. These studies typically make use of qualitative information, such as journal entries, written responses, interviews and focus groups.

There are also quantitative descriptive studies. These often use numeric data to examine the number and type of people who participated in a program. Data may be used to loosely interpret whether an individual’s initial scores differ from his or her final scores after an intervention.  

Descriptive research is often used when researchers are studying a new program and its benefits and are less concerned with statistical results. Experimental studies, on the other hand, systematically test the relationship between a specific program or programs and a predetermined set of outcomes.

Experimental Research Designs I: Randomized Controlled Trials

Experimental studies are used to systematically test the effects of a particular program. The “gold standard” of experimental designs is the randomized controlled trial (RCT). Using this approach participants have an equal likelihood of being randomly assigned to either a treatment group or a control group.

Treatment and control group participants are matched on a number of dimensions (e.g. age, sex, health status, years of practice) to make sure that the groups are relatively equivalent to each other. The purpose of random assignment is to attempt to control for extraneous factors, or covariates (e.g. age, sex, health status) that may unduly bias the results of a study.

Most experimental studies divide participants into two or more groups. The treatment group (often called the experimental group) refers to the collection of participants who receive the primary intervention (i.e. yoga program). Studies may have one or more of these groups, depending on the research objectives, however yoga researchers are typically interested in comparing a particular yoga program to a “no yoga” control condition.

The control group refers to those who are not invited to participate in the active condition under investigation (e.g. yoga program) as part of a study. While many assume that control groups are passive and do not receive any type of intervention, this is often not the case. Many studies use “active” controls.

A good example is a study comparing a yoga group to both an exercise condition and to a no treatment condition. In this case the yoga group receives the treatment, the exercise group is an active control group that receives an exercise intervention, and the no treatment group receives no special instruction. Both the exercise and no treatment groups are considered control groups in this case. The use of an exercise group as a control allows researchers to assess whether differences between groups are related to exercise in and of itself, or the act of practicing yoga.

Randomized studies can use either a ‘blind’ or ‘double blind’ approach. In double blind studies, neither researchers nor participants know which condition (group) a participant has been assigned to. As you can imagine, it is particularly difficult to conduct double blind yoga research, as those engaged in practices that resemble yoga are likely to assume that they have been assigned to a yoga condition. It is much easier for an experimenter to have no idea as to group assignment, unless the experimenter is both the yoga instructor and the researcher examining the data. This is not an ideal situation, and should be avoided when at all possible.

Experimental Research Designs II: Quasi-Experimental Designs

Quasi-experimental designs are similar to randomized controlled trials in that numeric (quantitative) differences between intervention and control groups are emphasized. Unlike RCTs however, these studies often use “convenience samples” or volunteers. In this approach, experimental groups often receive the yoga program as part of the formal study, and controls are placed on a waitlist and offered the program shortly after the formal research is completed.

Participants in these studies are not randomly assigned to either a treatment or control group. This means that the two groups may differ greatly on one or more key dimensions (e.g. age, sex, prior yoga experience, health status), which may significantly impact group differences and statistical outcomes. It is critical to examine whether the treatment and control groups differ at baseline when interpreting the results of these studies.

Studies of Studies: Meta-analyses and Systematic Reviews

Meta-analyses and systematic reviews are also considered to be research studies. These approaches systematically examine the existing research to ascertain the overall impact of published or reported interventions for a particular outcome (e.g. osteoarthritis). Such analyses of the literature help to identify what may work for individuals with a particular condition, and also evaluate the overall impact and quality of a body of research with the intention of making broad assumptions and recommendations.

At present there are a number of critical issues that make meta-analyses and systematic reviews of yoga research particularly difficult. These include a lack of consistency among yoga programs and their approaches, high variability in program duration and length, inconsistency of measurement strategies, and weak research designs (e.g. non-randomized samples), among others.

Testing Study Hypotheses: Statistical Analysis of the Data Collected

Good experimental studies are organized around a priori hypotheses. These hypotheses delineate the direction of specific outcomes that are anticipated as a function of participating in an intervention. The strategies for measuring these outcomes are also determined prior to the onset of research.

For example, a study may be designed to test the effects of a yoga program on stress. Researchers hypothesize that yoga group participants will exhibit lower levels of stress after completing an 8-week yoga program compared to a matched (age and sex) control group of sedentary participants. They measure all participants before and after the 8-week program on a number of conventional measures of stress including blood pressure, salivary cortisol, and participant’s self-reported stress ratings. They then compare the yoga and control group pre- and post-intervention data to test for differences.

These data are quantitative, meaning that numeric values are used to represent participant stress. The values are statistically analyzed based on the study’s a priori hypotheses. In very simplistic terms, these analyses test to see whether the yoga and control group values before and after the 8-week yoga program are statistically different at a level above chance. This means that differences between the two groups may be related to participation in the yoga program and not just an artifact of random variation.

Other Key Research Terms You Should Know

There are a number of other key terms that are typically used in research studies. It is important to understand what they mean so that you can use them properly.

Independent variable – An independent variable refers to what is manipulated. It is often considered the factor that will be associated with change. For example, in yoga research, the independent variable is often the yoga program, which “manipulates” participants to behave differently than usual.

Dependent variable – Dependent variables refer to the specific outcomes of interest in a study. These typically consist of outcomes related to the independent variable. For example, in a study of the effects of yoga for high blood pressure, the independent variable is the yoga program and the dependent variable might be systolic and diastolic blood pressure levels.

Most studies include multiple dependent variables. For example, we might hypothesize that 5 minutes of deep breathing a day (independent variable) would be associated with decreased heart rate, blood pressure, and subjective ratings of stress (3 dependent variables). Note that this assumes a linkage, not causality.

Validity and Reliability

Validity and reliability are the cornerstones of scientific research. Validity refers to whether or not something is factually sound. There are many different types of validity (internal, external, criterion, discriminant, convergent, predictive, concurrent, test-retest, and others), many of which relate to the statistical properties of the measures used and their relationships.

Validity: There are 3 types of validity that are conceptually critical for yoga research whether the studies are quantitative or qualitative (descriptive) in nature.

Internal validity refers to the integrity of a scientific study. To have strong internal validity in yoga research we need to be able to assume that the effects of a program are attributable to the program itself and not other factors or causes.

External validity refers to whether or not the results of a particular study can be generalized to other situations and sample of people. Even though a single study may demonstrate the benefits of a yoga program, those benefits may be an artifact of the sample, teacher or situation. It is essential that programs and their effects be replicated with different populations and by other researchers in order to establish the program’s external validity.

Ecological validity refers to the extent to which an intervention or program and its results can be applied in real-world settings. In the case of yoga research, some programs are delivered in laboratories, intensive retreats, or other contexts that are not representative of the places where most people are exposed to yoga. In these cases, it is essential that these programs be effectively delivered and successful in settings that are accessible to the population for which they are designed.

Reliability refers to whether or not the results of a study can be replicated. This means that a program is delivered in exactly the same way, using the identical measures and research design, and that the statistical results are identical (or very similar to) the original study. Programs and results with poor reliability often suffer from poor internal, external and ecological validity.

These are just a few of the key terms and ideas that we should be mindful of when evaluating and participating in research. In this emerging field it is essential that we form multidisciplinary alliances between researchers and yoga teachers and therapists so that we can share our knowledge in the service of creating a strong evidence base for the use of yoga as therapy.

Although yoga research has grown exponentially in recent years, we are only now beginning to see studies that consistently make use of rigorous research designs and methods. Part of this is due to the fact that yoga research has historically been poorly funded, which makes sophisticated designs and methods very difficult to execute.

It is essential that the field continues to address these issues and others including participant attrition, the consistency of approach, measures, and methods, and more precise description of the yoga technologies used. Even though the field of yoga research has a long way to go, the future is promising. 

B Grace Bullock, PhD, E-RYT, is the former Editor-in-Chief of the International Journal of Yoga Therapy. She is an author, intervention scientist and practitioner who has worked extensively in inpatient and outpatient behavioral health settings. Her research and clinical work explore the effects of integrating empirically supported psychotherapy with yoga therapy to relieve stress, anxiety, depression and other psychological illnesses, and to promote health and wellbeing for children and their families. She was the recipient of a Francisco J. Varela Research Award from the Mind & Life Institute. For more information contact Grace at bgracebullock@me.com or see http://www.mind-bodytherapy.com.