5 Research Questions and Aims

5.1.1 DESCRIPTIVE

Every study uses an element of description. Let’s say you recruit a sample of 100 people who suffer from the same disorder and conduct a trial to estimate the effect of a new drug on some clinical outcome. When you summarize what you know about these 100 people at the time they were recruited, for instance the average age of the group, you’re describing the sample. Descriptive summaries appear in nearly every research article. But we can distinguish between the use of descriptive statistics—e.g., what is the mean age of these 100 people, the sample—and descriptive research questions.

One common descriptive research question in global health follows this format:

What percentage of women of reproductive age in Nepal use a modern method of contraception?

As we will discuss later in the book, you could answer this question by conducting a survey of contraceptive behavior with a representative sample of women in Nepal. That’s what the DHS Program did in 2010 (Ministry of Health and Population, New ERA, and ICF International Inc. 2011).

⊕“Modern methods” like condoms, implants, pills, etc, are distinguished from (and are more effective than) “traditional methods” such as withdrawal and the rhythm method.

Researchers surveyed a random sample of 10,826 households across the country and interviewed 12,674 women between the ages of 15 and 49 about their health behaviors and preferences. They estimated that 43.2% of married women reported using some modern method of contraception.

Figure 5.1: Current use of contraception by age in Nepal. Source: DHS Nepal 2011, https://tinyurl.com/y4u5wfkv.

Of course this is what they learned from the sample, but the research question required inference to the all women in Nepal in this demographic (the target population). As you’ll learn in Chapter 13, there is some error involved in speaking with some but not all women in Nepal, and the researchers estimated that the true percentage probably ranged from 41.0% to 45.3%.11 I’m being a bit fast and loose with the interpretation of this confidence (or uncertainty) interval, but I’ll make up for it later. This is an example of descriptive inference to answer a descriptive research question.

5.1.2 PREDICTIVE/RELATIONAL

⊕Of course not everyone needs to be using modern methods of contraception. If you’re not sexually active, you’re not at risk for pregnancy. Or if you’re trying to get pregant, modern methods will make that challenging. Therefore, public health officials wanting to promote modern method use would take this indicator and combine it with several others in the dateset to estimate the “unmet need” for family planning: women who say that they want to prevent or delay pregnancy, but are not using contraception.

Description is essential to science and decision-making related to needs and resources. The result from Nepal suggests that more than half of married women of reproductive age were not using a modern method of contraception in 2010. This is a very useful thing to know if you work for the Ministry of Health and are concerned about promoting reproductive health.

But you probably also want to go the next step and ask, “What predicts modern method use?” Stated differently, what factors are associated with/correlated with/related to modern method use? Who is most likely to use modern methods? What are the barriers to modern method use? These are questions about the strength and direction of the relationship between two or more variables and represent our second category of research questions.

Figure 5.2: Predicted probabilities of use of modern method of contraception. Source: Yours truly using data from the DHS Nepal 2011 survey, https://tinyurl.com/y4u5wfkv.

If you inspect the above figure from Nepal once again, you will see that the cross-tabulation of any modern method by age shows that modern method use is more common among older women compared to younger women. Modern method use appears to increase with age, according to this simple descriptive summary.

So we might want to ask, “To what extent does age predict self-reported modern method usage among currently married women of reproductive age in Nepal?” We can answer this question with a statistical model called logistic regression. The figure shows the predicted probabilities of modern method use by age among currently married women. As we expected, modern method use is more common among older women.

5.1.3 CAUSAL

Figure 5.3: Unfortunately, no one can be told what the Matrix is. You have to see it for yourself. This is your last chance. After this there is no turning back. You take the blue pill, the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes…

Questions about the relationship between X and Y make up the bulk of literature in global health. However, sometimes we want to go beyond asking to what degree X and Y are related and ask, “Does X cause Y?”, or “What is the causal effect of X on Y?” These are causal questions. They reflect our desire to know whether our treatments, policies, and programs promote behavior change and make people healthier. I’ll introduce you to this topic of causal inference in more detail in Chapter 8, but let’s consider for a moment the nature of a causal question.

To answer a causal question, we must think “what if”. If you want to determine whether a new drug helps people recover faster from a disease, a fundamental problem you’ll face is that you can’t give this new drug and the old drug to the same person simultaneously. Once this person goes down the new drug path, they can’t go down the old drug path (at the same time). So we have to create a situation where we can ask, “What would have happened if this person had gone down the old drug path instead?” This alternate path is known as the counterfactual, and we’ll discuss it much more depth as we go. For now, I’ll simply describe it as the path not taken.

⊕Research in low-income countries has estimated that nearly all women want to prevent or space their next pregnancy after they have just given birth, but 6 in 10 women do not start a method of family planning after delivering a baby. Precisely when women become fertile after childbirth varies and is influenced by factors like breastfeeding, but in most cases ovulation returns before family planning is started. This puts women at risk of getting pregnant again before they want to. One option is to have an IUD inserted immediately after delivery of the placenta or within the first month postpartum.

Consider another example from Nepal. Pradhan et al. (2019) looked at the path not taken for counseling women on the benefits of having an intra-uterine device (IUD) inserted following childbirth to prevent a new pregnancy. The authors used a stepped-wedge design that you’ll meet in Chapter 9 to estimate the causal effect of offering postpartum family planning counseling on the proportion of new moms who opt to have an IUD inserted. By this design, the counseling intervention was ‘turned on’ in six hospitals in a stepwise fashion. The three hospitals in Group 1 received the intervention first in a staggered start. Approximately six months later, the remaining hospitals began offering the counseling intervention, one after the other. This figure demonstrates that rates of uptake appeared to jump in each group after the intervention was introduced. Pradhan et al. (2019) estimated that the intervention increased IUD uptake by 4.4  percentage points [95%CI: 2.8–6.4 pp].

Figure 5.4: Trends in PPIUD uptake. Source: Pradhan et al. (2019).

On the surface, questions of causal inference (or counterfactual prediction if you prefer) resemble questions of prediction/relation.22 See this discussion on Andrew Gelman’s blog where he frames causal models as a special case of predictive models. For X to cause Y, it must be true that X is related to or associated with Y.

What distinguishes these questions is often the research design we use to find answers, so that is where we will spend our time. And it will be time well-spent. Causal inference is one of the most important topics for applied global health researchers. When it comes to making decisions about prevention and treatment and how to allocate resources, we want to know what works, why, and for whom.

5.3.1 WRITING A CONCEPT NOTE OR SPECIFIC AIMS PAGE

Ideas don’t sell themselves. Before you can change the world, you have to convince collaborators to join your team and funders to believe in the work. One of the best ways to strengthen your pitch is to develop a short concept note. If you are applying for NIH funds, concept notes are known as Specific Aims documents and are always limited to 1 page.

Figure 5.5: Anatomy of a specific aims page. Source: Inspired by Sneck (2015). To view a full resolution version of this figure, visit https://tinyurl.com/y5s35jo5.

Most people will tell you that it pays to follow a known format when writing your Specific Aims page. Grant writing is not creative writing. Your 1-page Specific Aims page is all that some reviewers will read, often while flying at 30,000 feet, cramped in economy with no space for their laptop. Or after a full day’s work and kid bedtimes. You get the picture. Make your reviewer’s job easy by drafting a tight 1-pager following some conventions.

Example specific aims page

Here is an example specific aims page from my work, broken down in a common 4-paragraph format. It’s not a perfect example, but along with others (see here) it might help to guide your work.

In this example, my opening paragraph tries to hook you as the reader, tells you what we know, the gaps in our knowledge, and why filling this gap is a critical need. The next paragraph gives an overview of what I propose to do in the context of my long-term goals. Then come two specific aims with supporting details, followed by a short payoff paragraph that tells people why it makes sense to fund this proposal.

Figure 5.6: Specific aims example from a study of caregiver readiness to disclose a child’s HIV status to the child (7R21HD076695-020).

5.4.1 WHERE DO HYPOTHESES COME FROM?

In the classical view of science, we develop hypotheses out of existing theory, an explanation of some aspect of our world that has been confirmed through repeated, falsifiable tests. We’ll discuss the role of theory in global health in the following chapter, so for now let it suffice to say that theories can be a good source of hypotheses.

⊕Pro tip: If you’re looking to explore or describe some phenomenon and are struggling to identify a testable hypothesis, you can probably stop trying. You might be operating in a hypothesis-generating framework, rather than a hypothesis-testing framework.

It’s also worth noting that a study can include elements of both qualitative and quantitive approaches. We call this mixed methods research, and it’s a topic we’ll cover in Chapter 15.

We can also begin with our observations of the world and generate hypotheses to test. This can be an informal process, or it can take the form of formal qualitative research. Whereas quantitative studies are often deductive—state a hypothesis, deduce the consequences that would be consistent with the hypothesis, and collect data to test this hypothesis—qualitative studies are more often inductive or exploratory. In qualitative research, we make observations about the world and use these observations to describe why or how the world appears to operate the way it does. From these observations we can create testable hypotheses for future studies that, if not rejected, can generate theory.

5.4.2 WHEN ARE HYPOTHESES DEVELOPED?

Figure 5.8: The hypothetico-deductive model of the scientific method is short-circuited by a range of questionable research practices (red). HARKing, or hypothesizing after results are known, involves generating a hypothesis from the data and then presenting it as a priori. Source: https://cos.io/rr/

This probably seems like a redundant question because I already listed the order of the scientific method, and I acknowledged that you probably learned this order first hand in primary school. But here’s the thing: scientists sometimes Hypothesize After the Results are Known. This is known as HARKing, and it’s a no no (Kerr 1998).

To be clear: it’s 100% OK to collect data, analyze the results, and generate new ideas (hypotheses) to test in future studies. That’s called exploratory analysis, and if you label it as such everyone is happy.

But it’s not OK—in fact it’s dishonest and a form of fraud—to look at your data, develop hypotheses that conform to the data, and pass your new ideas off as a priori hypotheses that you tested. That’s lying. And it’s not science. You can’t falsify a hypothesis that you create after looking at the data.

I’m going to refrain from hitting you with some depressive statistics about the state of science today. It’s only Chapter 5. I’m still working on building your enthusiasm for research. We’ll revisit research sins like HARKing in Chapter 18, along with strategies for preventing such bad behavior.

5.5.1 GRANT PROPOSAL

Figure 5.10: Specific aims page from the O’Meara lab. For a full resolution version, visit https://tinyurl.com/y2k6jsa4.

Once you have your core team in place, it’s time to get funding to make your idea a reality. In the O’Meara lab’s example, they developed a proposal in response to this Funding Opportunity Announcement (FOA) from the NIH:

The purpose of this dissemination and implementation research funding opportunity announcement (FOA) is to support innovative approaches to identifying, understanding, and overcoming barriers to the adoption, adaptation, integration, scale-up and sustainability of evidence-based interventions, tools, policies, and guidelines. Conversely, there may be a benefit in understanding circumstances that create a need to “de-implement” or reduce the use of strategies and procedures that are not evidence-based, have been prematurely widely adopted, or are harmful or wasteful.

⊕The NIH offers different sized research grants. The R03 is the smallest grant with $100,000 in direct costs over 2 years. This mechanism is good for pilot or feasibility studies. The R21 comes with up to $275,000 in direct costs over 2 years. Researchers apply for an R21 when they want to test a new idea with some preliminary evidence. The R01 is the largest grant with an unlimited budget for work to be carried out over 5 years.

They directed their proposal to the National Institute of Allergy and Infectious Diseases (NIAID), one of the many NIH institutes participating in this funding opportunity. The team proposed two aims as part of an R01 grant:

Aim 1: Estimate the effect of antimalarial subsidy level on ACT purchase and adherence to RDT results. In the first phase of the project, we will test the effect of a range of subsidy levels on the demand for ACTs, and also test how the response to subsidy varies with or without information from diagnostic testing.

Aim 2: Evaluate the public health impact of targeted antimalarials subsidies through scale-up. In the second phase, we will determine the community-wide effects of targeting the antimalarial subsidy in both a high and a low malaria transmission region of Kenya through a partnership between CHWs and the private retail sector.

In 2014 when this proposal was funded, the payline for new R01s at NIAID was the 13th percentile. That means the O’Meara lab’s proposal scored within the top 13% of all proposals submitted. Think of how crushed you would be to have a proposal sitting at the 14th percentile!

5.5.2 FORMATIVE WORK, AIM 1

The O’Meara lab conducted two studies as part of this project. The first was a formative study to determine the optimal subsidy level for malaria treatment. Let’s use the paper they published on this Aim 1 work to examine how research problems, questions, and hypotheses come together in the Introduction of a scientific manuscript. Take a moment and download this article in BMJ Global Health by O’Meara et al. (2016). I’ve highlighted the authors’ descriptions of the research problem, what is known about the problem, the research question, and the hypothesis.

Figure 5.11: Example Introduction section published in BMJ Global Health. Source: O’Meara et al. (2016)

The research problem is that we don’t know how to combine diagnostic testing for malaria with drug subsidy programs in order to expand access to first-line treatments while ensuring that the medication is not overused by individuals without malaria. As noted in the Abstract:

There is an urgent need to understand how to improve targeting of artemisinin combination therapy (ACT) to patients with confirmed malaria infection, including subsidised ACTs sold over-the-counter.

The authors framed the objective of the study as follows:

We sought to study the relationship between a subsidised diagnostic test and a conditionally subsidised treatment intervention in order to understand how ACTs can be targeted to malaria cases in the retail sector. We designed an individually randomised experiment to determine the effect of a conditional subsidy for ACT on the decision to be tested for malaria when the test is subsidised or not, and the subsequent effect of the test on drug purchasing decisions.

⊕While most scientific papers follow the pattern of describing the research problem and outlining the objectives of the paper, you will find that it’s relatively uncommon, at least in the health sciences, to explicitly state the research question as a question. If you look at recent issues of a journal like The Lancet Global Health, you’ll also find that many articles do not even state explicit hypotheses.

We can use PICO to construct the implied research question:

Among individuals (>1 year) in Bungoma County, Kenya, does offering a subsidy for ACT conditional on testing positive for malaria increase the uptake of malaria testing and the rational use of ACTs compared to no subsidy?

If you’re coming from a discipline like economics, you might wonder if I forgot to paste the full Introduction of O’Meara et al. (2016). I did not. This is it. It’s common in the health sciences to publish very brief Introduction sections.

For comparison, here’s an example Introduction from Blattman, Jamison, and Sheridan (2017) published in American Economic Review. The first thing you notice is the difference in length. The complete typeset paper is 42 pages, compared to only 10 pages for the above example published in BMJ Global Health. The AER article is gated, but if you download the pre-print and read the Introduction, you’ll also get a sense for how economists structure scientific papers differently.

Figure 5.12: Example Introduction section published in American Economic Review. Source: Blattman, Jamison, and Sheridan (2017)

5.5.3 PUBLIC HEALTH IMPACT, AIM 2

With the formative work completed, the O’Meara lab moved on to the second aim to evaluate the public health impact of targeted antimalarials subsidies through scale-up. They designed and conducted a stratified cluster-randomised controlled trial in Western Kenya with 32 communities assigned to get usual care or free malaria testing and a partially subsidized voucher to purchase ACTs if testing positive.

Study protocol

As we’ll discuss in Chapter 18, it is becoming more common to publish a study protocol prior to starting data collection (or at least prior to viewing the data) that outlines your research question, hypothesis, study design, data collection methods, and analysis plan. Download the O’Meara lab’s protocol (Laktabai et al. 2017) from BMJ Open and notice how a study protocol, like a grant, uses future tense to describe what will be done.

Figure 5.13: Aim 2 study protocol. Source: Laktabai et al. (2017)

Trial registration

Regulatory bodies require that drug trials be prospectively registered with a database like ClinicalTrials.gov. Registration of non-drug trials is not mandatory in the same way, but most journals require it as a condition of publication of clinical trials.44 According to the NIH, a clinical trial is defined as, “A research study in which one or more human subjects are prospectively assigned to one or more interventions (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes.” Go here to see if your study is a clinical trial. There are many registries to choose from, but ClinicalTrials.gov is the oldest and most widely used. This is where the O’Meara lab registered their study for Aim 2 (NCT02461628).

Figure 5.14: Aim 2 trial registration. Source: ClinicalTrials.gov, https://tinyurl.com/yxundeds.