Distinguishing Immunologic and Behavioral Effects of Vaccination

Pharmaceutical interventions are commonly justified by their immunologic mechanisms of action, but they might also affect outcomes through other pathways. For example, recipients of vaccines have been reported to increase their number of social contacts due to perceived protective effects,1–3 although the extent of such behavioral changes varies across populations and time.4–7

Conventional vaccine trials are designed to identify the immunologic effects of vaccines.8 These trials often have blinded treatment and control groups9,10 and the rationale for (patient) blinding is precisely to eliminate the nonimmunologic effects of vaccination. Indeed, an ideal placebo control satisfies two criteria: it does not have any cross-reactivity with the pathogen in question and it is perceived to be indistinguishable from the vaccine, for example, by inducing common vaccine side effects such as fever or soreness in the place of injection. The second criterion is challenging to satisfy in many vaccine trials where inert saline vaccines are used as controls.10

The reliability of placebo controls has been studied in so-called unblinding assessments or manipulation checks,11,12 where trial participants are asked to guess the treatment they received. Differences in guesses between assignment groups indicate that the placebo was unsuccessful. Such differences have been observed in trials assessing appetite suppressive treatments,13 smoking cessation strategies,14 psychiatric drugs,15,16 back pain treatments,17 and other interventions.18,19

However, unblinding might be a consequence of the treatment being effective. If the treatment is noticeably beneficial and individuals are asked to guess their treatment group after the effect becomes evident, then these individuals might correctly guess their treatment status. Because unblinding assessments are difficult, the mandatory reporting of blinding success was revoked in the 2001 CONSORT guidelines.20,21 Thereafter, assessment of blinding in randomized controlled trials (RCTs) has become less frequent.16,18,19,22,23 In particular, we could not find examples of blinding assessment in vaccine studies.

In this article, we claim that an assessment of treatment beliefs, similar to unblinding assessments, is desirable in vaccine trials because this assessment can allow us to identify policy-relevant estimands. First, we formally study causal effects targeted by vaccine trials, scrutinizing their practical relevance. Even under perfect blinding and no interference, the conventional vaccine efficacy estimated from a trial might not be representative of real-world vaccine effectiveness. In addition, broken blinding challenges the interpretation of common vaccine estimands as parameters quantifying “immunologic efficacy.” Second, we describe how different, but related, estimands can be identified and estimated from conventional vaccine trials under testable, and often plausible, assumptions when a blinding assessment is conducted.

PRELIMINARIES

Consider data from a blinded RCT with n individuals who are assigned treatment A∈ at baseline, where A=1 indicates receiving vaccine and A=0 indicates placebo or other control. Together with A, we explicitly define the messageM∈, indicating whether the individual receives the message that the vaccine status is blinded (M=−1), that they are unvaccinated (M=0) or that they are vaccinated against the pathogen of interest (M=1). Unlike a real-world setting, where we would expect that A=M with probability one (w.p.1), blinding in an RCT implies that the message M is fixed to −1. Furthermore, let L be a vector of measured covariates, which might affect the outcome Y. We treat L as discrete for simplicity of presentation, but all results hold for continuous L by replacing probabilities with probability density functions and sums with appropriate integrals. Let S∈ be an indicator of a possible side effect, and let B∈[0,1] be a variable quantifying the degree to which an individual believes that they have received the vaccine, where B=1 corresponds to being convinced about having received the active vaccine and B=0 being convinced about having received control; here we would expect that the belief depends on the type of control, for example depending on whether the control is simply no treatment or an inert (placebo) vaccine. Finally, let E be a variable quantifying how much an individual has been exposed to the infectious agent. We do not assume that E is measured and leave the domain of E arbitrary. In addition, we will occasionally introduce an unmeasured variable U as a common cause of at least two of the previously introduced variables.

As assumed in most analyses of vaccine RCTs, suppose that the trial participants are drawn from a near-infinite super-population where interactions among participants are negligible. Therefore, we omit the i subscript from random variables. The assumption that interactions between participants in the trial and the population are negligible with respect to the outcome of interest implies an assumption about no interfer-ence, so all potential outcomes we subsequently present are well defined. However, our arguments also apply to certain settings where interference is present, as discussed in eAppendix A; https://links.lww.com/EDE/C95.

We use superscripts to denote counterfactuals. In particular, let Ba,m be an individual’s belief about their vaccine status when the treatment and the message are set to a and m, respectively. When there is no blinding, for example, after the vaccine has been made available for the entire population and A=M w.p.1, we would expect that Ba,m=m. Let also Ea,m quantify the exposure of the study participant when treatment is fixed to a and the message to m. As with E, the domain of Ea,m is left arbitrary. Hence, our results do not depend on the variable type (e.g. binary, count or continuous) assumed for Ea,m. If receiving the vaccine can cause a side effect shortly after vaccination, say within 7 days,10 we further define Sa to be the indicator that the participant experienced this side effect. Let Ya be the disease status some fixed time (e.g. 1 month) after an individual was assigned treatment level A=a, which is measured without misclassification. Finally, let Ya,m be the outcome had treatment level was fixed to A=a and the message to M=m. Henceforth we assume consistency with respect to all counterfactuals defined and corresponding observed data, for example, if A=a then Y=Ya.

CAUSAL EFFECTS AND TARGET TRIALS Conventional Two-arm Trial

Consider first the average treatment effect of being vaccinated (A) on the clinical outcome (Y), when the treatment allocation is fixed to be blinded (m=−1),

E(Ya=1,m=−1)vs.E(Ya=0,m=−1).

This effect is identified by design in a conventional, blinded, two-arm vaccine trial, henceforth denoted by TII. We deliberately set m=−1 as part of the intervention indicated in the superscripts, because blinding is a crucial feature of the intervention assessed in TII. While studies of vaccine effects usually state estimands of the form E(Ya=1) and E(Ya=0), without indicating the message M, we make this distinction to clarify that other, subtly different estimands can be important for policy decisions. Our variable definitions are related to, but different from, the definitions given by Murray,24 who explicitly formulated a counterfactual definition of a per-protocol placebo effect, see eAppendix B; https://links.lww.com/EDE/C95.

Because conventional vaccine trials enforce m=−1, such trials are, at least implicitly, targeting an immunologic vaccine effect: the intention of blinding is to eliminate certain (psychological and behavioral) effects of receiving the vaccine. Suppose now that we offer the vaccine to individuals in a real-world setting, outside the trial. Even if the trial and real-world settings share similar conditions, for example, individuals are drawn from the same super-population, and the transmission rates are equal in both settings, the effect of the vaccine in the real-world setting might differ from the effect in the RCT. Individuals in the real-world setting are, unlike the vaccinated trial participants, informed about the treatment they received (M=A w.p.1). In particular, a vaccinated individual knows that they received a vaccine (B=1) and this knowledge can lead to changes in their behavior. For example, the vaccinated individual might reduce their protective behaviors, and thus increase their risk of being exposed.

Because people might change their behavior after vaccination, the total effect,

E(Ya=1,m=1)v.s.E(Ya=0,m=0),

which quantifies the joint effect of receiving both the vaccine ingredient and the message, is a relevant parameter for policy makers when deciding vaccine policies. This effect is different from the effect given in (1). That is, the effect in (1) is analogous to the effect targeted in a successfully blinded experiment, where the intention might be to eliminate placebo effects by fixing m=−1; the effect in (2) is the effect in an unblinded trial and captures different pathways by which vaccination can affect the outcome of interest. For example, if vaccination leads to reduced use of protective measures, the knowledge of being vaccinated might counteract the protective immunologic effect of the vaccine.

However, the effect in (2) is not identifiable from the data in TII without additional assumptions. We will therefore consider hypothetical trials that allow identification of such effects, including a feasible, minor modification of TII.

Hypothetical Four-arm Trial

Consider a four-arm trial where each individual is assigned the blinded treatment A∈, and then immediately given a message M∈, stating that they, possibly contrary to fact, received the control (M=0) or active vaccine (M=1). In this trial, henceforth denoted by TIV, it is possible that the message (M) is the opposite of the actual treatment assignment (A), that is,Pr(M≠A)>0. By design, in TIV we identify

E(Ya,m)v.s.E(Ya′,m′),

fora,a′,m,m′∈

Such a trial design, also known as a “balanced placebo design,”25 has been implemented to examine the effects of nausea treatments,26 nicotine and alcohol,25 and caffeine.27 To the best of our knowledge, this design has never been implemented to study vaccine effects. Conducting such a vaccine trial is ethically problematic because the participants are given misleading information about vaccination status that might, for example, affect their risk through behavior.25 Even if the trial is practically infeasible, we can still conceptualize a study that jointly assigns A and M at random, which would allow us to separate the immunologic and behavioral effects of receiving the vaccine. For example, the contrast

E(Ya=1,m)v.s.E(Ya=0,m)

is, like the contrast in (1), expected to quantify an immunologic effect of receiving the vaccine, because individuals in both arms are told that they have the same vaccination status, m∈.

On the other hand, the contrast

E(Ya,m=1)v.s.E(Ya,m=0)

quantifies a behavioral effect of the vaccine, in the sense that both groups receive the same biological vaccine component (a), but one of the groups is told that they, contrary to fact, did not. Thus, this contrast quantifies how knowledge (belief) of being vaccinated changes the outcome, for example, through risk-increasing behavior. Furthermore, the total effect, (2), would be identified from TIV without additional assumptions.

Because of the ethical and logistical issues with conducting TIV, we present conditions ensuring that we can use data from TII to identify and interpret (3) and (4) as immunologic and behavioral effects, respectively.

Identification Based on Relations Between the Two-arm and Four-arm Trials

To relate the outcomes in TII and TIV, consider the belief B, quantifying the degree to which an individual believes that they received active vaccine (higher values of B) or control (lower values of B). In particular, B=1 means that the individual is convinced that they were vaccinated. While the results we present in this work are applicable for continuous B∈[0,1], to simplify the notation we henceforth focus on a binary belief, B∈.

If the four-arm trial TIV is successful, we would expect that the message M deterministically causes the belief B.

Assumption 1B=Mw.p.1. whereM∈.

In TII, individuals receive no message, which we define as fixing m=−1, but they might still form beliefs about the treatment received. When the belief B affects the risk of exposure E, the counterfactual quantity identified in TII, E(Ya=1,m=−1), would be less relevant to the outcomes in the setting where people know their vaccination status, as is usually the case in practice.

However, it is feasible to measure the belief B in TII by asking whether an individual believes they received active vaccine or placebo. We denote by TIIB the two-arm trial where B is also measured. By introducing the belief variable B, we can formalize the notion that receiving the vaccine affects our risk of infectious disease outcomes both through the immunologic effect of a vaccine and through behavior.

Suppose first that the belief determinism holds in the four-arm trial TIV, that is B=M w.p.1. Consider now the six-arm trial which incorporates the arms of the two- and four-arm trials introduced so far: let TVI be the trial where A∈ and M∈ are randomly assigned jointly, but independently of each other. Suppose further that the message M only affects Y through the belief, which we explicitly state as an isolation condition.

Assumption 2 (M partial isolation) The only causal paths from M to Y are directed paths intersected by B.

In our setting, Assumption 2 requires that the external message about the treatment status only affects the outcome Y through our belief about the treatment status. Assumption 2 seems to be plausible in practice; to violate this assumption, the message M must affect Y outside of the belief B, which will be contrived in many settings. For example, Assumption 2 holds in the directed acyclic graph (DAG) in Figure 1C.

FIGURE 1.: Directed acyclic graphs (DAGs) describing

TII

,

TIV

, and

TVI

. A, DAG describing a two-arm trial

TII

, where

M

is deterministically equal to

−1

. Thus, the node

M=−1

is a trivial constant and is only included in the graph for clarity. B, DAG describing the (hypothetical) four-arm trial

TIV

, where the bold arrow indicates the assumed determinism between the message

M

and the belief

B

. C, DAG describing the hypothetical six-arm trial

TVI

, where

A∈

and

M∈

are randomly assigned.

Consider also the following assumption, inspired by previous work on separable effects.28–30

Assumption 3 (Y Dismissible component condition)

Y⊥⊥VIM|L,A,B,

where ⊥⊥VI denotes independence in TVI.

We can directly read off that this assumption holds with L=∅ in the DAG in Figure 1C, describing a six-arm trial TVI. In Figure 1C, the node E is not needed to evaluate our identification conditions, but including E clarifies that M only affects Y through exposure to the infectious agent,E.

Assumption 3 would be expected to fail, except in some special cases with perfect cancelations, whenever Assumption 2 fails. However, Assumption 3 can also fail when Assumption 2 holds, for example, when there are unmeasured common causes of B and Y, as illustrated by the path B←U→Y in Figure 2.