We selected an initial cohort of PLWH (i.e., one inpatient or two outpatient encounters for HIV during fiscal years (FY) 2011 to 2017). Fiscal years begin on October 1 and end on September 30 the following year (e.g., FY11 is October 1, 2010, to September 30, 2011). We then identified those who had at least registered to use MHV between October 1, 2012, and April 1, 2017. During this period, two types of MHV accounts were available to all patients, an Advanced account and a Premium account. When they registered, they automatically received an Advanced account which provided access to Rx refill. To gain access to the other tools examined in this study, MHV registrants had to authenticate their identity, through in-person authentication (IPA), which some patients did not do. Due to variation in access to MHV tools depending on account type, we utilized different subsets of our cohort of MHV registrants to evaluate the impacts of Rx refill on outcomes versus the impacts of all other tools. Additionally, our observation of MHV engagement was restricted to FY13 and later; FY13 was the first FY that MHV data were available for research.

For analyses examining Rx refill, we included all MHV registrants who registered for MHV between the aforementioned time-period (i.e., “refill access cohort”). For analyses examining all other tools, we included all MHV registrants who IPAed during the same time-period (i.e., “full access cohort”). By defining these cohorts according to these dates, we initiate our observation of patients at the time when the MHV tool became accessible to them. We therefore reduce selection bias by adjusting for baseline measurements taken immediately prior to their choice to gain access to the MHV tool. These groups are not mutually exclusive, nor is one group a direct subset of the other. Also, given the longitudinal structure of our study, a patient could be included in one or both cohorts for different lengths of time. For example, a PLWH who registered for MHV on October 13, 2012 and IPAed on August 5, 2014 would belong to only the refill access cohort for several intervals but would belong to both cohorts for all intervals following their IPA date.

Data were obtained primarily from VA electronic health record (Corporate Data Warehouse). Additional sources of data were used to measure housing status and geographic characteristics, and they are described in greater detail in prior work.13 We observed patients from their MHV registration (or IPA) date until September 30, 2018, or until their death if before the end of the study period. Institutional Review Board (IRB) approval was obtained from the Stanford University IRB.

Data Structure

The impacts of MHV tool use on outcomes were assessed in 6-month intervals, according to the VA FY. Whereas we began observing PLWHs on their registration date (or IPA date), patient-time did not contribute to estimation of the outcome models until we had observed (1) required baseline measurements in the six months prior to their registration date, (2) at least two full intervals of MHV registrant status (required to obtain the values for MHV tool use and 6-month lagged MHV tool use), and (3) the last full intervals of outcome observation occurred after the interval in which MHV tool use was measured. The figure in Appendix 1 depicts an example of the temporal order in which covariates and outcomes were measured using the example of outcome measurement in the second half of FY14.

Exposures

The following indicators were examined in a given 6-month interval as the exposures of interest: (1) any Rx refill use, (2) Rx Refill use to refill an ART, (3) secure messaging use, (4) view appointments use, and (5) view labs use. For each interval, we designate these indicators as 1 if a PLWH used the tool at least once, and as 0 if they did not use the tool.

Time-invariant Covariates

All of these covariates were measured before the patient observation entry data (except non-VA care): age, race/ethnicity, sex, FY half of cohort entry, and years since the PLWH’s first VA recorded HIV diagnosis. Measurement details can be found in prior work.13, 16 Non-VA care was included as a fixed indicator of non-VA care that was paid for by the VA at any point in the study period.

Time-varying Covariates

Measures of time-varying covariates were taken in each 6-month interval from a PLWH’s entry into observation until their death or the end of the study period. These covariates include ZIP code-level residence rurality, ZIP code-level residence Area Deprivation Index, housing status, number of outpatient visits, number of outpatient visits related to HIV, number of inpatient stays, primary Veterans Integrated Service Network (VISN) of care, the Elixhauser Comorbidity Index, and ICD-10 codes for depression, bipolar disorder, psychoses, post-traumatic stress disorder (PTSD), substance use disorder (SUD), and alcohol use disorder (AUD). Additional details are provided in prior work.13

Missing Data

PLWH who did not have access to MHV tools beyond Rx refill during a specific time period were assigned zeros for other MHV tools in that wave. In the case of a missing area deprivation index measure for the current time period, baseline, or lag 6 months, an average of all time intervals was taken and imputed. PLWH missing this measure for all time intervals were excluded from the cohort. Missing race/ethnicity were assigned to an “unknown” category, which was then pooled with American Indian or Alaska Native, Native Hawaiian or Other Pacific Islander, and Asian to form the “Unknown/Other” race/ethnicity category. PLWH missing rurality information were assigned to an “unknown” category for that time period. PLWH missing baseline VISN information were excluded.

Primary Outcomes Medication Adherence

Proportion of Days Covered (PDC) is the percentage of days a PLWH theoretically has their medication based on VA pharmacy data. It is the preferred medication adherence measure of the Pharmacy Quality Alliance and used in previous studies of ART.18,19,20 An ART regimen includes at least 2 classes of drugs that minimize viral resistance. To be counted as adherent, a PLWH must have 2 or more classes of ARTs on any given day and prescribed from FY12 to FY18. Each medication and respective class of drug can be found in Appendix 2 as well as additional details for calculating adherence.21 Adherence was treated as a continuous variable with values ranging from 0 to 1 (vs a binary variable). Adherence from the 6 months after MHV tool use was assessed in models to better judge temporal effects.

Viral Load Testing and Suppression

VA laboratory data was used to find cohort instances of viral load testing and whether those labs resulted in suppressed or unsuppressed viral load, specifically nucleic acid–based quantitative viral load testing. Additional details for metric refinement are in Appendix 3. In each 6-month period, the most recent test was chosen as the viral load suppression value of interest. PLWH were considered to have received a viral load test in a 6-month period if any test result in that time had an interpretable numeric value. For viral load suppression, a last-observation-forward approach was used. Both viral load suppression and viral load test receipt were treated as binary variables. The viral load test receipt from the 6 months after MHV tool use was assessed in the model to better judge temporal effects. The denominator for viral load suppression includes those who had a baseline viral load test; they stay in the cohort if they meet other criteria, including VA activity and no record of death.

Statistical Analyses

Longitudinal marginal structural models, using inverse probability of treatment weighting (IPTW), were used to examine the relationship between MHV tool use and the primary outcomes.22 These models can address possible selection bias, loss-to-follow-up, and complex time-varying confounding, making them a highly useful approach in this instance.23 Appendix 3 provides details about this modeling approach. The outcome models were estimated using IPTW-weighted generalized estimating equations. Specifically, we estimate the effects of MHV tool use on adherence, viral load suppression, and viral load test receipt, conditional on baseline and time-fixed covariates. Robust standard errors, an autoregressive correlation structure of order one, clustering by patient, and fixed effects for baseline VISN were employed (see Appendix 4 for R packages used). As noted by Althouse (2016)24, corrections for multiple comparison may obscure relevant findings in an exploratory study such as this one; confidence intervals are provided to support interpretation of our findings.