A total of 468 participants opted-in and were exposed to the 2wT intervention. Participants were followed during their first year on ART and spent a median of 52 weeks (interquartile range [IQR] 34 – 52) in the study. Table 2 summarizes implementation fidelity by 2wT core component.
Table 2 Fidelity by intervention core componentCore component 1: Participant receives weekly motivation messagesAs designed, participants spent a median of 52 weeks (IQR 34 – 52) in the study, in which 52 motivation messages were expected. In practice, the 2wT platform sent out motivation messages for 44 (IQR 30–47) enrolled weeks and messages were successfully delivered for 31 (IQR 16 – 42) enrolled weeks, corresponding to receiving a motivation message for 75% (IQR 56%—83%) of enrolled weeks (Appendix 1). There was substantial heterogeneity in message delivery patterns across participants (Appendix 2). Message delivery success was higher for older (77%; IQR 62%—85%) compared to younger participants (71%; IQR 50%—83%) (p = 0.01) and for female (77%; IQR 60%—85%) compared to male participants (73%; IQR 50%—83%) (p = 0.01). Over 19,787 combined participant weeks, a motivation message was sent for 17,434 participant weeks (88% of enrolled weeks), and successfully delivered for 13,423 participant weeks (68% of enrolled weeks). Overall, message delivery was unsuccessful for 23% of transmitted motivation messages. Of the 4,012 undelivered messages, 3,200 (80%) expired, 588 (15%) had ‘sent’ status, 180 (4%) were rejected, and 44 (1%) failed.
Evaluating the distribution of motivation message delivery status per week over time (Fig. 2), weeks that were clear anomalies can be identified in which a substantially lower number of messages were sent out by the platform than expected (weeks 14 – 18 and week 43 of 2022) or in which a substantially higher number of messages expired (week 48 of 2021).
Fig. 2Distribution of the number of motivation messages sent by the 2wT platform over time by message delivery status
In survival analysis, we found that 59% of participants consistently received messages and did not experience a gap of 3 consecutive weeks without motivation messages while enrolled in the study (Fig. 3). For the 190 (41%) participants who experienced undelivered motivation messages for three or more consecutive weeks, the median time to event was 13 weeks (IQR 5 – 25 weeks).
Fig. 3Kaplan–Meier survival plot showing time until the first occurrence of a gap of ≥ 3 weeks in motivation message delivery
Core component 2: Participant is proactively reminded of a scheduled appointmentParticipants had a median of 6 (IQR 4 – 7) clinic appointments during the study period. They were expected to have received a proactive reminder for 5 (IQR 3 – 6) of these appointments (no messages were expected for appointments attended > 3 days early). Typically, the 2wT platform sent proactive reminders for 5 (IQR 3 – 6) clinic appointments and reminders were received by participants for 4 (IQR 2 – 5) appointments, an expected appointment reminder delivery success percentage of 83% (IQR 67%—100%) (Appendix 1). Message delivery of expected proactive appointment reminders did not differ between younger and older participants or between male and female participants. Collectively, participants had 2,513 clinic appointments during the study, for 2,353 of which reminders were expected. Participants were sent proactive reminders for a total of 2,120 clinic appointments (90% of eligible appointments), and received them for 1,819 appointments (77% of eligible appointments). Of the 301 (14%) transmitted appointment reminders that were not delivered, 290 (96%) expired and 10 (3%) had ‘sent’ status. Both transmission and delivery of appointment reminders were generally high (> 75%) throughout participants’ follow-up trajectory (Fig. 4A). After an initial decline, reminder transmission appeared to improve later in follow-up. Delivery of transmitted messages, on the other hand, appeared to decline over the course of follow-up. Additionally, appointment reminders were transmitted for 74 appointments and delivered for 67 (91%) appointments that did not appear in the EMRS, indicating a potential system error.
Fig. 4Delivery success of proactive appointment reminders and missed appointment reminders by appointment number*
*Appointment interval heterogeneity explains the variation in the number of appointments participants have within the same time period
Core component 3: Participant is reminded after a missed appointmentParticipants missed a median of 2 (IQR 1 – 3) appointments throughout follow-up. A missed appointment reminder was expected for 1 (IQR 1 – 2) of these appointments (no message was expected for appointments missed by less than 2 days). Generally, the 2wT platform sent reminders for 1 (IQR 0 – 2) missed appointment and participants received reminders for 1 (IQR 0 – 1) missed appointment, an expected missed appointment reminder delivery success percentage of 67% (IQR 0%—100%) (Appendix 1). Message delivery of expected missed appointment reminders did not differ between younger and older participants or between male and female participants. Collectively, participants missed 868 appointments, for 606 of which, they were expected to receive missed appointment reminders (appointment missed by 2 or more days). Participants were sent reminders for 468 (77%) of these eligible missed appointments and received reminders for 349 (58%) of these eligible missed appointments. For 119 eligible missed appointments, transmitted reminders were not delivered; for 109 (92%) of them, the reminders expired and for 10 (8%), the reminders had ‘sent’ status only. Transmission of missed appointment reminders improved throughout follow-up while delivery of transmitted missed appointments decreased with missed appointment number (Fig. 4B).
For the 262 missed appointments that were missed by fewer than 2 days, for which no reminder message should have been sent out, missed appointment reminders were transmitted in 149 cases and delivered in 111 (74%) cases. Additionally, the 2wT platform sent out 406 missed appointment reminders and delivered 345 (85%) missed appointment reminders for appointments that had been attended by participants on or before their scheduled appointment date. Finally, missed appointment reminders were sent out for 30 appointments that were not found in the EMRS; 25 (83%) were delivered. Taken together, in 585 (56%) of the 1,053 instances in which missed appointment reminders were transmitted, they were not transmitted as intended.
Core component 4: Participants can interact with the 2wT platform to report transfers request visit rescheduling and confirm planned visit attendanceTransferParticipants who indicated that they would not return to clinic by responding with a ‘0’ to an appointment or missed appointment reminder prompt, received an automated follow-up prompt inquiring whether they had transferred. A total of 150 follow-up prompts were delivered, to which participants responded affirmatively 47 (31%) times. These participants were further followed-up: true (temporary) transfer had taken place in 9 (19%) of these cases. Of the 468 2wT intervention participants, 39 (8%) participants had ‘transfer out’ as their final study outcome at 12 months: 3 (8%) had reported this transfer via 2wT and 36 (92%) had communicated this through other means (e.g., during clinic visits, in a call with 2wT staff, or reported transfer when traced).
ReschedulingThroughout the study, 326 requests were made to reschedule clinic appointments. Of these requests, 115 requests (35%) were made by 82 participants (18%) in response to 2wT rescheduling prompts, and 211 requests (65%) from 153 participants (33%) were spontaneous requests. Of the 115 prompted rescheduling requests, 46 (40%) resulted in appointment changes. The 211 spontaneous requests resulted in 196 (93%) appointment changes.
Visit attendance confirmationTo explore participant responsiveness throughout follow-up, the percentage of appointments and missed appointments for which participants responded to delivered reminder messages was plotted per appointment and missed appointment number (first, second, third, etc.), respectively (Figs. 5A & 5B). For both proactive appointment reminders and missed appointment reminders, participant responsiveness appeared to decrease over time.
Fig. 5Participant response to successfully delivered (missed) appointment reminders by (missed) appointment number*
*Appointment interval heterogeneity leads to variation in the number of appointments participants have within the same time period
We evaluated the response to subsequent proactive appointment reminders for participants who did not respond to the initial reminder. Among participants who did not respond to a reminder sent three days before a scheduled visit, but received a second reminder one day before the visit, 26% (237/921) responded to the second appointment reminder. A similar analysis was conducted for missed appointment reminders. Of the participants who did not respond to a successfully delivered reminder two days after their missed appointment, but received a second reminder three days later, 15% (8/54) responded to the second reminder. Among those who did not respond to the second reminder but received a third reminder eleven days after their missed appointment, 17% (8/48) responded to the last reminder.
Message muting and phone number changesDuring the study period, 2wT participants could request to stop receiving motivation messages and/or reminders. Of the 468 intervention participants, 20 (4%) participants requested to mute messages, with 16 (75%) wanting both message types muted, and 4 (25%) requesting to mute motivation messages only. The average time to motivation message muting was 39 days (IQR 20 – 77 days) and to reminder muting was 48 days (IQR 15 – 78 days). During the study, 16 (3%) participants provided a second phone number to the study team. For 14 (88%) participants, this was an alternative/additional number and for 2 (13%) participants this concerned a phone number change.
Effect of core component implementation on clinic attendanceIn univariable analysis, for every 10% more successful delivery of expected motivation messages, participants had 1.13 (95%CI: 1.06 – 1.20, p < 0.001) times higher odds of attending clinic on the scheduled date and 1.12 (95%CI: 1.03 – 1.22, p = 0.01) times higher odds of returning to care within 14 days following a missed appointment (Table 3). Proactive appointment reminder receipt and response (any answer) to an appointment reminder were similarly associated with higher on-time clinic attendance in univariable analysis (OR 1.83, 95%CI: 1.44 – 2.33, p < 0.001 and OR 1.52, 95%CI: 1.19 – 1.95, p < 0.001, respectively) (Table 3). No association was found between missed appointment reminder receipt or response and return to care within 14 days.
Table 3 Mixed-effects logistic regression analysis of the effect of core component implementation on clinic attendanceIn multivariable analysis, after adjusting for demographic characteristics and other core components, the association between a 10% increase in expected motivation message receipt, proactive appointment reminder receipt, and appointment reminder response, respectively, and on-time clinic attendance remained significant (Table 3). A 10% motivation message delivery increase was associated with 1.08 times higher odds (95%CI: 1.01 – 1.16, p = 0.03) of on-time clinic attendance. Participants with a successfully delivered appointment reminder (compared to not) had 1.35 times higher odds (95%CI: 1.03 – 1.79, p = 0.03) of attending that appointment on the scheduled date. Responding to a delivered appointment reminder was associated with a further increase in the odds of on-time attendance (aOR 1.47, 95%CI: 1.16 – 1.87, p = 0.001). No associations were found between motivation message delivery, missed appointment reminder receipt, and missed appointment reminder response and timely return to care following a missed appointment in multivariable analysis.
Comments (0)