Translating global evidence into local implementation through technical assistance: a realist evaluation of the Bloomberg philanthropies initiative for global Road safety

In 2014, BIGRS initiated a common TA program in Accra, Bogotá, and Mumbai. TA interventions, individuals providing and receiving TA, the city context, and the national-level road safety context influenced implementation. Table 4 outlines interventions and outcomes. Interventions are grouped under two outcomes: (1) improved road safety capabilities (via capacity building and data) and (2) increased the uptake of evidence-informed road safety interventions (via infrastructure, enforcement, and policy support).

Table 4 Comparing the case studies across the intervention categoriesa

We present one example per case that demonstrates how different interventions worked together to achieve different outcomes in case study cities. Interventions (i), mechanisms (m), contexts (c), and outcomes (o) are denoted in the text. Reference to KII data is provided as M# for Mumbai, B# for Bogotá, A# for Accra, and G# for KIs working across multiple cities. 

Transforming junctions on Mumbai’s congested streets

When BIGRS began, Mumbai’s road safety officials used high-level figures on traffic fatalities supplemented with national or state-level statistics to guide road safety decision making (c). The city-specific data required to target road safety interventions was buried in paper-based police records of variable quality (c). A government official in Mumbai describes:

"That is [a] very big [problem] because we are not like other countries, we are not getting the data correctly.” – M17

In response, BIGRS’ TA first sought to improve surveillance data. An embedded surveillance coordinator partnered with the police to catalog and analyze city surveillance data and package it in new annual city road safety reports (i) [40]. Infrastructure assessments (i) further demonstrated how specific road junctions contributed to injuries and mortality (o). BIGRS staff described the change in data availability:

“[Before] there were no reports at all […]. Now I have […] a 40-page report that talks about who the road users are, […] a list of high-risk junctions and corridors […], a map that details the hotspots where crashes are occurring […] which vehicle is causing maximum crashes, […] the time of the day, the month of the year, the day of the week.” – M8

New data demonstrated that half of traffic-related crash victims were pedestrians, which was further reported by local city media [41, 42]. Providing granular, city-specific data shifted the focus (m) of government towards pedestrian safety (M8, M12, M17). The same BIGRS staff member described:

“The government didn't know that so many pedestrians were dying in crashes. These reports help bring that to light. And when that came to light, they started taking a more serious approach.” – M8

The emphasis on pedestrians was echoed by city government officials, who agreed that the data was illuminating (M12, M17). But the city also required solutions for this perception shift to lead to concrete action. The same government official describes the challenges (c):

“We lack the best instrument in the old system to make the road elevated, or a road underpass is very difficult because the traffic on that high main road. […] That is very critical because it we are really facing problems.” – M17

BIGRS provided the safe system approach as a solution– but how would it work in Mumbai? This question was a central concern in all cities, especially in densely populated Mumbai, where participants described the street as a ‘contested space’ (M9, M2, M21) (c). Implementing the safe system approach in Mumbai required a complex adaptation process (G5, M2, M9, G6, M7) (i). A BIGRS infrastructure partner described:

“We’re constantly trying to balance Global Best Practices versus what can be done in an Indian city while pushing boundaries to be able to think outside the box. […] it's helpful to show International Best Practices, but also at the same time, balancing it out with what’s actually possible in Indian cities.” – M9

Implement-ability was top of mind for city government officials taking risks by trying a new approach (c). BIGRS staff had to recognize those risks and work collaboratively with city government counterparts to understand how international approaches could work within local realities (B15, M13) (c). A BIGRS staff member described this:

“[When you introduce international examples], there are a lot of questions and pushback saying, ‘how could this be done [here]? That was also instructive to us. How do you deal with such situations?” – M13

Short-term demonstration projects – for example, temporarily changing traffic flow using cones and other local, low-cost materials (i) – allowed city government counterparts to see the safe system approach in action on their street in a low-risk context, demonstrating that a new approach was possible (B15, M11, G7, M9, B7, M17). A city official describes:

“People are generally not aware of the things [happening outside India], [but] the problems are same. […] That can be taken only if you can show them the models […] because firsthand information from those people is much more important.” – M17

Seeing the possibility of change was perceived to shift the focus of road safety towards vulnerable road users and especially pedestrians (M18B, B15, M11, G7, M9, M15, M18, M17, M2, G2, M9, M7, M14, M8) (m). It also created a ‘how-to’ moment, enabling city government counterparts to internalize both the concept and implementation feasibility (m). A city official describes what he learned:

“[I learned] new technical things, that might be there's been a certain technical change in junction design or in the road design. […] we were not able to do that thing nicely already […] We were able to grab that opportunity properly.” – M18

Once city government counterparts understood the potential of the safe system approach (o), BIGRS TA worked with city officials to use the data and select specific junctions for re-design (i). Data was critical because it helped target infrastructure improvement to junctions with an outsized number of crashes. A BIGRS staff described:

“Now [Mumbai city government are] not just randomly doing the interventions. They're very focused on where crashes are occurring, who the victims are, who the perpetrators are, and how to ensure that these crashes don't occur at all.” – M8

Once the mechanism for change was triggered, transforming junctions started with a pilot (i). Pilots ensured that the safe systems approach was feasible and appropriately adapted to the context, that its impact on different road users was understood and planned for, and that the re-design successfully reduced crashes [43,44,45]. Pilots also allowed the city government to understand public sentiment about the changes (c) [43]. If the public was supportive, this reduced the risk to city officials trying a new approach (m). The city government counterpart’s confidence in new approaches grew (m). This was further reinforced by data showing that infrastructure redesign positively impacted traffic flow [43]. A pilot’s success was described as leading to exponential growth in implementation (M21, M1) [44, 46, 47]. A BIGRS partner described:

“If you see our work, it has exponentially grown in impact. […] From that one [pilot] corridor, we […] build a relationship and trust […] and so we got a chance to do design in the intersection. Then you try it with temporary sort of barricades and then it became a big thing. And then one thing just kept leading to another to another.” – M21

As implementation took off, BIGRS engaged local media to spread awareness about the junction transformations (i) (M11). After seeing firsthand what could be accomplished, the city government also committed to improving high-risk intersections in the city. However, despite growing momentum and support from both city government and city engineers to transform individual junctions, the bottom-up pilot approach presented practical scale and sustainability challenges despite this government commitment (M9, M13, documents) (c). A BIGRs partner described:

“It's a challenge at times, when the city does not have the funds allocated in that year. If you do manage a successful pilot and the city takes on doing it, then it's great because they can be scaled up. But in many cases […] pilots are sort of left as just that.” – M9

BIGRS partners described city government approvals as challenges preventing scale. In contrast, city government participants urged respect for government processes and timelines, which they saw as paramount to success (c). In managing these processes, city officials also took on significant work to enable each infrastructure re-design (M12, M18) – a contribution that often went unacknowledged (c).

Comparison of Mumbai’s infrastructure experience with other BIGRS cities

The Mumbai infrastructure example is illustrative of common dynamics. In Bogotá, capacity building was similarly perceived as successful when it used hands-on components specifically relevant to the participants (B15, G2, B9, B7, B14), and when facilitators used a coaching model that emphasized the participant’s experience (B15, B5, B9, G2).

Accra’s and Bogotá’s infrastructure TA were also targeted at bottom-up approaches (G6, A9, A5, A6, A8, M9) and guided by city-specific data (i), but with limited scale. In Accra, BIGRS focused on low- or no-cost interventions (e.g, changing signal times for pedestrian crosswalks, widening pedestrian medians (i)) (A5, A6, A8) because the city did not control the infrastructure budget and could not budget for new interventions. BIGRS also worked with the city to re-design the infamously dangerous Lapaz intersection to improve pedestrian safety, which was funded directly by BIGRS via a small grants program (i) [48]. In Bogotá, tactical urbanism demonstrated speed-calming measures, and feedback from road users was gathered (i) (B2). However, despite promising pilots, the lack of BIGRS’ ability to influence upstream changes to road procurement tenders and design guidelines limited the scale of infrastructure outcomes in each city.

Enforcing road safety legislation in Accra

In Accra, road safety legislation existed but needed to be enforced (c). In the words of a national road safety agency staff, “there’s no real commitment in solving some of these things.” (A17). BIGRS’ enforcement interventions started with relationship building (i). A BIGRS partner describes:

“How important it is to have this relationship with the high-level police officers. Because we cannot just go to a city or road police agency and say that this is what we want to do.” – A22

Trainings on the safe systems approach (i) and evidence-based enforcement operations were enabled by leadership support from the Superintendent of Police (A4) and the Mayor of Accra who championed road safety and several BIGRS initiatives (A4, A6, A3, A5, A8, A1, program documents) [48, 49] (c).

However, translating training into implementation quickly stalled because the police force required equipment and certification for implementing enforcement operations (c) (A1, A4, A6, program documents). BIGRS’ partners then donated new drink-driving and speed enforcement equipment under the condition that the city utilized the equipment to conduct enforcement pilots (i). These donations were accompanied by training and certification processes (A1, A4, A25, program documents) (c).

While the lack of equipment could be directly addressed by BIGRS, the disconnect between city-level enforcement efforts and Ghana’s centralized policing structure could not be so easily overcome. City police did not have the authority to conduct enforcement operations (c), so in exchange for the donated equipment, the police formed a dedicated tri-partite pilot task force with the authority to use the donated equipment in a series of roadside speed and drink-driving enforcement operations (i).

New training and improved accuracy of the equipment were perceived to reduce conflict between police and citizens during enforcement and improved transparency in the enforcement operations (A1, A4, A25, program documents), reducing the perception of risk of public blowback (m). A high-ranking police officer describes the perceived increase in acceptability from the public:

“they don’t complain, they go to the court […] because you’ve told us that the device arefor enforcement operations (A4, A25) limiting further […] the very latest speed device, speed detection devices [equipment] because we’ve told the whole world about it.” – A25

The collective intervention – piloting the enforcement approach, supported by training and in tandem with appropriate equipment – was also received positively by the police. A senior police officer described a shift in focus towards ensuring road safety (m):

“What I’ve realised is, what a positive impact on our capability to be able to ensure road safety. [..]. With the devices, we can go to the route when they see us, all cars, cars you know approaching the robot, reduce their speed and that has really resulted in a lot of improvement.” – A25

However, while the pilot taskforce did conduct enforcement operations, a series of upstream barriers prevented the taskforce from scaling up. Most practically, the police force still lacked dedicated vehicles for enforcement operations (A4, A25) limiting further implementation. More broadly, the social and political context (A1, A4) (c) remained unconducive to enforcement. A BIGRS staff describes:

“During [the enforcement pilot], we did a special round of data collection for speed, and the data showed that there was a reduction in speed. However, the moment could not be sustained. Some of the feedback they got from the police was that [the] police could not boldly or fearlessly enforce.” – A1

Another challenge was that the required authority to change enforcement practices was vested in national agencies instead of the city government, limiting the ability of city police to institutionalize new enforcement operations (A4, program documents) (c). Finally, the transfer of police was described as a challenge to sustaining enforcement interventions (A4, A2, program documents) (c). A BIGRS partner described:

“We can work with person, everything agreed, and then just before we roll out, he's been transferred or there's a rotation, and we have to change everything.”— A22

Comparison of Accra’s enforcement experience with other BIGRS cities

Across the enforcement TA provided in the three cities, building trust with senior police officers was repeatedly emphasized (A4, B4, M16, G7, G10). Using former senior police officers from other countries was seen as key to building that neccessary trust (B4, M16, G7, G10) (c).

Like Bogotá, Accra’s enforcement interventions took place within broader city road safety prioritization (c), and BIGRS donations ensured police had the right equipment (i), leading to increased enforcement (B8, G12, A4, A1, A22, A25) (o). However, Bogotá’s enforcement was described as widespread and sustained (B8, G12), while in Accra, enforcement operations remained limited (o). The authority of the police to conduct enforcement was the key difference (c).

In contrast, India was moving towards an automated speed enforcement model, which contrasted with the model proposed by BIGRS (c). Although automated and roadside enforcement co-exist (and they did in Bogotá), BIGRS’ roadside enforcement model did not align to the broader policy agenda in Mumbai and was not implemented.

Reducing city-wide speed limits in Bogotá – an example of policy change

Before BIGRS, improved mobility had been the focus of several consecutive city administrations (c). A BIGRS partner described the favorable baseline environment:

“Bogotá has been concerned about road safety for a long time. [Bogotá] already had a Road Safety Directorate; […] there was already a direction with a super great team. It was easy to work in Bogotá because institutionally, they were already armed.” – B13

During Phase II, a new Secretary of Mobility with a public health background further elevated road safety in city administration (c) which was perceived as critical to the city's subsequent policy change (B1, B11, B12, B13, B14, B15, B16). A city official explained:

“It is about setting priorities. So, we [the secretariat], from the first day, said the priority is road safety, and we will do everything possible to make it so.” – Bogotá 2

Alongside a change in government, BIGRS also hired new embedded staff, some who were former members of city government, all who were local to Bogotá, and all who were passionately committed to improving road safety (c) (B12). However, support for road safety did not immediately translate to speed. Instead, city officials were interested in reducing drunk driving and were explicitly resistant to tackling speed (c). This was due both to concerns that reducing speed would increase traffic and also a perceived lack of concern from the population over speed (G12, G5, B8, B12). A BIGRS staff recalled:

“Even when communicating to [the] Mayor, he had the issue of road safety in his heart the main thing he communicated and did not want to do. ‘Do not slow down on arterial roads’” – B12

However, BIGRS’ analyses of city data (i) identified that speed was a serious concern on arterial roads at night (B8, B12, G5, G8). A BIGRS partner in Bogotá described:

“The first thing I did was share with the Police the data that clearly showed that most of the deaths occurred at night or early in the morning when most roads were empty.” – B8

This was further demonstrated by a modeling study (i) showing both the relationship between speed and the crash rate and that the change in speed limits would not impact average travel times. This study was important evidence, which was only possible because the city’s existing speed detection infrastructure provided the modeling data (B12) (c).

The presentation of this novel information to city officials was perceived to shift the focus of city officials by demonstrating that speeding was prevalent at night when roads were empty, and that reducing speed wouldn’t worsen traffic (m). City officials used this data to select five arterial road corridors with high speeds, crashes, and deaths to pilot a reduced speed limit of 50 km per hour (kmph) (o and i).

The speed reduction pilot required close collaboration between the Secretariat of Mobility and the police to conduct nighttime enforcement (c). However, the police lacked necessary nighttime radar equipment (c) (B8, G12), a gap subsequently filled by BIGRS’ donations (i). TA was provided for the police to use the equipment and to conduct safe nighttime operations (i), increasing enforcement campaigns in the pilot speed management corridors (o). A BIGRS staff described:

“ [It] was clear when you make enforcement operations visible, like speed enforcement down that avenue. In a matter of months, we already saw a reduction [in speed].” – G12

The new roadside enforcement was complemented by automated speed detection cameras (c); however, the public was skeptical of the speed cameras' threatening the pilot’s success (c). Public messaging campaigns were therefore developed using city data to demonstrate the rationale behind the speed reduction and enforcement (i). A BIGRS staff described:

“Legitimacy has to do with road users' acceptance of this type of control. […] What decisions were made? Make visible the places where photodetection cameras are installed. They were published on the website of the Ministry of Mobility, and there was a strong media drive to make these cameras visible and associate the cameras with the issue of life-saving cameras.” –B8

BIGRS also provided monitoring and evaluation support (i) which quickly demonstrating the pilot’s effectiveness (o). A BIGRS staff described:

“In a matter of months, I already saw a reduction [in deaths]. That gave the Secretary of Mobility the confidence, trust like, ‘OK, like this is working, we are reducing deaths where we are not messing up traffic. Let's do it.’” – G12

A city official recalled the importance of the pilots:

“Yes, yes, yes, that was very well done. The expressive power of those corridors, of the first ones” – B14

Because of the positive pilot results, the city increased the number of corridors with lowered speed limits (o). The results of the pilot were also shared with the public, reinforcing the message that the speed reduction corridors were lifesaving interventions (G12, G5) (i) and further reducing the perception of risk in lowering the speed limit by building public support (m). As the pilot gained increased support, city counterparts used the data to develop a technical document justifying the lowered speed limits to Bogotá’s city council. A BIGRS staff described:

“To be able to argue before the City Council, it was necessary to argue with objective judgment elements […] Why did they decide to slow down? Not because it occurred to us. No, the speed was lowered because this technical document allows us to support making that decision.” – B8

Aided by the pilot’s success and with the support built through public messaging campaigns, the city council maintained the 50 km/h speed limit on the pilot corridors (o). However, the city council initially did not have the authority to change city-wide speed limits permanently (c), preventing scale-up until a window of opportunity opened in 2020. During the 2020 COVID-19 pandemic, a state of emergency was declared, giving temporary executive authority to the Mayor (c). Although the Secretary of Mobility (the champion of the pilot) had changed, their successor became a new champion. They successfully argued that the speed limit reduction was preventing traffic crashes, thereby reducing non-COVID-19 health emergencies and freeing up healthcare capacity during the pandemic. This allowed the Mayor to extend the speed reductions city-wide in alignment with the WHO’s advised 50 km/h (o).

Reflecting on Bogotá’s experience with BIGRS, a city official described how BIGRS’ comprehensive TA approach was important in supporting the city’s road safety vision:

“We wanted to build how this systemic vision of approaching the problem. And then Bloomberg supported us with communications, technical, infrastructure, traffic calming, and enforcement issues.” – B2

City officials and BIGRS staff alike credited city leadership for continuously supporting road safety throughout several administrations and for giving political support to technical staff who brought changes to the city (c). One government official commented:

“Everyone, I think, without exception, has supported this work. I believe that the first requirement to choose a city is that there is willingness. What has been in Bogotá, really, is the political will of the leaders to carry it out. Without it, you do nothing.” – B14

Comparison of Bogotá’s policy experience with other BIGRS cities

The scale of change in Bogotá’s road safety programming stands apart from the other case studies. Second to this was Accra; the city government formed a new road safety council and developed the city's first Pedestrian Action Plan (o). Like Bogotá, BIGRS in Accra leveraged city prioritization for road safety and provided city-specific evidence (i), which focused city stakeholders’ efforts on the importance of pedestrian safety (A8, A1, A5, A3, program documents). Also, like Bogotá, the Mayor was a champion who lent convening power to the development of Accra’s action plan (4, A6, A3, A5, A8, A1, program documents) (c). The Accra Pedestrian Action Plan was further perceived to improve coordination of different road stakeholders towards a common goal (A8, A1, A5, A3, A6, program documents).

In Mumbai, in contrast, BIGRS staff and partners described a lack of an individual champion with the authority to advance road safety policy and planning at the city level as a key challenge (M10, M11, M12, M13, M14, M15, M21).

Revised program theory

The revised program theory for BIGRS should be considered an initial attempt to synthesize across both positive cases (where outcomes were observed) and negative cases (where outcomes were limited by specific factors) to distill a set of higher-level statements about how BIGRS works at the city level and the contexts that enable or constrain its success.

The first program theory is improved road safety capabilities, focused on capacity and data use interventions described by BIGRS staff and partners as precursors to implementation in each case study city.

Program theory for improved road safety capabilities:

Providing TA to increase capacity and data use (i), if delivered via trusted and credible TA providers who provide hands-on coaching support tailored to city needs and with counterparts interested in engaging with road safety, can strengthen road safety capabilities (o) because it shifts the focus of city officials towards evidence-informed approaches and creates a how-to moment to improve road safety through the safe system approach (m). This outcome is enabled by city prioritization of road safety (c) and can be disrupted if city government officials change (c).

The second program theory is increasing the uptake of evidence-informed implementation of road safety interventions. In this theory, capacity building and data now comprise the necessary context that supports the interventions, and BIGRS and city officials are characterized as working together to implement.

Program theory for increasing the uptake of evidence-informed implementation of road safety interventions:

If trusted and credible TA providers, working with and through city champions (c), undertake a successful pilot (i), guided by city-specific data that targets interventions (c), and with facilitation of city implementation via dedicated equipment, training and other supportive resources (i), then this can increase the uptake of evidence-informed road safety interventions (o). This occurs because a pilot builds confidence that the safe systems approach is feasible in a specific road context (m), and it reduces the perception of risk in adopting a new approach (m) by allowing city officials to gauge public sentiment. The scale and sustainability of the outcome(s) are determined by the city’s existing prioritization of road safety, the authority of the individuals and road safety agencies targeted in the intervention, and existing socio-cultural norms (c). It can be disrupted if city government officials change (c).

留言 (0)

沒有登入
gif