WHAT IS ALREADY KNOWN ON THIS TOPIC

While most low/middle-income countries (LMICs) have made little progress in reducing road traffic deaths, official statistics from Brazil suggest strong improvements since 2012.

There are substantial discrepancies between official statistics and motor vehicle insurance claims and global health statistical estimates produced by the Global Burden of Disease (GBD) study and WHO’s Global Health Estimates.

WHAT THIS STUDY ADDS

We show that although official statistics underestimate the true road traffic death toll by about 31%, the officially reported reductions in traffic deaths in the last decade are likely real.

GBD estimates of road traffic deaths do not track the trends evident in the underlying death registration data and underestimate the extent of recent improvements in Brazil.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Brazil presents a rare LMIC road safety success story. A high-level evaluation of the country’s road safety programmes can provide important guidance to other countries.

GBD will need to improve methods for modelling trends in road traffic deaths before it is a reliable source for monitoring progress during the Second Decade of Action for Road Safety 2021–2030.

Introduction

The first UN Decade of Action for Road Safety, 2011–2020, ended with most low/middle-income countries (LMICs) showing little progress in reducing traffic injuries.1 2 Brazil was a notable exception, reporting that their road traffic deaths peaked in 2012 (44 812 deaths) and declined by 29% by 2019.3 While this decline is much less than the 50% reduction that the Decade of Action aimed to achieve,4 it nevertheless suggests a remarkable performance which may present a model for other countries to emulate.

However, there are several discrepancies that raise questions about the reliability of Brazil’s official statistics. Official reporting of road traffic deaths in Brazil is based on death certificates, and derived from the Mortality Information System (SIM) database, of the national health system, Sistema Único de Saúde.3 SIM registered 31 945 road traffic deaths (15.1 per 100 000 population) in Brazil in 2019. However, Seguradora Líder DPVAT, which manages mandatory auto insurance (Law 6, 194/74), reported paying 40 721 traffic fatality claims in 2019, 27.5% more than deaths reported by SIM.5

One explanation for the discrepancy is because of fraudulent claims that have received substantial media attention.6 However, it is extremely unlikely that fraud occurred frequently enough to explain the large discrepancy (in 2019, DPVAT reported only 553 fraudulent claims, 1.4% of fatality claims).5 Another explanation that has been suggested in our discussion with experts in the country is that insurance claims are reported based on the year when claims are paid, rather than the year of the event. Nevertheless, such an explanation cannot justify a consistently higher count of insurance claims over many years when trends in deaths have been relatively stable. In fact, over the 13-year period shown in figure 1, there were 25% more insurance claims (130 500 deaths) than officially reported deaths, and insurance claims were higher than official statistics in every year except 2016.

Figure 1

Comparison of official statistics of road traffic deaths in Brazil with insurance claims and estimates by IHME and WHO. DPVAT: Danos Pessoais causados por Veículos Automotores de via Terrestre; GBD, Global Burden of Disease; GHE, Global Health Estimates; GSRRS, Global Status Reports on Road Safety; IHME, Institute for Health Metrics and Evaluation; SIM, Sistema de Informação sobre Mortalidade.

Comparisons of official statistics with estimates developed by international agencies provide further evidence to suggest under-reporting in official statistics. Figure 1 shows road traffic deaths estimated by two major global health studies: (1) the Institute of Health Metrics and Evaluation’s Global Burden of Disease (IHME-GBD) study and (2) the WHO’s Global Health Estimates (WHO-GHE). GHE are the basis of modelled estimates reported in WHO’s Global Status Reports on Road Safety (GSRRS). Note that while official statistics only tabulate deaths registered in SIM with road traffic crash as their listed cause, GHE and GBD aim to provide estimates of the true death toll after adjusting for the quality of reporting by SIM.

Both GBD and GHE estimate more deaths than official statistics over the entire period but GBD’s estimates are substantially higher. GBD estimates for 2019 are 46% higher than official statistics and only slightly (10%) higher than insurance claims. Besides the difference in magnitude, the discrepancy in road death trends is noteworthy. GBD estimates show a much slower decline in road traffic deaths. Unlike GHE, which track the trends in official statistics, GBD estimates of road traffic deaths are relatively flat, showing a decline of less than 10% since 2012, much less than the decline reported in official statistics.

Thus, figure 1 raises two important questions that are the focus of this study: (1) Are official statistics of road traffic deaths in Brazil under-reported and by how much? and, (2) Are road traffic deaths declining as rapidly as suggested by official statistics? In this study, we systematically assessed the quality of the Brazilian death registration system, especially the quality of cause coding, and assess how this affects estimates of the magnitude and trend of road traffic deaths.

MethodsData source

We extracted death registration data (ie, tabulations of number of deaths for each cause, disaggregated by age and sex) for Brazil from the WHO Mortality Database (WHOMDB, June 2021 revision) for the period 2002–2019.7

Map external cause codes

We reclassified all deaths coded to external causes into 48 categories of specified external causes of death, and 21 categories of partially specified external causes as defined by the GBD-2010 Injury expert group.8 The categories specific to road traffic deaths are shown in table 1.

Table 1

Case definition for road traffic deaths and reattribution strategy for deaths coded to partially specified causes

Assess completeness

We assessed completeness of the registration system by calculating the ratio of registered deaths and independent estimates of all-cause deaths in the country. There are two global agencies that estimate all-cause deaths: (1) IHME, whose estimates are used to adjust for completeness in GBD9 and (2) United Nations Population Division’s (UNPD) World Population Prospects estimates,10 which are used by WHO to adjust for completeness in GHE. The two sources produce estimates that can differ substantially for some countries.11 12 These differences affect road traffic death estimates because cause-specific mortality estimates are scaled-up to ensure their aggregate is consistent with all-cause death totals.

Reapportion partially specified causes of death and adjust for completeness

We estimated national road traffic injury deaths using an algebraic process of redistribution of partially specified causes of death that we have described previously.13 Briefly, this method for reattribution involves a series of steps, where the deaths in each of the partially specified categories are redistributed over the set of well-specified categories to which they could possibly belong, in proportion to the numbers of deaths in those categories before redistribution. The redistribution is done within age-group and sex-group. This method is similar to that used in WHO’s estimates (GHE) and was used in GBD until it was replaced by a new method14 that is more complex and less transparent to external researchers. Finally, we scaled up the age-specific and sex-specific road traffic death estimates to adjust for under-registration (ie, completeness).

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Results

The proportion of deaths from all causes that are registered by SIM has risen steadily over the last two decades and the system is nearly complete now. In 2019, completeness was 95.5% based on comparison with IHME’s estimate of all-cause deaths, and slightly higher (98.2%) based on UNPD’s estimate (see online supplemental figure A1).

Figure 2 shows the relative magnitude of deaths coded to those partially specified causes that affect estimates of road traffic deaths. The proportion of deaths coded to transport that are unknown (ie, unknown if they are road traffic or another mode of transport) has steadily increased but the proportions are small. In 2019, these amounted to less than 3% of transport deaths. The proportion of unintentional injury deaths that are unknown (ie, unknown if they are road traffic or other types of unintentional injury) has declined and was only 2.6% of all unintentional injury deaths in 2019. The proportion of injuries that are not further specified (ie, unknown if they are road traffic or other types of intentional or unintentional injury) shows an unusual pattern with elevated values in 2008 (6% of all injuries) and a spike (8.4%) in 2019. Almost all the deaths (98%, 11 822 deaths in 2019) coded to this partially specified cause group were coded to ICD-10:Y34 (Unspecified Event, Undetermined Intent). Finally, the proportion of deaths coded to the broadest group of unspecified causes (unknown cause of death) has declined from 12% in 2002 to less than 5% in 2019.

Figure 2

Percentage of deaths coded to partially specified cause categories relevant to estimates of road traffic deaths.

Overall, the proportions of deaths coded to each partially specified cause are small compared with the threshold that is typically used to assess the quality of death registration systems.15 Nevertheless, they have a relatively large cumulative effect on road traffic deaths estimates. Figure 3 shows the incremental effects of the adjustments to account for deaths coded to partially specified causes and completeness of reporting. The effect of reapportioning unspecified transport deaths and unspecified unintentional injuries is relatively small, especially in recent years. The effects of reapportioning the broader partially specified categories, and the adjustment for completeness of death registration, are a bit larger. In total, these adjustments result in an estimate that is 31% higher than deaths specified as road traffic (baseline).

Figure 3

Effect of reapportionment of partially specified causes of death that may include road traffic deaths.

The cumulative effect of these adjustments has not changed significantly over the last two decades. As a result, the fully adjusted estimates have a time trend that is similar to the official statistics (ie, the baseline in figure 3). For instance, official statistics suggest that between 2012 and 2018 road traffic deaths declined by 27%, while our fully adjusted estimate suggests they declined by 25%.

Figure 4 shows that the adjustments bridge much of the gap between official statistics and GBD estimates. Furthermore, our estimates are similar to the deaths reported by insurance claims. However, while our fully adjusted estimates are lower than GBD for the last 5 years (2015–2019), they were higher than GBD for the previous 5 years (2010–2014). Thus, our fully adjusted estimates suggest a sharper decline in traffic deaths in recent years than that estimated by GBD. Figure 5 helps explains this difference in trends predicted by GBD and our estimates. Figure 5 compares how well the GBD-2019 corrected mortality estimates (in red) fit the input death registration (yellow circles). The input data have already been adjusted to account for deaths coded to partially specified deaths and corrected for completeness (ie, the input data shown are equivalent to our fully adjusted estimate). It is evident that the GBD-2019 estimates do not track the rapidly increasing deaths in the underlying data during the late 2000s and the rapidly declining trends in the last decade.

Figure 4

Comparison of our fully adjusted estimate with official statistics, insurance claims and GBD estimates. DPVAT: Danos Pessoais causados por Veículos Automotores de via terrestre; GBD, Global Burden of Disease; IHME, Institute for health metrics and evaluation; SIM, Sistema de Informação sobre Mortalidade.

Figure 5

Comparison of the GBD-2019 estimates for Brazil with the adjusted vital registration data used as GBD-2019 model inputs. (These are annotated screenshots of the GBD causes of death visualisation tool, CODVIZ: https://vizhub.healthdata.org/cod/). CoD, causes of death; GBD, Global Burden of Disease; VR, vital registration.

DiscussionQuality of road traffic injury surveillance in Brazil

Brazil is among the few countries that use vital registration as the source for official road traffic death statistics. This is a serious shortcoming because death registers are poorly suited to provide the information needed for effective road safety management—that is, for identifying risk factors, developing a safety strategy, implementing targeted interventions and evaluating outcomes. In countries with mature road safety programmes, crash-scene investigations by police provide detailed information about crash circumstances, vehicles involved, road environment and behavioural risk factors. However, police-based surveillance in Brazil is highly fragmented with no single database that covers the entire country. Expanding the coverage of the Federal Highway Police database, as recommended by the 2015 National Road Safety Capacity Review by the World Bank,16 could be an effective solution to a comprehensive and complete national road traffic database.

Until a comprehensive traffic safety surveillance system is established, the national death register (SIM) is an important data source for estimating the scale of the road safety problem and for tracking progress in reducing deaths. Our analysis shows that SIM contains a large number of deaths classified to partially specified causes. Reattributing these deaths results in an estimate that is similar to insurance claims and 31% higher than registered road traffic deaths.

Our analysis assumes that every cause of death is equally likely to be assigned to partially specified causes. We have previously shown17 that our approach of age, sex-reapportionment is likely the best approach for reattribution if age and sex are the only other information available about these deaths. However, cause of death information may not be missing at random. For instance, Soares Filho18 suggests that in Brazil deaths in hospitals are often coded to unspecified causes because doctors do not have the prerogative to determine legal causes if a police report is unavailable. Therefore, it is important that future work on improving road traffic injury estimates should include empirical studies that assess the biases in deaths coded to partially specified causes. For instance, Mandacaru et al 19 used probabilistic record linkage to link death certificates to police records and hospital records in five cities. They reported that the proportion of traffic deaths missed by SIM varied considerably by city, ranging from 4.2% in Curitiba to 33.5% in Teresina, which is broadly consistent with our estimates, and confirms that under-reporting in official statistics is a significant problem. Extending such work to a nationally representative sample of deaths could provide more insights into under-reporting and the true death toll. Finally, our analysis was conducted with data aggregated at the national level because data at sub-national level were not available to us. However, we expect that there are large differences in quality of coding across provinces, and that replicating our analysis at the province-level will provide more accurate results.

Implications for global road safety efforts

Notably, our analysis confirms that road traffic deaths in Brazil have declined by 25% since 2012, which is close to the decline estimated by official statistics but much more than GBD-2019 estimates. These declines have been driven by strong declines in pedestrian deaths and motor-vehicle occupant deaths (see online supplemental figure A3). This is a remarkable achievement because most LMICs have failed to show meaningful progress during the last decade despite extensive global advocacy.2 4

Although assessing what has driven these declines in road traffic injuries was beyond the scope of this study, there are a number of recent and ongoing road safety initiatives that may be resulting in improving road safety in Brazil. Before–after evaluation of the Federal Highway Police’s Rodovida campaign, which included an enforcement and promotion campaign on federal roads, shows beneficial effects on the risk of serious crashes.16 Efforts to enforce drink-driving laws in some states have benefited from legislative change that allows police to use behavioural evidence of intoxication for drivers who refuse to take a breath test.16 There is also increasing attention to improvements to the safety of road infrastructure and the urban built environment, with the state of Bahia committing to a minimum 3-star iRAP safety rating for new and rebuilt roads.20 Evaluations of the Bloomberg Philanthropies Global Road Safety Programme’s activities in Brazil, report improved safety of infrastructure21 and reduced speeding in Sao Paulo and Fortaleza.22 23 A high-level evaluation of the effect of these and other interventions on the national and province-level road traffic injury toll can explain what has worked in Brazil and provide guidance to other LMICs.

Implications for global modelling efforts

Although the discrepancy between GBD and GHE estimates of traffic deaths in Brazil was not the focus of this study, it is surprising and problematic. In 2019, GBD estimates were 31.5% higher than GHE estimates and the two estimates had non-overlapping uncertainty ranges. Discrepancies between GBD and GHE estimates are understandable in information poor contexts (eg, many parts of sub-Saharan Africa), or when one of the projects incorporates data sources that are not used by the other project. However, both studies use the same underlying data source (national death registration data) for Brazil and conceptually similar methods for estimating road traffic mortality. Furthermore, Brazil’s death registration data are considered to be of high quality. IHME rates the quality of Brazil’s death registration data at 4 (out of 5) stars, the same rating it gives many Western European countries.9 Thus, the discrepancy between GBD and GHE in Brazil illustrates that even in countries that have arguably the best health sector data, global health statistical projects (like GBD and GHE) disagree substantially on their point estimates and predicted trends.

Relatedly, our findings raise concerns about GBD’s ability to correctly estimate trends in road traffic deaths and monitor progress towards Sustainable Development Goal (SDG) 3.6. While our analysis confirms the declines in national road traffic deaths since 2012 reported in official statistics, GBD estimates are much flatter. We show that the discrepancy arises not from adjustments for completeness and reattribution of partially specified causes, but because GBD’s final modelled estimates do not closely track the trends apparent in the (adjusted) underlying data (figure 5). For this step, GBD uses a tool called CODEm (Cause Of Death Ensemble model) that runs a large number of different models (including different statistical approaches, different units of analysis and different choices of covariates) and uses an ensemble of models that perform best in out-of-sample prediction tests.24 25 CODEm ensures consistency across all cause-specific models by scaling their final total to match all-cause deaths in each age, sex, country and year. While ensemble models have been shown to outperform the best component models for diseases and a range of other fields,24 we show that in the particular case of road traffic deaths in Brazil, current GBD models do not track the underlying data. We suggest that IHME should invest additional effort to improve the modelling of road traffic deaths, including a focus on better modelling of trends so that GBD estimates can be used to track progress towards the SDGs related to road traffic during the second decade of action for road safety.

Ethics statementsPatient consent for publication

Not applicable.

Ethics approval

Not applicable.

Acknowledgments

We thank our project advisory group for feedback on preliminary results.