Health facility characteristics

All districts councils in Mwanza region were targeted for the assessment from which service delivery points (health facilities providing immunization) were randomly sampled proportionally from each council. Among the health facilities assessed, few reported to have received supportive supervision and of those almost half had received written feedback to help referring to actions points and recommendation following supportive supervision (Table 2).

Table 2 Health facility characteristicsFacility operations and management consolidated results (Heat map)

EVM assessment results has shown that regional overall score at health facilities was 53% (Table 3). All categories had scored below 80% WHO benchmark. The score ranged from 72% for C3-information technology to 43% for C5-policies and procedures, which had the lowest score. Of the all criteria, only M3-supportive supervision of the eighty facility and operation management criteria exceeded the benchmark score of 80%. Waste management (E9), as one of the other criteria scored 68% and 15% was obtained for MI, the annual needs forecasting.

Table 3 Facility operations and management consolidated results (Heat map)

The overall score on vaccine management was 53% and varied between health facilities with average highest score of 76% and lowest scores of 27% which are all below 80% WHO benchmark. Only 68 (67%) had at least reached the average score between 50% and 80% (Table 4).

Table 4 Health facility (HF) scores by district

The assessment conducted on the categories on health facility operations and management function which showed vitiations on performance and none of the category reached the WHO benchmark (Fig. 1).

Fig. 1

Overall EVM category scores of health facilities assessed, n = 102

Human resource capacity performance

The overall score was 53% for the human resource capacity with highest score of 66% on waste management followed by 62% on temperature management and the lowest scores was on annual needs forecasting with 14% (Table 5). The attributes for the human resources performance are described in Table 5.

Table 5 Human resources capacity on staffing, training and knowledge

Staffing availability The performance score on the staffing levels in most of the health facilities was below the WHO benchmark on the requirement to have at least one staff assigned to undertake various duties on vaccine management. The highest score that met the minimum acceptable level was on availability of technicians to support refrigerators maintenance from the district with 94 (94%) which met the recommended WHO score of 80% (Table 5). The availability of technicians was also reported through key informant interview (KII), where the District Immunization and Vaccines Officers (DIVOs) reported to support refrigerator maintenance upon being reported by the health facilities.

“We normally receive the notification when the refrigerators get faults and we go to assess and fix what we find and when we fail then we report to high level……” KII_03.

Training of staff responsible for vaccine handling and on monitoring vaccine temperatures had a score of 57 (56%), routine refrigerator maintenance 77 (75%) and vaccine stock management 58 (57%). However, most of the vaccine handlers reported to have not received training on vaccine and related supplies forecasting 9 (9%) and managing cold chain equipment inventory 4 (4%) (Table 5).

This was also discussed during KII. All district managers, reported that most vaccine handlers in health facilities have not received trainings. One stated:

“In my District, most of healthcare workers taking care of the vaccines (vaccine handlers) have not received the training because they are either new or they have been re-located (staff rotation) from other departments or facilities and they only receive some skills when we go for supportive supervision……” KII_06.

Knowledge It was reported that the knowledge on key principles and procedures of temperature management and vaccine stock management both had scored 70% and with least score on vaccine management 50 (49%) (Table 5). The same challenge was reported by all district managers that most of the vaccine handlers have low knowledge on vaccine management practices. One indicated:

“It is challenging as the staff when they come in the RCH (reproductive and child health clinics) especially with vaccination services we find they have low knowledge on vaccine management and hence we find improper vaccine management and even the refrigerator is not well maintained as we expect ……...” KII_05.

However, in temperature management most of the health facilities (HF) scored 90% to have not exposed vaccines to low damaging temperatures and only 76% reported to have not exposed vaccines to high temperatures.

Health facility operations by EVM criteria

Effective vaccine management operations were assessed on various attributes for performance as a result of practices undertaken in vaccine management. Table 6 shows scores for the practices on each EVM criteria.

Table 6 Operational score on the criteria

On temperature management, most of the health facilities scored 90% to have not exposed vaccines to low damaging temperatures but only 76% reported vaccines to have not been exposed to high temperatures, hence having a significant proportion of vaccines exposure to high temperature (Table 6).

With maintenance and repair of CCE, 92 (90%) of health facilities assessed reported to carry out repair for CCEs (Table 6).

Regarding stock management, most of the requirements had below 50% scores with the exception to stock records which had >  = 50% scores (Table 6).

Looking at vaccine management, despite of the health facilities being found with temperature alarms for low temperature that require to conduct shake test, none of the health facilities attempted to conduct a shake test and find if the vaccines can pass for use after exposure to freezing conditions (Table 6).

Information technology for vaccine data management

Three requirements out of six had scored above WHO benchmark. The available data management tools had scored above lowest acceptable WHO score with exception to vaccine stock record form (vaccine and related supplies ledgers) which had 51% and manual temperature monitoring form (Temperature monitoring charts) with 66% (Table 7).

Table 7 Information technology

The challenge on availability and use of updated data recording tools was also reported among district managers. One explained:

“…for the past three years and more now, the program has not provided printed updated stock records……we have been trying to improvise while we wait for the supply of updated tools. And as I said, some have not been using properly the temperature monitoring charts”. KII-08

Policies and procedures

Majority of the assessed health facilities missed the SOPs and hence the low scores of which the highest was 62% for vaccine vial monitor (VVM) followed by vaccine stock transactions. The lowest score was on SOPs for refrigerator maintenance which scored 12% (Table 8).

Table 8 Availability of standard operating procedures (SOPs)Vaccine management performance

The human resource capacity, availability and use of SOPs and data management tools as inputs categories to facilitate effective vaccine management practices have all met the WHO minimum standard for the temperature management and waste management. However, they had an average score for the maintenance and repair for not following the standard reporting procedures when the refrigerator need repair. The lowest scores were on stock management, annual forecasting and annual work planning score due to some of the records missed required fields, most of the health facilities had no vaccines forecasts and weak monitoring of workplans, respectively (Fig. 2).

Fig. 2

Vaccine management performance by criteria (n = 102)