Influenza viruses cause hundreds of thousands of deaths and millions of hospitalizations annually and remain a global health threat despite the first preventive vaccine being licensed for wider use in 1945 (Iuliano et al., 2018; Courville et al., 2022; Barberis et al., 2016). Influenza vaccines need to be administered annually and require regular updating to keep up with mutational antigenic drift (Khalil and Bernstein, 2022). A meta-analysis of data from 5 seasons (2010–2014) estimated the influenza vaccine effectiveness at 41% for preventing hospitalizations among adults when circulating influenza viruses were well-matched to the vaccine viruses. Vaccine efficacy is lower in adults aged 65 years or older compared to adults aged <65 years (37% vs. 51%, respectively) and drops substantially when there is a mismatch between vaccine strains and circulating strains (14% among elderly) (Rondy et al., 2017). Therefore, there is a strong need to develop improved influenza vaccines that are less susceptible to strain variation (drift or shift) and elicit a broader and more durable protective immune response in all recipients, especially in older adults (Khalil and Bernstein, 2022).

Since cellular immune responses are less impacted by mutational changes of the virus than humoral immune responses, there is a growing interest to supplement or even replace hemagglutinin in the vaccines with other, more conserved, influenza proteins that are less mutation-prone such as the nucleoprotein, matrix proteins and the polymerase complex (Janssens et al., 2022). Reliable assays are needed to measure the magnitude and quality of cell-mediated immune responses. Different immunological assays to monitor antigen-specific cellular immune responses are available and some are used more frequently than others (Janssens et al., 2022; Britten et al., 2008a).

Interferon-gamma Enzyme-Linked ImmunoSpot (IFN-ɣ ELISpot) and flow cytometry-based Intracellular Cytokine Staining (ICS) are suitable and well-accepted assays to measure the cellular immune responses induced by influenza vaccines. However, numerous variables in the execution of these assays lead to high variability in assay results and impede the comparability of data generated by different laboratories. A critical variable that determines the reliability of the test results is the quality of the sample, in particular the cell viability (Higdon et al., 2016; Nazarpour et al., 2012; Weinberg et al., 2009; Kierstead et al., 2007; Bull et al., 2007). Ideally, T cell responses generated in vitro perfectly mirror their behavior in vivo. T cell assays are known to be quite delicate and even minor changes in test conditions can have a significant impact on the test results (Zhang et al., 2009). Standardization of immunoassays remains difficult, especially when standards or reference materials are lacking (Britten et al., 2008a; Britten et al., 2008b). To harmonize assays between laboratories, it is key to identify those variables that have the most impact on assay performance. Harmonization efforts do not necessarily prevent laboratories to use their own reagents and instruments but will increase the validity and comparability of data generated across multiple laboratories, independent of their level of experience (van der Burg et al., 2011). There are signs of increased awareness of the gains of assay harmonization across several scientific fields, such as cancer and infectious diseases (e.g., HIV vaccine trials network) research communities (Britten et al., 2008b; Janetzki et al., 2008; Gill et al., 2010; Maecker et al., 2005a; Jaimes et al., 2011; Maecker et al., 2005b). Assay harmonization can already have a beneficial effect at an early phase of clinical development.

Additionally, commercial platforms providing External Quality Assessment (EQA) programs can assist individual labs in demonstrating their proficiency in performing immunoassays. However, these proficiency panels are scarce and often consist of small panels of test samples, making it difficult to accurately evaluate the performance of a laboratory.

Efforts have been made in the EU-funded FLUCOP project to develop harmonized Standard Operating Procedures (SOPs) to execute influenza-specific IFN-ɣ ELISpot and ICS assays (https://flucop.eu/, n.d.). Pilot studies were done to identify sources of variability during sample and data analysis. Assay variables considered to strongly induce variation were harmonized, such as the cell seeding density (ELISpot) and the minimal antibody panel (ICS). For other variables considered to have less impact (e.g., cell counting techniques, analytical instruments and certain reagents), either recommendations were proposed or sometimes no guidelines at all (Table 1). This allows for quite some flexibility and reflects the actual situation of multicentric studies today. Subsequently, both assays have been qualified to demonstrate their specificity, linearity and reproducibility. The results of these qualification efforts have been recently published (Waerlop et al., 2022; Begue et al., 2022).

This paper describes the final work performed in Work Package 2 in the FLUCOP consortium. A proficiency test or interlaboratory comparison was conducted in which the laboratories participated that also contributed to the process of assay harmonization. This allowed the participants to evaluate their proficiency in executing one or both CMI assays among a group of laboratories. At group level, this provided information regarding (Iuliano et al., 2018) the reliability of the data generated by the participating laboratories (i.e., how well samples can be distinguished from each other despite the presence of a measurement error), (Courville et al., 2022) the agreement (i.e., how close the values are for repeated measurements across labs) and (Barberis et al., 2016) the impact of harmonization by the introduction of an SOP. Ultimately, this allowed us to provide recommendations on how to harmonize and optimize assay performance to obtain reproducible and comparable data in a multi-laboratory setting. Furthermore, since the same panel of samples was used in both the IFN-ɣ ELISpot and ICS proficiency tests, both methods could be compared.