130 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43Biomédica 2019;39(Supl.2):130-43 Serosurveillance for vaccine-preventable diseases: A look inside the pertussis experience Doracelly Hincapié-Palacio1; Marcela Acevedo1, María Cristina Hoyos1, Jesús Ochoa1, Catalina González1, Paula Andrea Pérez1, Adriana Molina1, Blanca Isabel Restrepo2, Marcela Arrubla2, Adriana Patricia Echeverri2, Rita Elena Almanza3, Luz Denise González3, Eduardo Santacruz-Sanmartín3, Norma Elena Orrego3, Daniel Arango4, Aura María Gutiérrez5, Olga Lucía Londoño6, Luz Maribel Toro7, Mónica Ríos8, Oscar Villada9, Luz Aida Mejía10, Paola Andrea Arenas11, Diego García12, Elkin Osorio12 1 Seminario Taller de Epidemiología Teórica, Grupo de Epidemiología, Facultad Nacional de 2 Secretaría Seccional de Salud y Protección Social de Antioquia, Gerencia de Salud Pública, 3 Subsecretaría de Salud Pública, Dirección Técnica de Planeación, Secretaría de Salud de 4 Centro de Investigaciones, E.S.E., Hospital General de Medellín, Medellín, Colombia 5 Unidad de Investigaciones, E.S.E., Metrosalud, Medellín, Colombia 6 Subdirección Científica, E.S.E., Hospital San Vicente de Paúl de Caldas, Medellín, Colombia 7 Comité de Investigaciones, Clínica del Prado, Medellín, Colombia 8 Cirugía Ginecobstetricia, Sociedad Medica Antioqueña S.A., Soma, Medellín, Colombia 9 Unidad de Investigaciones, Hospital Universitario de San Vicente Fundación, Medellín, Colombia 10 Unidad de Investigaciones, Clínica El Rosario, Medellín, Colombia 11 Unidad de Vigilancia Epidemiológica y Control de Infecciones, Clínica Las Américas, Medellín, 12 Programa Ampliado de Inmunizaciones, Dirección de Promoción y Prevención, Ministerio de Introduction: Serological surveillance (serosurveillance) provides the most direct measure of herd immunity of vaccine-preventable diseases. Little is known about the opportunities and challenges of serosurveillance experiences, particularly pertussis. Objective: To describe the process of serosurveillance for vaccine-preventable diseases with an emphasis on the experience of pertussis in the metropolitan area of Antioquia (Valle de Aburrá) in 2015 and 2016 and analyze the contributions and challenges for its sustainability. Materials and methods: We described the planning and conduction of serosurveillance of pertussis antibodies of mothers and in the umbilical cord at the time of delivery in eight hospitals based on random sampling and their capacity to advance the serosurveillance periodically. We compared the contributions and the challenges of this experience with other probabilistic and non-probabilistic programs. Results: We achieved the participation of hospitals and mothers respecting the delivery care process. We established a serum bank following ethical and technical guidelines. This program based on the random selection of hospitals and mothers has enabled the estimation of antibodies prevalence in mothers and in the umbilical cord, which has been possible given the high coverage of hospital care during childbirth at a lower cost and fewer risks than a population-based survey in conflictive areas. The main challenges for the sustainability of this program are the creation of stable jobs and access to funding and legal and methodological long-term frameworks. Conclusions: Hospital serosurveillance as described is an option to monitor the impact of vaccination on the population. Our experience could be reproduced in other regions under similar conditions if the above-mentioned challenges are solved. Keywords: Pertussis; seroepidemiological studies; immunization; antibodies; immunity, herd; Colombia. Serovigilancia de enfermedades prevenibles por vacunación: una mirada desde la experiencia con la tosferina Introducción. La vigilancia serológica es la forma más directa de medir la inmunidad de rebaño frente a las enfermedades prevenibles por vacunación. Poco se sabe acerca de las oportunidades y los desafíos de las experiencias de serovigilancia, en general y, específicamente, la de la tosferina. Objetivo. Describir el proceso de serovigilancia de enfermedades prevenibles por vacunación con énfasis en la experiencia en el caso de la tosferina en el área metropolitana de Antioquia (Valle de Aburrá) en el 2015 y el 2016 y analizar lo que dicha experiencia ha aportado y los desafíos que persisten para su sostenibilidad. Original article doi: https://doi.org/10.7705/biomedica.v39i4.4181 Received: 08/02/18 Accepted: 31/01/19 Published: 06/02/19 Citation: Hincapié-Palacio D, Acevedo M, Hoyos MC, González C, Ochoa J, Pérez PA, et al. Serosurveillance for vaccine-preventable diseases: A look inside the pertussis experience. Biomédica. 2019;39(Supl. 2):130-43. https://doi.org/10.7705/biomedica.v39i4.4181 Corresponding author: Doracelly Hincapie-Palacio, Calle 62 N° 52-59, Medellín, Colombia Telephone: (574) 219 6827; fax: (574) 219 6830 doracelly.hincapie@udea.edu.co Author contributions: Doracelly Hincapié-Palacio, Marcela Acevedo, Maria Cristina Hoyos and Jesús Ochoa: Study conception and design Doracelly Hincapié Palacio and Marcela Acevedo: Drafting of the manuscript All authors participated in the collection and interpretation of the data and in the critical revision. Funding: Ministerio de Salud y Protección Social and Universidad de Antioquia, Agreements 313/ 2015 and 448/2016 Conflicts of interest: The authors declare no conflicts of interest. The Ministerio de Salud y Protección Social of Colombia participated in the discussion of the pertussis study design and in the review of this manuscript, which did not influence the analysis or the decision to publish the study results. Salud Pública “Héctor Abad Gómez”, Universidad de Antioquia, Medellín, Colombia Medellín, Colombia Medellín, Medellín, Colombia Colombia Salud y Protección Social, Bogotá, Colombia https://doi.org/10.7705/biomedica.v39i4.4181 https://doi.org/10.7705/biomedica.v39i4.4181 http://crossmark.crossref.org/dialog/?doi=10.7705/biomedica.v39i4.4181 131 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience Materiales y métodos. Se describió el proceso de planeación y el desarrollo de la serovigilancia de tosferina en el momento del parto en ocho hospitales seleccionados al azar, así como la capacidad para adelantar el programa de manera periódica. Se compararon los aportes y los desafíos en el curso de esta experiencia con los de otros programas poblacionales probabilistas e institucionales no probabilistas. Resultados. Se logró la participación de los hospitales y de las madres con pleno respeto del proceso de atención del parto, y se conformó un banco de sueros siguiendo lineamientos éticos y técnicos. El programa permitió estimar la prevalencia de anticuerpos en la madre y en el cordón umbilical, lo que se facilitó por la alta cobertura de atención hospitalaria del parto, a un menor costo y menos riesgos que los programas poblacionales en zonas conflictivas. Los principales desafíos para la sostenibilidad del programa son la estabilidad laboral del personal de salud, así como normas y una financiación de largo plazo. Conclusiones. La serovigilancia hospitalaria es una opción para monitorizar el impacto poblacional de la vacunación. Esta experiencia se podría extender a otras regiones en condiciones similares si se resuelven los retos mencionados. Palabras clave: tos ferina; estudios seroepidemiológicos; inmunización; anticuerpos; inmunidad colectiva; Colombia. Serosurveillance is a strategy of epidemiology that seeks to monitor the dynamics of disease transmission using serological tests that establish the interrelation of the prevalence or incidence of infection and the behavior of susceptibility or immunity in population groups (1). This can help to determine the level of antibodies required to achieve herd immunity, to identify groups of susceptible individuals, to evaluate the persistence and duration of protective antibodies, to determine the effectiveness of vaccination, to document the elimination of diseases, and to define the need to modify the primary schemes including reinforcements or modifying the frequency of their application (1-3). Serosurveillance was promoted in the 1960s by the World Health Organization (WHO) by establishing three reference serum banks based on the experience of seroepidemiological studies conducted since the 1930s and later promoted in the post-war period (4-7). These programs have been part of the vaccination programs; however, due to the reemergence of diseases, the conformation of a global serological bank has recently been proposed (8). The data used to monitor the elimination and control of diseases come from epidemiological surveillance and vaccination coverage, which can be limited by inaccuracies in the population estimation, low vaccination effectiveness, reduced duration of protection, changes in the sensitivity and specificity of surveillance over time, and presence of subclinical or subnotified cases, among other (9). The strategies used by serosurveillance are the determination of antibodies with samples for convenience from residual sera available in laboratories or the use of probabilistic population samples through seroprevalence surveys (10). In the present study, we analyzed the experience with pertussis serosurveillance using a probabilistic hospital approach in the metropolitan area (Valle de Aburrá) of Antioquia, identifying the capabilities, opportunities, and challenges for advances in the periodic and systematic use of this approach, and reviewing several publications on the subject. In Colombia, pertussis vaccination began in 1980 with the DPT vaccine (diphtheria, pertussis, and tetanus) using a three-dose schedule (2, 4, and 6 months) and two vaccine boosters (18 months and 5 years). Subsequently, with the resurgence of the disease between 2010 and 2012, vaccination of pregnant women with the acellular vaccine began in 2013 in Antioquia and Bogotá, which was later extended to the whole country (11). 132 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43 In Antioquia, the second most populated department of Colombia with more than 7 million inhabitants, we conducted a serosurveillance program in the metropolitan area (Valle de Aburrá) between 2015 and 2016, which estimated the prevalence of IgG PT antibodies against pertussis in mothers and the umbilical cord as a measure of antibody transfer. Additionally, the health status of the children was monitored during the first six months of life (12). The purpose of describing the serosurveillance experience was to facilitate the extension of this approach to other regions of Colombia and similar countries. Materials and methods We described the experience of planning and conducting pertussis serosurveillance in the metropolitan area of Antioquia (Valle de Aburrá) between 2015 and 2016 including the methodological aspects, ethics committee approval, disclosure, data collection and management, collection of blood samples, and custody and conservation of information. The results of the pertussis seroprevalence study are shown in another publication (12). The capabilities to consolidate a serosurveillance program in eight randomly- selected hospitals and the Public Health Laboratory of Antioquia were analyzed after collecting the information with a structured interview. A questionnaire was designed to be answered by those in charge of clinical and public health laboratories. It included the following variables: Availability of locative and technological resources for the reception, processing, and conservation of blood samples; availability of hospital beds for delivery and postpartum care; previous experiences in serosurveillance and processing of samples from vaccine- preventable diseases, and the identification of opportunities and challenges related to the implementation of serosurveillance as a regular and systematic program. We also analyzed the contributions and challenges regarding the orientation of vaccination and epidemiological surveillance of hospital-based serosurveillance in light of other published experiences of probabilistic and non-probabilistic serosurveillance programs. Results Description of the procedures for hospital serosurveillance The target population for the survey included mothers who came to the hospital for labor, were 37 weeks or more of gestation and agreed to participate in the study. We did not include mothers of multiple gestations, with fever in the previous 72 hours (chorioamnionitis and sepsis), treated at the intensive care unit, and in the advanced stage of labor. The study sample was made up of vaccinated and unvaccinated mothers against acellular pertussis during pregnancy (n=1,000). The size estimation of the sample was made according to the data of antibodies averages of vaccinated and unvaccinated pregnant women and their range or standard deviation according to reports from previous studies (minimum value of 2, maximum of 360, variance of 7,569 to 8,010, 95% confidence, and 80% potency) (13-15). The sample size was 500 vaccinated mothers and 500 unvaccinated mothers. However, due to the difficulty of recruiting unvaccinated mothers during the study period (two years after starting the vaccination of pregnant women against pertussis), unvaccinated mothers had to be recruited before vaccination and at the time of delivery in two of the randomly-selected hospitals. 133 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience The sampling was multi-stage and stratified by municipalities and by hospital conglomerates. The primary sampling units were hospitals in the municipalities of Valle de Aburrá. The secondary sampling units were all the mothers who came to hospitals for delivery care and met the inclusion criteria. The affixation of the sample was proportional to the childbirths attended within each hospital in the municipality. One municipality had to be excluded because the public hospital chosen randomly had to suspend the childbirth service due to financial deficits. Furthermore, two selected private hospitals could not participate because the childbirth services were in the process of closing down. For the random selection of hospitals, we considered the number of deliveries in 17 urban hospitals of the metropolitan area of Antioquia (Valle de Aburrá) and the data from nine municipalities of this sub-region including: Tdap vaccination coverage among pregnant women, incidence of pertussis, and distribution of the population according to the type of affiliation to the health social security system. After signing the informed consent, mothers were surveyed on basic aspects including sociodemographic factors, antecedents of pregnancy, childbirth, vaccination, and disease. Other information sources were the daily delivery books (manual or electronic depending on availability), the national electronic registry of the expanded program of immunizations (PAIWeb), and hospitals’ medical records. We prepared a procedure manual including a detailed description of participants selection and the procedure for obtaining informed consents; technical procedures for the randomization and identification of participants; data collection and completion of the survey; sample handling procedure and construction of the serum bank; follow-up of the newborn and the procedures related to the handling of the data, the analysis plan, and the mathematical modeling of prevention and control options. The proposal was endorsed by the Ethics Committee of the Facultad Nacional de Salud Pública “Héctor Abad Gómez” of the Universidad de Antioquia (session 129-15 Oct., 2015) and the ethics or research committees of the health institutions selected. We sought the support of the directorates or medical councils, the personnel in charge of epidemiological surveillance, the hospitals’ infection control units, the immunization programs, and the quality control and occupational health units. The number of birthing beds, delivery rooms, and postpartum beds varied according to the public or private nature of the health institutions (table 1). All health institutions had data from mothers on manual or electronic medical records. The characteristics of the information systems, the clinical history, and the equipment of specific areas are summarized in table 2. Table 1. Characteristics of hospitals participating in the pertussis serosurveillance, Valle de Aburrá, 2015-2016 Number Public Private Total I1 I2 I3 Total I4 I5 I6 I7 I8 Birthing beds Delivery care rooms Postpartum beds 22 8 54 6 3 24 10 3 20 6 2 10 57 17 135 16 2 16 18 5 75 10 3 12 5 5 24 8 2 8 134 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43 Table 2. Capabilities for serosurveillance according to the participating health institutions I1 I2 I3 I4 I5 I6 I7 I8 I9 Serum bank available No No No No, only serum storage for 15 days No, only serum storage for 15 days No, only serum storage for 30 days No No, only serum storage for 30 days Yes Blood bank program Yes No No No Yes No Yes No Yes Own laboratory headquarters Yes Yes No No No No Yes Yes No Processing samples with own resources Yes Yes Yes Yes Yes Yes No Yes Yes Hiring with another laboratory for the processing of samples Yes Yes No Yes No Yes No Yes No Analysis of samples for vaccine- preventable diseases Hepatitis B and C Rubella Rapid test for hepatitis B Rubella, but sending it to the reference laboratory Hepatitis B and C Rubella and chickenpox Hepatitis B and C Hepatitis D Rubella and chickenpox Hepatitis B, hepatitis C, hepatitis D Rubella, chickenpox and measles Hemophylus influenzae B Neisseria meningitidis A Neisseria meningitidis B Neisseria meningitidis C Streptococcus pneumoniae in LCR Bordetella pertussis Tuberculosis Hepatitis B chickenpox Hepatitis B Hepatitis A Rubella, chickenpox, measles and rotavirus Bordetella Tuberculosis Hepatitis B and C Rubella, chickenpox and tuberculosis Hepatitis A and B Rubella, chickenpox, measles and tuberculosis Meningitis Mumps Hiring for the processing of samples of vaccine- preventable diseases Yes, rubella Yes, measles mumps Yes, pertussis Measles Mumps Pertussis No Yes, rubella Pertussis No Yes, measles Mumps Pertussis No Availability of laminar flow cabin/biological safety cabin Yes, simple Yes, simple Yes, simple Yes Yes No Yes Yes Yes Air-conditioned locations Yes Yes Yes, partially Yes Yes Yes Yes Yes Yes Refrigerators available for specimen storage between 2 and 8 °C with temperature sensor and calibration plate (number of) Yes: 2 Yes: 2 Yes: 1 Yes: 5 Yes: 1 cold room Yes: 2 Yes, 1 cooling cellar Yes, 1 cooling cellar Yes: 8 Availability of refrigerators for reagent storage between 2 and 8 °C with temperature sensor and calibration plate (amount) Yes: 5 Yes: 2 Yes: 2 Yes: 8 Yes: 2 Yes: 1 Yes: 3 Yes: 2 Yes: 6 Availability of freezer at -20 °C with temperature sensor and with calibration plate (number of) Yes, 1 for reagents 1 for samples Yes, 3 freezers Yes, 2 freezers Yes, 1 cold room 1 freezer Yes, 1 freezer cellar Yes,1 freezer serum bank Yes, 1 cold room Yes, 1 cold room Yes, 3 freezers Availability of software and / or alarm systems for temperature monitoring Yes, systematized control with alarm: UBI 2 No, manual monitoring 2 times a day No, manual monitoring 2 times a day with digital thermometer Yes, satellite control with audible alarms and cell phone Yes, but it is also verified manually. No, manual record of the temperature, refrigerators with display for control The software is pending for implementation. No, reading the temperature on the digital panel (display) Yes, computerized control with software- mediated alarm No, manual monitoring 2 times a day 135 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience We took the samples for serosurveillance within the regular process of delivery care. We explored the procedures that were part of the route of inclusion and attention of mothers, as well as the technical laboratory procedures as explained by the health personnel (figure 1). We checked on the physical space, required inputs, and technical personnel involved in all the stages of the clinical and laboratory processes. In this way, we minimized the interference in the procedures already established in these places, ensuring the fluidity and continuity of the serosurveillance program. We disseminated the information on the disease, the vaccination, and the project using printed educational materials and a web page (www.epiteorica. com) addressed to health personnel, family, and visits. We provided information to the mothers only when they were willing to listen to it according to their progress in labor. After providing the information, we obtained the consent and assent of the underage mothers and we conducted a brief survey. It was not possible to obtain blood samples from the children due to ethical and cultural objections; for this reason, we only Recruitment of pregnant women at delivery Data collection Handling samples Data management Admission for delivery care Basic data survey Entry of samples to the department’s Laboratory Registration of samples in serum bank format Distribution of serum or plasma in three aliquots Quali-quantitative technical assembly SERION ELISA classic Bordetella pertussis Freezing at -25 °C, until analysis Report and delivery of sample results Cross check of received samples and shipping formats Processing and cleaning of data Creation and documentation of the database Individual report of results Data analysis - mathematical modeling - dissemination of findings Cross-checking of results database and results reporting software Quality control Data custody Marking cryovials identification labels Final marking of cryovials Assignment of serum bank key Blood sampling Conservation and shipment of samples Umbilical cord sampling during delivery Tube check and sample registration Immediate shipment of samples to the laboratory Centrifugation and refrigeration between 2 and 8 °C Shipping of samples to the department’s laboratory Blood sample taken from mother at delivery room Verification of inclusion criteria Invitation to participate Informed consent signature Obtention of basic anonymous data of non participants Accept End Yes No Figure 1. Process of collection and management of serosurveillance data 136 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43 carried out the clinical monitoring of the occurrence of pertussis during the first six months of age of the babies. The consent included the endorsement of later use of the sera collected for future investigations related to vaccine- preventable diseases. Besides, we obtained basic information from non- participating mothers. The reagents to process the samples were selected taking into account the highest sensitivity and specificity reported in the literature (16). To ensure traceability of samples in all processes after they entered the laboratory, data were recorded in terms of collection, centrifugation, temperature and cooling time, transport, fractionation, and final inclusion in the serum bank. In each specimen, a code was assigned according to its type (G if it was from the mother and C if it was a cord), fraction number (F1, F2, F3 or FU if the sample was a single fraction), position in the cryobox (from A1 to J10), and date of entry to the serum bank. Databases on the handling of samples from the serum bank, data on laboratory results (antibodies from the mother and the umbilical cord), and the survey of mothers and children followed for six months were created and combined as necessary. The results of the laboratory and the interpretation of their meaning were sent to each mother through a letter delivered by regular mail or with the support of the vaccination service staff in some cases. We defined the databases documentation, analysis, and custody procedures omitting personal identification information. The analysis took into account: a) The inference of the results of the sample to the population by estimating the inverse probability of being chosen and the weighing of variability at each stage of random sampling. This allowed us to estimate the confidence interval of proportions and weighted averages; b) the estimation of the trans-placental transport ratio as that of umbilical cord blood to maternal antibodies, and c) the mathematical modeling of disease-transmission dynamics by formulating a system based on differential equations of the relevant states and parameters and the deduction of the reproductive number that allows for estimating the critical proportion of immunity to achieve control, herd immunity or disease elimination. We systematized the data in Access™ (Microsoft) databases while surveys and consents were digitized for 15-year-period conservation. In the survey regarding the capacity of the institutions to implement a serosurveillance program, we observed that none of them had any program. All participating laboratories had an area for sample reception and fractionation and processing with adequate lighting and ventilation, as well as areas for the conservation of samples and reagents for both refrigeration and freezing under controlled temperature conditions. All had air conditioning, except Institution 3, which had partial flow (table 2). Only the department’s Public Health Laboratory had ultra-freezers at -70 °C with sensor and/or temperature monitoring using a system or specific software. However, the installed capacity of its three freezers was nearly 100% occupied. All laboratories met the requirement of regular preventive maintenance of refrigerators and freezers with calibration support. They also had a power regulator system (supply plant) to support electrical faults or interruption in the light current. 137 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience The participating institutions agreed on the aspects necessary for the implementation and strengthening of a serosurveillance program, such as training in the subject, centralization of the program in a reference entity, and dissemination of information highlighting the usefulness and importance of the program for evaluating the impact of vaccination and the epidemiological characterization of public health events. They also mentioned the need to have regulations to support custody, sample handling, and accreditation systems, as well as the provision of equipment and suitable assets to give solid support to the program and carry out cost-benefit analyses. All of them expressed interest in participating in the serosurveillance program. The main challenges identified were funding to ensure program sustainability and health personnel under indefinite-term contracts with sufficient time for program activities. Experiences of probabilistic population serosurveillance México, the Netherlands, and the United States have documented the contribution of serosurveillance based on random population studies (table 3). In 1987, México conducted a national seroepidemiological survey and implemented a national serum bank, although it had advanced pertussis seroprevalence studies since the 1960s (17,18). In 1995 and 1996, the Netherlands conducted a study to evaluate the national serosurveillance immunization program including the creation of the national serum bank; the study was subsequently replicated in 2006 and 2007 (19,20). In the United States, the National Health and Nutrition Survey (NHANES) conducted a serological study on antibodies against certain vaccine-preventable diseases in the 1960s and an annual program since 1999. This program received legal approval for data collection and management (The Public Health Service Act 42 USC 242k) (21-25). Based on these experiences, a population seroprevalence study was conducted in 2009 in Medellín, which estimated the prevalence of antibodies in the population (26-30) and obtained detailed information on the history of vaccination and the biological and social factors that could influence the presence of antibodies and guide vaccination policies based on the evidence of local epidemiology, as well as the use of mathematical modeling to estimate the critical proportion of immunity and the basic and effective reproductive number indicating the level of achievement of disease elimination (27,31). The percentage of participation in this population-based seroprevalence study was 88,5% (26) during a time of armed conflict in the city, which implied high costs, greater efforts and additional time. Experience with non-probabilistic institutional serosurveillance In 1986 and 1987, England and Wales implemented a convenience sampling to obtain sera to monitor changes in the prevalence of antibodies before and after the measles vaccination program (table 3) (5). In the 1990s, Australia considered serosurveillance as an important component of its surveillance system for vaccine-preventable diseases and in that same year, serum samples were compiled from associated laboratories to evaluate measles vaccination strategies (32,33) (table 3). In Europe, a serosurveillance network was formed in 1996 with the initial participation of six countries to standardize serological methods, which was started in some cases from residual samples available in laboratories and in others, from population studies (34) (table 3). A typical case was measles serosurveillance with samples obtained from the 1996-2000 and 2001-2003 periods in 18 countries in order to identify susceptible cohorts and adjust 138 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43 Table 3. Studies on serosurveillance experiences according to the approach used (random or not sampling) Approach Population / References Origin Vaccination orientation Guidance of surveillance Challenges Random population United States (21) (22) (23) (24) (25) NHANES began in the 1960s. Since 1999 it has been carried out annually. It provides evidence for the evaluation of the impact of vaccination against human papillomavirus, analyzing seropositivity in the pre- vaccination period and the association of annual seropositivity (subtypes 6, 11, 16, and 18), and the self-reported history of cancer from 2003 to 2010. It allows estimation of the change of seroprevalence (2003-2010), the impact on public health and cost effectiveness of vaccination against hepatitis A from 2006 to 2106, using data on seroprevalence, notification data, and a model of the dynamics of the transmission. Opportunity is required in the detection and communication of clinically relevant findings. Be alert to changes in methods or data management. This is updated in the documentation available on the internet. Some variables are only available through the Research Center due to confidentiality considerations. It is necessary to improve the comparability of the tests and analytical methods. The results are generalized to non- institutionalized individuals. Mexico (17) (18) They established a system of National Health Surveys in 1985 (General Directorate of Epidemiology, National Institute of Statistics, Geography and Informatics, state governments). The need to reinforce vaccination against measles was detected according to the distribution of susceptibility in children from 1 to 9 years old. The low seroprevalence of antibodies against poliomyelitis in children younger than 10 years in the southern region of the country allowed to identify the need to strengthen surveillance and vaccination in this location. Serological pertussis surveys were performed with difficulty in the comparison due to differences in the sample design, geographic extension, and number of samples (1965, 1966, 1973, and 1987-1990). Optimal use of collected samples (hemolysis, low volume, identification data not available) Simultaneous realization of 18 projects on sample processing. External control of the quality of the sample processing Verification of contamination-free samples with sterility tests Technical limitations in processing and analysis: Storage and processing capacity, availability of reagents, persistence of antibodies and opportunity for processing, cross-reactions, and storage conditions of samples and reagents Netherlands (19) (20) They formed a serum bank from cross- cutting studies in 1995/1996 and 2006/2007. Analysis of additional strategies in sampled populations: Immigrants and communities that reject vaccination for religious reasons. Available data on vaccination and associated factors Additional data on mathematical modeling of social contacts, genetics of the immune response and detection of food allergies and vaccines Expensive though efficient for studying a broad spectrum of diseases The response rate in the second survey was lower than in the first (55 vs. 33%); they have non-response data to analyze possible bias. Participation of the public health service recognized at the local level Support of clinics and regional administrators to access population records Medellín, Colombia (26-28, 30) A serosurveillance initiative was conducted based on a seroprevalence study in 2009 providing documentation of the elimination of rubella and during a mumps epidemic. The effect of vaccination against hepatitis B was evidenced by determining the increase of immunity vaccination in a birth cohort fifteen years after the beginning of the mass vaccination, the need to complete the vaccination scheme, and the factors potentially associated with the presence of immunity (natural and by vaccine). Estimated population immunity to achieve the elimination of rubella in men and women from 6 to 64 years of age residing in urban and rural areas through mathematical modeling of the proportion of susceptible individuals by age at the time of documentation of the degree of progress in the elimination of the disease The proportion of seronegativity and factors that could influence the occurrence of outbreaks of mumps were identified in an epidemic season. Fieldwork was conducted in conditions of insecurity due to the resurgence of violence. With the support of community leaders, participation exceeded 85%. The limitations in the funding of reagents did not allow for simultaneous processing of the available vials. The freezing-thawing of the vials was avoided. The analysis of the recommendations to be given to each seronegative individual for rubella and parotiditis and individuals with chronic hepatitis was required before the ethical commitment of explanation and delivery of laboratory results. The Ministry of Health supported the vaccination activities and epidemiological surveillance of these individuals. 139 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience Residual samples England and Wales (5) The first serological survey was conducted before the beginning of measles, mumps, and rubella (MMR) vaccination in 1988. Addition of the second dose of MMR in 1996; suggested vaccination campaign in adults against MR; documented decision not to implement universal vaccination against hepatitis B, and introduced booster doses against diphtheria and tetanus in 1994. Construction of a stock of sera to analyze other diseases (rubella, parotiditis, chickenpox, hepatitis A, parvovirus B19, Norwalk virus, etc.). Evaluation of the effect of the initial scheme and vaccination campaigns against measles Use of mathematical models to predict the occurrence of a measles epidemic Complementary data on the incidence of hepatitis B Rapid detection of the diphtheria emergency after the epidemic in the former USSR Control of over-representation of women by prenatal care in the random selection of residual serum. No data were available on the ethnic group or country of birth. The selection bias was reduced because the health system has free access. Reduction of costs and discomfort in obtaining samples Australia (32) (33) The first survey was conducted in 1999 to evaluate the effect of the measles vaccination campaign before and after its implementation in 1998. A cohort of young adults susceptible to measles was identified and a campaign of MMR was conducted in 2001. In initial surveys, no impact on the immunity of measles, mumps, diphtheria, and tetanus was observed, but they recommended innovative and additional strategies in varicella and whooping cough. Understanding of epidemiology from mathematical modeling of varicella transmission parameters and estimation of the probability of occurrence of measles epidemics Need to strengthen surveillance in the case of a change of inactivated polio vaccine Identification of some risk of hepatitis B in the northern region and natives of Asia It may be more difficult to identify and control potential biases because the characteristics of participants and non-participants are unknown. They tried to reduce the selection bias capturing samples from outpatients rather than hospitalized patients. They obtained samples selected randomly by age group rather than by state or territory. They observed similar results from random population sampling and residual samples except for diseases with oversampling due to increased uptake by diagnosis (e.g., hepatitis C). The cost of collecting and preserving samples for each test in random sampling was about seven times more expensive than the equivalent in samples for convenience. A decrease in the participation of laboratories in subsequent surveys by fusion or disappearance of laboratories, limitations of time availability, financing, and change of priorities vaccination strategies accordingly. Also, these data were compared with the incidence reported in the surveillance and vaccination coverage. The analysis revealed that seven countries had a high susceptibility prevalence and low vaccination coverage, which led to the outbreaks in some of these countries (35). Discussion The uses of serosurveillance have been widely analyzed in the literature in terms of their contributions to vaccination and epidemiological surveillance programs and it is considered the most reliable strategy to monitor the population impact of vaccination (8,9). The advantages and disadvantages of population and institutional serosurveillance approaches have been analyzed extensively (10,33). In this study, we used a hospital approach, which had the advantage of obtaining estimates of the seroprevalence from the mother and the umbilical cord during labor and from probabilistic samples whose data can be inferred from the target population. NHANES: National Health and Nutrition Examination Survey 140 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43 In our experience, the percentage of participation in population-based seroprevalence studies was high despite the resurgence of the armed conflict, which made household surveys risky. Providing detailed information on the usefulness of the study through different means such as household surveys, the use of newsletters, and telephone calls fostered participation. The participation of pregnant women in the determination of antibodies against pertussis at the time of delivery and the follow-up of the newborn was achieved using "active" recruitment strategies (36), such as the involvement of the family and the health personnel, showing respect for the process of childbirth care, the willingness of hospitals to participate, the support of the health secretaries, and the documentation of the research experience undertaken by the university team in charge of the project through different means (web page and others). We also reviewed experiences of non-probabilistic institutional serosurveillance generally based on residual samples available in laboratories or blood banks from health services users. This approach requires less time, effort and investment than the population approach but it has limited possibilities of population inferences given its convenience sampling. This study reported on the experience of a hospital serosurveillance approach during 2015 and 2016 based on the combination of the institutional approach and random selection of participating hospitals. This approach had the advantage of optimal handling of the samples in the hospitals’ laboratories and their subsequent transfer to a public health laboratory for processing. In addition, the risks of health personnel in charge of collecting the data areas affected by armed confrontations were reduced, as well as the costs and the time required for fieldwork. The existence of hospital admission records allowed for the analysis of differences between participating and non- participating mothers to identify and control a potential selection bias (12). The mathematical modeling constructed in local seroprevalence studies allowed to take advantage of the data collected in their course and in epidemiological surveillance for identifying disease control and elimination options (27,37). However, several challenges remain and should be faced to advance in this type of initiatives. First, it is necessary to have a regulatory framework to support serosurveillance as a complement to epidemiological surveillance of vaccine-preventable diseases. Second, the hiring of indefinite term health personnel is required to ensure the sustainability of the serosurveillance program and the continuity of the activities at all levels. Third, training on the usefulness of serosurveillance and the construction of strategies for resolving difficulties is also required. These challenges relate to the organization of health services provision. During the serosurveillance, we observed how some obstetric and gynecological services shut down or were reduced, as well as financial deficits which affected the quality of childbirth-related attention. Hospital serosurveillance can be affected by health services coverage. Nevertheless, the coverage of hospital care during childbirth has increased (38), which may reduce this limitation. Other sources of information on hospital serosurveillance, such as blood banks, allowed us to estimate only the prevalence of diseases associated to the risk of transmission via transfusion and depend on the access to and the acceptance of health services. The sampling at the time of delivery is preferable to collect residuals derived from prenatal control such as umbilical cord samples, which is a practice allowed. 141 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience Ethical considerations restricted the collection of umbilical cord samples from the children, but they were monitored during the first six months of life for pertussis (12). Due to budgetary restrictions, we processed only pertussis antibodies, although three vials of serum were available. Other studies have analyzed the transfer of multiple diseases antibodies, such as diphtheria, tetanus, and meningitis (19,39), as well as the utility of influenza (40) and chickenpox (41) serosurveillance. This study aimed at the identification of serosurveillance capacity in the participating institutions and the creation of a serum bank, as currently there are around 5,000 specimens available for further analysis, which is close to the adequate number required for monitoring population immunity in other countries (19) and responded to the current tendency of studying antibodies transfer from the mother to the umbilical cord or the newborn as recommended in mass immunization or epidemics management strategies, especially when seroprevalence or serosurveillance programs exist (42,43). This paper described the serosurveillance activities carried out in 2015 and 2016 in the participating institutions, but given that epidemiological and vaccination strategies vary with time, they should be repeated in five or ten years to monitor changes in population immunity against diseases associated with transplacental transfer of antibodies (20). Serosurveillance can guide vaccination and prevent the occurrence of epidemics and the resurgence of diseases at local and global levels (8). The experience documented in this work regarding hospital serosurveillance programs could be replicated in other places under similar conditions as an opportunity to strengthen public health laboratories and integrate vaccination and surveillance programs of vaccine-preventable diseases. Acknowledgments We thank the support of the participating mothers and families and of the administrative and assistance staff at all institutions. We express our special recognition to the members of the fieldwork and administrative teams for phases I and II: Julián Echeverry, Teresa Mutumbajoy Tandioy, Diana Castiblanco B., Marta Cadavid, Consuelo Gallego Carmona, Carmen Emilia Ramírez, Mayra Milena Jiménez, Tatiana Palacio, Natalia León, Jennifer Pulgarín, Olga Pérez, Paola A. Rueda, Laura Rivera S, Laura Castro, Paula Tuberquia, Jacqueline Palacios, Alexandra Porras, Vilma Salazar, Nelly Berrío V, Catherine Volcy, Lina Ospina, Patricia Abad A, Luisa Muñoz, Tatiana Franco, and Andrés Blandón. References 1. Domínguez A, Salleras L. Encuestas seroepidemiológicas. Manual de vacunas de pediatría. Madrid: Comité Asesor de Vacunas; 2008. p. 106-17. 2. Ochoa AR. Técnicas inmunoenzimáticas para ensayos clínicos de vacunas. La Habana: Finlay Ediciones; 2013. p. 65-75. 3. Cardeñosa-Marín N. Estudios seroepidemiológicos. Revista Española de Salud Pública. 2009;83:607-10. 4. Paul JR. Serological epidemiology and the function of serum banks. Arch Gesamte Virusforsch. 1965;17:465-71. 5. Osborne K, Gay N, Hesketh L, Morgan-Capner P, Miller E. Ten years of serological surveillance in England and Wales: Methods, results, implications and action. Int J Epidemiol. 2000;29:362-8. 6. Magos C, Sánchez F, Gutiérrez G, Tapia R. Banco nacional de sueros. Salud Pública de México. 1992;34:136-47. 142 Hincapié D, Acevedo M, Hoyos MC, et al. Biomédica 2019;39(Supl.2):130-43 7. Evans A. Surveillance and seroepidemiology. In: Alfred E, editor. Viral Infections of Humans: Epidemiology and control. New York and London: Plenum Medical Book Company; 1983. p. 43-64. 8. Metcalf CJ, Farrar J, Cutts FT, Basta NE, Graham AL, Lessler J, et al. Use of serological surveys to generate key insights into the changing global landscape of infectious disease. Lancet. 2016;388:728-30. https://doi.org/10.1016/S0140-6736(16)30164-7 9. Cutts FT, Hanson M. Seroepidemiology: An underused tool for designing and monitoring vaccination programmes in low- and middle-income countries. Trop Med Int Health. 2016;21:1086-98. https://doi.org/10.1111/tmi.12737 10. Wilson SE, Deeks SL, Hatchette TF, Crowcroft NS. The role of seroepidemiology in the comprehensive surveillance of vaccine-preventable diseases. CMAJ. 2012;184:E70-6. https://doi.org/10.1503/cmaj.110506 11. Ministerio de Salud y Protección Social, Instituto Nacional de Salud. Lineamiento estratégico para la introducción de la vacuna DPaT (Difteria - tos ferina acelular -tétanos) en el esquema del Programa Ampliado de Inmunizaciones - PAI para mujeres gestantes de las cohortes 2013 y 2014 Colombia, 2013. Bogotá: Ministerio de Salud y Protección Social; 2013. p. 29. 12. Hincapié-Palacio D, Hoyos MC, Ochoa J, Montoya N, García D, Osorio E. Effect of maternal immunization against pertussis in Medellín and the metropolitan area, Colombia, 2016-2017. Vaccine. 2018;36:3984-91. https://doi.org/10.1016/j.vaccine.2018.05.020 13. Hardy-Fairbanks AJ, Pan SJ, Decker MD, Johnson DR, Greenberg DP, Kirkland KB, et al. Immune responses in infants whose mothers received Tdap vaccine during pregnancy. Pediatr Infect Dis J. 2013;32:1257-60. https://doi.org/10.1097/INF.0b013e3182a09b6a 14. Plans P, Jansa J, Doshi N, Harrison TG, Plasencia A. Prevalence of pertussis antibodies in umbilical cord blood samples in Catalonia, Spain. Pediatr Infect Dis J. 2008;27:1023-5. https://doi.org/10.1097/INF.0b013e318179264b 15. Nooitgedagt JE, de Greeff SC, Elvers BH, de Melker HE, Notermans DW, van Huisseling H, et al. Seroprevalence of Bordetella pertussis infection during pregnancy measured by IgG antibodies against pertussis toxin. Clin Infect Dis. 2009;49:1086-9. https://doi.org/10.1086/605575 16. Riffelmann M, Thiel K, Schmetz J, Wirsing von Koenig CH. Performance of commercial enzyme-linked immunosorbent assays for detection of antibodies to Bordetella pertussis. J Clin Microbiol. 2010;48:4459-63. https://doi.org/JCM.01371-10 [pii] 10.1128/JCM.01371-10 17. Magos-López C, Sánchez-Villarreal F, Gutiérrez G, Tapia-Conyer R. Banco Nacional de Sueros. Salud Pública México. 1992;31:136-47. 18. Hernández-Ávila M. Resultados de serología de la Encuesta Nacional de Salud 2000. Salud Pública de México. 2007;49:321-3. 19. van der Klis FR, Mollema L, Berbers GA, de Melker HE, Coutinho RA. Second national serum bank for population-based seroprevalence studies in the Netherlands. Neth J Med. 2009;67:301-8. 20. De Melker HE, Conyn-van Spaendonck MA. Immunosurveillance and the evaluation of national immunization programmes: A population-based approach. Epidemiol Infect. 1998;121:637-43. 21. Zipf G, Chiappa M, Porter KS, Ostchega Y, Lewis BG, Dostal J. National health and nutrition examination survey: Plan and operations, 1999-2010. Vital Health Stat 1. 2013;56:1-37. 22. Introcaso CE, Dunne EF, Hariri S, Panicker G, Unger ER, Markowitz LE. Prevaccine era human papillomavirus types 6, 11, 16 and 18 seropositivity in the U.S.A., National Health and Nutrition Examination Surveys, 2003-2006. Sex Transm Infect. 2014;90:505-8. https://doi.org/10.1136/sextrans-2013-051490 23. Liu B, Taioli E. Associations between human papillomavirus and history of cancer among U.S. adults in the National Health and Nutrition Examination Survey (2003-2010). Br J Cancer. 2014;111:1448-53. https://doi.org/10.1038/bjc.2014.414 24. Kruszon-Morán D, Klevens RM, McQuillan GM. Change in hepatitis A seroprevalence among U.S. children and adolescents: Results from the National Health and Nutrition Examination Survey 2003-2006 and 2007-2010. Vaccines (Basel). 2013;1:105-19. https://doi.org/10.3390/vaccines1020105 25. Dhankhar P, Nwankwo C, Pillsbury M, Lauschke A, Goveia MG, Acosta CJ, et al. Public health impact and cost-effectiveness of hepatitis a vaccination in the United States: A disease transmission dynamic modeling approach. Value Health. 2015;18:358-67. https://doi.org/10.1016/j.jval.2015.02.004 https://doi.org/10.1016/S0140-6736(16)30164-7 https://doi.org/10.1111/tmi.12737 https://doi.org/10.1503/cmaj.110506 https://doi.org/10.1016/j.vaccine.2018.05.020 https://doi.org/10.1097/INF.0b013e3182a09b6a https://doi.org/10.1097/INF.0b013e318179264b https://doi.org/10.1086/605575 https://doi.org/JCM.01371-10 [pii] 10.1128/JCM.01371-10 https://doi.org/10.1136/sextrans-2013-051490 https://doi.org/10.1038/bjc.2014.414 https://doi.org/10.3390/vaccines1020105 https://doi.org/10.1016/j.jval.2015.02.004 143 Biomédica 2019;39(Supl.2):130-43 Serosurveillance from the pertussis experience 26. Hincapié-Palacio D, Lenis-Ballesteros V, Ospina MO, Toro OL, Díaz FJ. Seroprevalence of rubella in Colombia: A birth-year cohort analysis. Rev Saúde Pública. 2013;47:1080-91. https://doi.org/10.1590/S0034-8910.2013047004749 27. Hincapié-Palacio D, Ospina-Giraldo J, Lenis-Ballesteros V, Ospina-Ospina MC, Arroyave- Cadavid M, Hoyos-Muñoz N, et al. Inmunidad colectiva contra la rubéola según una encuesta poblacional en Medellín, Colombia. Rev Panam Salud Pública. 2012;32:101–8. https://doi.org/10.1590/s1020-49892012000800003 28. Santacruz-Sanmartín E, Hincapié-Palacio D, Ospina MC, Pérez-Toro O, Bernal-Restrepo LM, Buitrago-Giraldo S, et al. Seroprevalence of mumps in an epidemic period in Medellín, Colombia. Vaccine. 2015;33:5606-12. https://doi.org/10.1016/j.vaccine.2015.08.088 29. Cadavid D, Hincapié-Palacio D, Ospina M, Bernal L, Buitrago S, Pérez O, et al. Status for hepatitis B virus infection and socioeconomic variables: A multiple correspondence analysis. J Viral Hepat. 2014;21:36-7. https://doi.org/10.1111/jvh.12333_27 30. Cadavid-Betancur DA, Ospina MC, Hincapié-Palacio D, Bernal-Restrepo LM, Buitrago- Giraldo S, Pérez-Toro O, et al. Seroprevalence of hepatitis B and factors potentially associated in a population-based study in Medellín, Colombia. Vaccine. 2017;35:4905-12. https://doi.org/10.1016/j.vaccine.2017.07.084 31. Hincapié-Palacio D, Ospina-Giraldo J, Gómez-Árias RD, Uyi-Afuwape A, Chowell-Puente G. Simulating measles and rubella elimination levels according to social stratification and interaction. Rev Salud Pública (Bogotá). 2010;12:103-15. 32. Gidding H. Australia’s national serosurveillance program. N S W Public Health Bull. 2003;14:90-3. https://doi.org/10.1071/NB03027 33. Jardine A, Deeks SL, Patel MS, Menzies RI, Gilbert GL, McIntyre PB. An evaluation of the Australian National Serosurveillance Program. Commun Dis Intell Q Rep. 2010;34:29-36. 34. Osborne K, Weinberg J, Miller E. The European Sero-Epidemiology Network. Euro Surveill. 1997;2:29-31. 35. Andrews N, Tischer A, Siedler A, Pebody RG, Barbara C, Cotter S, et al. Towards elimination: Measles susceptibility in Australia and 17 European countries. Bull World Health Organ. 2008;86:197-204. https://doi.org/10.2471/BLT.07.041129 36. Sutton EF, Cain LE, Vallo PM, Redman LM. Strategies for successful recruitment of pregnant patients into clinical trials. Obstet Gynecol. 2017;129:554-9. https://doi.org/10.1097/aog.0000000000001900 37. Ospina J, Hincapié-Palacio D. The critical proportion of immune individuals needed to control hepatitis B. Proceedings of SPIE. 2016;9863:1-10. https://doi.org/10.1117/12.2222129 38. Departamento Administrativo Nacional de Estadística. Nacimientos Bogotá: DANE. Accessed on: October 20, 2017. Available from: http://www.dane.gov.co/index.php/ estadisticas-por-tema/salud/nacimientos-y-defunciones/nacimientos/nacimientos-2015 39. de Voer RM, van der Klis FR, Nooitgedagt JE, Versteegh FG, van Huisseling JC, van Rooijen DM, et al. Seroprevalence and placental transportation of maternal antibodies specific for Neisseria meningitidis serogroup C, Haemophilus influenzae type B, diphtheria, tetanus, and pertussis. Clin Infect Dis. 2009;49:58-64. https://doi.org/10.1086/599347 40. Laurie KL, Huston P, Riley S, Katz JM, Willison DJ, Tam JS, et al. Influenza serological studies to inform public health action: Best practices to optimise timing, quality and reporting. Influenza Other Respir Viruses. 2013;7:211-24. https://doi.org//10.1111/j.1750-2659.2012.0370a.x 41. Tafuri S, Gallone MS, Gallone MF, Cappelli MG, Chironna M, Germinario C. Evaluation of a vaccination strategy by serosurveillance data: The case of varicella. Hum Vaccin Immunother. 2015;11(4):897-900. https://doi.org/10.1080/21645515.2015.1009818 42. Barkoff AM, Grondahl-Yli-Hannuksela K, He Q. Seroprevalence studies of pertussis: What have we learned from different immunized populations. Pathog Dis. 2015;73. https://doi.org/10.1093/femspd/ftv050 43. Gonik B, Puder KS, Gonik N, Kruger M. Seroprevalence of Bordetella pertussis antibodies in mothers and their newborn infants. Infect Dis Obstet Gynecol. 2005;13:59-61. https://doi.org/10.1080/10647440500068289 https://doi.org/10.1590/S0034-8910.2013047004749 https://doi.org/10.1590/s1020-49892012000800003 https://doi.org/10.1016/j.vaccine.2015.08.088 https://doi.org/10.1111/jvh.12333_27 https://doi.org/10.1016/j.vaccine.2017.07.084 https://doi.org/10.1071/NB03027 https://doi.org/10.2471/BLT.07.041129 https://doi.org/10.1097/aog.0000000000001900 https://doi.org/10.1117/12.2222129 http://www.dane.gov.co/index.php/estadisticas-por-tema/salud/nacimientos-y-defunciones/nacimientos/n http://www.dane.gov.co/index.php/estadisticas-por-tema/salud/nacimientos-y-defunciones/nacimientos/n https://doi.org/10.1086/599347 https://doi.org//10.1111/j.1750-2659.2012.0370a.x https://doi.org/10.1080/21645515.2015.1009818 https://doi.org/10.1093/femspd/ftv050 https://doi.org/10.1080/10647440500068289 _ENREF_1 _ENREF_2 _ENREF_3 _ENREF_4 _ENREF_5 _ENREF_6 _ENREF_7 _ENREF_8 _ENREF_9 _ENREF_10 _ENREF_11 _ENREF_12 _ENREF_13 _ENREF_14 _ENREF_15 _ENREF_16 _ENREF_17 _ENREF_18 _ENREF_19 _ENREF_20 _ENREF_21 _ENREF_22 _ENREF_23 _ENREF_24 _ENREF_25 _ENREF_26 _ENREF_27 _ENREF_28 _ENREF_29 _ENREF_30 _ENREF_31 _ENREF_32 _ENREF_33 _ENREF_34 _ENREF_35 _ENREF_36 _ENREF_37 _ENREF_38 _ENREF_39 _ENREF_40 _ENREF_41 _ENREF_42 _ENREF_43