Introduction
CHDs represent the most common type of birth defect around the world. About one in every four newborns with a heart defect has a critical CHD. Reference Barreto1 In Mexico, a single-hospital study reported a CHD prevalence of 6.4 per 1000 live newborns in 2010. Reference Solano-Fiesco, Aparicio-Osorio and Romero-Ramírez2 In addition, the prevalence estimations of CHD and critical CHD were made based on global reports and the natality rate. Reference Jiménez-Carbajal, Pérez and Luna3,Reference Calderón-Colmenero, Cervantes-Salazar and Curi-Curi4 Nevertheless, there is still a lack of data available on how to determine the actual prevalence of CHD and critical CHD in Mexico.
By definition, a critical CHD will require both surgical and non-surgical interventions (e.g., prostaglandins, volume replacements, corrective or palliative surgery, and/or catheter interventions) in the first 28 days of life. Reference Barreto1 If a critical CHD is not detected perinatally, up to 43% of affected newborns will arrive at the emergency room with severe haemodynamic collapse, thus reducing the possibility for successful outcomes following a surgical intervention. Reference Mellander and Sunnegårdh5 This is consistent with the increased CHD mortality rate observed in recent years. One study reported that by 2016, the CHD mortality rate increased by roughly 25% compared to the previous statistics published in 1998; significant differences were observed between private and public healthcare, even among those living in rural and urban areas. Reference Torres-Cosme, Rolón-Porras and Aguinaga-Ríos6
There are three timely strategies that can be used to detect critical CHD, and that increase the possibilities for better outcomes: clinical examination, prenatal evaluations, and pulse oximetry screening. Historically, clinical examination represented the main detection strategy for heart defects. The presence of a heart murmur was usually key for detection. However, the prevalence of heart murmurs in newborns is low Reference Mahle, Newburger and Matherne7 between 0.6% and 4.2%, highlighting the importance of a more accurate screening method.
In developed countries, the prenatal detection rates for critical CHD are higher than 50%; for example, in Germany, the rate is 60%; in Ireland, 90%; in the United States, 50%; and in Canada, 50%. Reference Corcoran, Briggs and O’Connor8 In Mexico, prenatal diagnoses were shown to be feasible in specialised fetal cardiology centres, despite the challenges associated with expanding their use. This highlights the fact that performing diagnoses at these centres will not be a reality over the short-medium term. Reference Cruz-Lemini, Nieto-Castro and Luna-Garcia9 Even with these strategies in place, there is still an urgency to increase the detection rate of critical CHD.
Internationally, pulse oximetry screening has become a widely used strategy for detecting some types of critical CHD. Thus, the number of newborns identified prior to discharge has increased significantly. Evidence from the New Jersey study showed that even 60% of diagnoses among newborns with a critical CHD were solely attributable to their failed screening. Reference Garg, Van Naarden Braun and Knapp10 In addition, several studies demonstrated pulse oximetry screening as a cost-effective option for approaching them. Reference Kochilas, Lohr and Bruhn11–Reference Peterson, Grosse, Oster, Olney and Cassell14 In 2011, the United States of America health department included pulse oximetry screening for critical CHD in their Recommended Uniform Screening Panel. With its expansion worldwide, one of the most recent meta-analyses estimated that the prevalence of critical CHD cases detected by pulse oximetry was 6 in 10,000 presumably healthy, late-preterm and full-term newborn infants. Reference Plana, Zamora and Suresh15
Several algorithms have been proposed to evaluate newborns for critical CHD. The most widely used is often referred to as the American Academy of Pediatrics. Reference Kemper, Mahle and Martin16 Other variations are referred to as the New Jersey Reference Garg, Van Naarden Braun and Knapp10 and the Tennessee Reference Mouledoux, Guerra, Ballweg, Li and Walsh17 algorithms. The modifications differ slightly in terms of their sensitivity and specificity. Even small differences in these indicators greatly impact the workload of and follow-up by healthcare practitioners. Reference Hom, Martin and Oster18
Screening by pulse oximetry is a promising strategy; however, important decisions still need to be made to optimise screening, follow-up, and the associated steps necessary to confirm the diagnosis, as well as to increase the sensitivity and specificity of this approach for evaluation. Particularly, opportunities related to screening algorithm selection, peripheral oxygen saturation thresholds for different altitudes, medical practitioners’ misinterpretation of the algorithm, data transfer to a central system, use of a reliable database, standardisation of operations, and other prospects for further expansion of the programmes have been reported. Reference Oster, Aucott and Glidewell19 Additional limitations are present during pulse oximetry screening due to the physiological changes noted in the newborn’s circulatory system. Other factors affect the peripheral oxygen saturation, such as the motion artefact, nonessential human manipulation, and periods of desaturation during sleep, crying or feeding. Reference Mahle, Newburger and Matherne7 Most of these areas of opportunity and limitations reported previously in screening programmes can be surpassed to minimise the false-positive rate.
Critical CHD screening efforts have been described in single-hospital studies; however, both multicentre studies examining early detection experiences, and epidemiological monitoring studies are still needed. Our study aimed to share a preliminary 3-year experience in a multicentre pulse oximetry screening programme, which focused on the detection and diagnosis of critical CHD. We further emphasised the improvement made to our programme based on the opportunity areas previously described. Additionally, we compared two different algorithms implemented in a group of hospitals throughout the years, with the expectation that a reduction in the false-positive rate would be observed. As part of this analysis, we assessed the variability of this rate within the American Academy of Pediatrics screening period.
Materials and methods
Location
An independent critical CHD screening programme (Cárdi-k®) was implemented at the well-baby nurseries in hospitals forming part of CHRISTUS Muguerza Sistemas Hospitalarios S.A. de C.V. The results of five hospitals located below 600 m (1968 feet) above sea level in the northeast region of Mexico are reported.
Critical CHD screening programme
The screening programme focused on two areas of opportunity: operational standardisation and data management. In relation to the former, ongoing training and a uniform follow-up protocol were developed and became key aspects of the programme. Furthermore, a specialised device used for pulse oximetry screening was introduced to the screener’s practice and was essential throughout the programme implementation.
The screening device incorporated two pulse oximeters for simultaneous peripheral preductal and postductal readings. An operator-independent software within the device automatically selected and interpreted the peripheral oxygen saturation readings according to the algorithm. The New Jersey algorithm was used between February 2016 and July 2017, and the American Academy of Pediatrics algorithm from August 2017 until the present. This change was introduced as an attempt to reduce the number of false-positive results since the American Academy of Pediatrics algorithm requires only one extremity to be 95% or more for the peripheral oxygen saturation, while the New Jersey algorithm requires both extremities to have a reading of 95% or more.
The information obtained via the screening device (patient demographics and the screening results) was automatically transferred and stored in a central database accessed through a web-based platform, which had the capacity to inform the programme cardiologists, coordinators, and the newborn’s nursery directors. The newborns who failed the screening received immediate attention with clinical examination and/or transthoracic echocardiogram to reject the presence of a critical CHD.
This screening programme only included newborns eligible for pulse oximetry screening (i.e., those who were asymptomatic); thus, patients with clinical symptoms of critical CHD followed other protocols. Moreover, the follow-up and intervention data for those newborns diagnosed with a critical CHD remained independent of these registries and were unavailable to the authors.
Retrospective study
To evaluate the implemented multicentre critical CHD screening programme, we carried out a retrospective analysis, which included 9098 newborns screened from February 2016 to July 2019. The data included eligible reports from late-preterm and term (34–41 weeks’ gestational age by Capurro B), live-born, asymptomatic neonates (without a cardiac murmur, cyanosis, abnormal vital signs, or dysmorphic features) screened within 72 hours of birth in a well-baby nursery.
We used descriptive statistics to estimate the overall critical CHD prevalence and specificity for each algorithm: the New Jersey and the American Academy of Pediatrics algorithms. A chi-squared test (α = 0.05) was used to evaluate the relationship between the two algorithms in terms of their false-positive rate, as well as the variability within the American Academy of Pediatrics screening period.
For this study, two groups of reports were formed: the group of screening reports interpreted under the New Jersey algorithm and the reports interpreted under the American Academy of Pediatrics algorithm. The findings may be attributed to the algorithm or to other variables (e.g., experience acquired by the user, technology used, and the newborn’s physiological state). The proposed analysis aimed to reduce the impact of such factors by evaluating both algorithms in the same population. Thus, a reinterpretation of the peripheral oxygen saturation measurement was performed for the screening reports first interpreted under the New Jersey algorithm to the American Academy of Pediatrics algorithm criteria. This new group was then compared with the screening reports obtained before the reinterpretation.
Furthermore, to determine whether there was any variability within the group of screening –reports interpreted under the American Academy of Pediatrics algorithm–, three, 8-month periods were evaluated (Subperiod 1: August 2017–March 2018; Subperiod 2: April 2018–November 2018; Subperiod 3: December 2018–July 2019). This approach was proposed to evaluate false-positive rates within the American Academy of Pediatrics Group.
Results
Critical CHD screening programme
The approach of asking an independent unit to manage pulse oximetry screening in several centres represented a successful strategy to optimise and standardise the current practice.
It is worth mentioning that the screening process was implemented just after hearing screening to avoid operational difficulties (e.g., irritability). In addition, the use of screening devices featuring motion-tolerant pulse oximetry, with automated and simultaneous preductal and postductal collection of physiological data, reduced user bias and standardised their practice when screening. The operator-independent software allowed the data to be collected without further human manipulation, rendering its reliability. Moreover, the integration of the New Jersey and American Academy of Pediatrics algorithms in the device enabled accurate and rapid decision-making. In addition, the demographic and clinical data were transferred to the designated cloud server, sending notifications to the hospitals’ coordinators and specialists, promoting efficient communication and surveillance by healthcare professionals.
The registered nurses responsible for the newborns underwent initial training to learn more about how to identify the most appropriate time to conduct the screening. They were taught about the newborn’s physiological changes and the environmental conditions that may affect screening, the follow-up protocols, as well as the proper handling of the medical device. In addition, we leaned into a continuous and personalised training strategy, which identified areas for opportunity on a regular basis. The programme assured a transthoracic echocardiogram and/or clinical examination to every newborn who screened positive in order to provide timely interventions if needed.
Retrospective study
From February 2016 to July 2019, a total of 9098 newborns were screened (Fig 1). The population was categorised by gestational age and age at screening. In relation to the former, 0.24% (22) were identified as preterm, 7.46% (679) as late-preterm, and 92.30% (8397) as term. In addition, of those identified as term and late-preterm, 36.75% (3336) were screened before 24 hours, 61.96% (5624) were screened between 24 and 72 hours, and 1.28% (116) were screened after 72 hours.
Figure 1. Flow chart of screened newborns’ outcomes. The eligible population is delimited by dashed lines. CCHD = critical CHD; h = hours.
Our eligible population included asymptomatic, term and late-preterm newborns screened within 72 hours of birth. With respect to the eligible population, the average gestational age was 38.05 weeks (standard deviation: 1.09 weeks). Seventy-eight newborns were referred to and received an evaluation by a paediatric cardiologist or a neonatologist. Of these, 72.15% were screened out of the study by clinical examination, and the remainder underwent a transthoracic echocardiogram. Overall, six infants with critical CHD, six with CHD, and one with pulmonary arterial hypertension were found (Table 1). Therefore, the estimated prevalence of critical CHD detected by screening was 6.70 cases per 10,000 newborns (CI 95: 11.4–0.6). It is important to mention that the second patient listed in Table 1 had no clinical criteria that led to exclusion from the analysis despite the low peripheral oxygen saturation levels detected.
Table 1. CHDs and other findings detected via screening in asymptomatic term newborns
CCHD = Critical CHD.
Regarding the screening algorithms used to interpret the eligible population results, 32.27% (2892) of the newborns were screened under the New Jersey algorithm and 67.72% (6068) of the newborns were screened under the American Academy of Pediatrics algorithm (Table 2), obtaining a false-positive rate of 1% (CI 95: 0.64%–1.37%) and 0.71% (CI 95: 0.50%–0.92%), respectively.
Table 2. Comparison between the New Jersey and the American Academy of Pediatrics algorithms regarding their specificity, and a reinterpretation
After the statistical evaluation in which we compared the group of screening reports interpreted under the New Jersey algorithm and the reports interpreted under the American Academy of Pediatrics algorithm, we concluded that the difference in the false-positive rate was not related to the population (χ2 = 2.12, p = 0.14). Furthermore, to evaluate the impact of both algorithms on the same population –that is, to reduce the influence of various factors– we performed the comparison of the screening reports group interpreted under the New Jersey algorithm and the new group derived from the former (i.e., the screening reports between February 2016 and July 2017 reinterpreted under the American Academy of Pediatrics algorithm criteria). Our results showed that neither screening algorithm was related to the false-positive rate (χ2 = 0.47, p = 0.49). In other words, the peripheral oxygen saturation assessment of the screened population interpreted under the New Jersey algorithm and reinterpreted under the American Academy of Pediatrics algorithm resulted in false-positive rates with no significant differences. Also, the change in the screening algorithm over time did not result in an immediate reduction in the false-positive rate.
Even though no relationships were found among the analysed variables, a slight reduction in the false-positive rate was observed between the group of screening reports interpreted under the New Jersey algorithm and the reports interpreted under the American Academy of Pediatrics algorithm – Subperiod 1, consistent with the change in the screening algorithm. In addition, a moderate reduction in the false-positive rate was confirmed between Subperiods 1 and 2 within the reports interpreted under the American Academy of Pediatrics algorithm. This can be attributed to an intervention that was performed at the end of the first subperiod, which consisted of thorough training on probe placement and alignment, as well as recognition of the newborn’s conditions that might impact the peripheral oxygen saturation readings (e.g., motion, irritability, and body temperature). This training was carried out through clinical simulation workshops which promote the assimilation of knowledge. Moreover, visual aids were installed to reinforce the training.
During the critical CHD screening programme, there were no false-negative patients reported. This was due to the lack of long-term follow-up of the newborns after discharge. Nevertheless, the neonatologists and cardiologists who participated in this study were not informed of any further hospital admissions among the screened newborns, or of subsequent diagnoses of critical CHD. In addition, the results of this evaluation were obtained as part of an ongoing programme, where constant evaluations were performed to monitor and oversee this experience.
Discussion
Over the last decade, the implementation of screening programmes for critical CHD has been a priority for several worldwide health associations with the purpose of decreasing associated mortality. Reference Riede, Wörner, Dähnert, Möckel, Kostelka and Schneider20 Challenges are still present where we highlight the screening process, algorithms selection, and the need for ongoing training, data collection, and follow-up. Reference Mahle, Newburger and Matherne7,Reference McClain, Hokanson and Grazel21,Reference Mahle, Martin and Beekman22
Critical CHD screening programme
Activities performed during the critical CHD screening process in different programmes are prone to human error, leading to malpractice or yielding false-negative results. Particularly, the subjective selection of the appropriate fluctuant peripheral oxygen saturation value, Reference Oster and Kochilas23 the potential misinterpretation of the algorithm (representing near 18% of screening scenarios, as per previous reports), Reference Oster, Kuo and Mahle24 and the transfer of the peripheral oxygen saturation values by hand from the monitor to a physical or electronic record jeopardise the timely intervention for newborns with a critical CHD.
The subjective nature of value selection and the manual transfer of data for storage represent areas of opportunity to ensure that accurate dataset are maintained. Reference McClain, Hokanson and Grazel21 We based our practice on a specialised screening device where, once the pulse oximeters were placed on the newborn, the process was automated to collect, analyse, interpret, and communicate each result using a specialised screening device. In addition, in our case, the entire decision tree was integrated within the device, which was linked to each patient’s file. This computer-based tool combining the automated peripheral oxygen saturation value and interpretation may result in significant reductions in human error. Reference Oster, Kuo and Mahle24
Even though automated quality control activities enhance patient records, multicentre data collection, storage, and management are still a concern. We have demonstrated the feasibility of an automated system capable of extracting data from the device and transmitting it to a data centre. This system ensures that the information is readily available for further integration into an electronic medical record.
Thus far, the comprehensive, multicentre screening programme presented herein is the first of its kind to be reported in Mexico. This is relevant since remarkable limitations are commonly found in similar studies due to our socio-demographic context. Reference Sola, Rodríguez and Young25
Retrospective study
In a recent meta-analysis, the median prevalence of critical CHD was estimated as 6 out of 10,000 presumably healthy, late-preterm and full-term newborn infants detected by pulse oximetry. Reference Plana, Zamora and Suresh15 Using similar eligibility criteria and with reliable, standardised methods of data extraction, our study prevalence matched the median rate reported globally.
Internationally, the variations in the screening algorithms consisted of different peripheral oxygen saturation thresholds, the peripheral oxygen saturation differential between extremities, the number and time between re-screenings, the combination of pulse oximetry and clinical examination, and screenings conducted using the foot only or both hand and foot, among others. Reference Mahle, Newburger and Matherne7,Reference McClain, Hokanson and Grazel21,Reference Mahle, Martin and Beekman22 In our case, the study analysed two consecutive periods during which a different screening algorithm was implemented. The selected algorithms only varied in terms of the peripheral oxygen saturation threshold and misinterpretations arising from human error were significantly reduced with the use of a specialised device.
The variable that was chosen for comparative purposes was the false-positive rate. A lower false-positive rate leads to reduced health costs, delayed discharge, and a reduction in the emotional burden for the parents. The two periods were analysed to discover which factors significantly affected the false-positive rate: the algorithm or other variables associated with the screening practice (Table 3).
Table 3. P-values calculated for the chi-squared tests between groups
First, we analysed the algorithm’s impact. In spite of concluding that the difference in the false-positive rate was not related to the use of either the New Jersey or American Academy of Pediatrics algorithms, the percentage of false-positives was consistently lower for the American Academy of Pediatrics algorithm. This result is similar to the previously reported literature in which the New Jersey algorithm demonstrated a slight increase in the number of false-positive cases. Reference Hom, Martin and Oster18 Based on these conclusions, we suspect that there are other elements in the screening practice that have yet to be described, and which may have influenced the false-positive rate, more than the algorithm itself.
We believe that ongoing training is paramount when developing an efficient screening programme. Although we did not find a relationship between experience and false-positive rate (p = 0.28), we identified a tendency toward a reduction in the false-positive rate throughout the entire second period (Table 3).
At the end of the programme, we achieved a 0.50% false-positive rate in the five hospitals involved. Nevertheless, the false-positive rate is still high when compared with other reports (false-positive rate: 0.06% [95 CI]: 0.03%–0.13%; n = 19 studies). Reference Plana, Zamora and Suresh15 From our point of view, the main differences were the incorporation of specialised devices leading to standardisation, as well as the number of hospitals involved. The former led to enhancements and greater confidence in the critical CHD screening programmes, although we are uncertain about their impact on clinical practice and in false-positive rate, with the latter revealing that replicability is a challenge due to the high involvement of healthcare personnel.
Further, a significant number of newborns had clear signs of other mild diseases or physiological changes that explained their hypoxaemia. In our programme, clinical examination helped to exclude almost 78% of newborns. Therefore, we see this high number of newborns without a transthoracic echocardiogram as an area of opportunity to improve our programme’s protocol. The reason why the treating paediatrician tended to screen out newborns by clinical examination, and would not perform a transthoracic echocardiogram for all presumptive positives, was primarily economic, consistent with other screening programmes reports. Reference McClain, Hokanson and Grazel21
Conclusions
Taking into consideration the critical CHD prevalence rate identified in the present study, we can extrapolate it to northeastern Mexico, and this rate could serve as a surrogate value for further comparisons. In our case, the calculated prevalence rate was similar to those reported in other larger-scale studies adopting comparable inclusion criteria. Cárdi-k® previously addressed proposed challenges by incorporating new technology and strategies to enhance our programme’s efficiency and replicated it in five hospitals during an active screening programme. Two important aspects that need to be highlighted are the communication efficiency achieved using an integrated screening programme and the confidence in results attained by using the specialised device.
Our results suggest that whilst a change in the algorithm’s peripheral oxygen saturation value threshold is not related to the false-positive rate, other factors (i.e., medical practitioners’ misinterpretation of the algorithm, data transfer to a central system, use of a reliable database, physiological changes in the newborn’s circulatory system, motion artefact, and periods of desaturation during sleep, crying, or feeding) also play a fundamental role. To surpass these factors and achieve the lowest false-positive rate possible, we recommend that greater focus be placed on developing medical programmes that are automated and standardised, such as what was achieved through the web-based screening system presented herein, as well as the continuous training that was employed. Moreover, the uniformity of the screening process led to greater confidence in the screening results and facilitated data interpretation and follow-up for each newborn who required it.
The programme featured a significant number of newborns who did not undergo a transthoracic echocardiogram. We see this as an area of opportunity to improve our programme’s protocol. Ultimately, we aim to increase the percentage of all presumed positive newborns with a transthoracic echocardiogram evaluation and will encourage other programmes to change from a clinical examination-centered programme to a transthoracic echocardiogram-centred programme.
Acknowledgement
English-language editing of this manuscript was provided by Journal Prep Services. We thank Genomi-k’s team –in particular we thank Daniel Guerrero, Amira De Luna and Carolina Araiza– for all their support during this project.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflicts of interest
None.
Ethical standards
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Research Ethics Committee with registration number CONBIOETICA 19 CEI 011-21016-10-17.
Open access
