Multiple breath washout quality control in the clinical setting
Bettina S. Frauchiger1*, Julia
Carlens1,2*, Andreas Herger3,
Alexander Moeller 3, Philipp
Latzin1, Kathryn A. Ramsey1#
1. Pediatric Respiratory Medicine, Department of Pediatrics,
Inselspital, Bern University Hospital, University of Bern,
Freiburgstrasse 8, 3010 Bern, Switzerland†
2. Clinic for Paediatric Pneumology, Allergology and Neonatology,
Hannover Medical School, Carl-Neuberg-Strasse 1, 30625 Hannover, Germany
3. Division of Respiratory Medicine and Children’s Research Center,
University Children’s Hospital Zurich, Zurich, Switzerland †
† Institutions where research was mainly conducted
* Co-first authors: Authors contributed equally
# Corresponding author:
Kathryn Ramsey
Inselspital, Bern University Hospital,
Freiburgstrasse 8, Bern 3010 Bern, Switzerland
Email:
Kathryn.ramsey@extern.insel.ch
Funding: This project was funded by the Swiss National Science
Foundation, Grant Nr. PZ00P3_168173 (Latzin) / 1; 32003B_182719
(Ramsey)
Keywords: Multiple breath washout, lung clearance index, quality
control, cystic fibrosis, children
Running head: Quality control for multiple breath washout
Wordcount (without abstract): 3270 words
Abstract: 220 words
Author contributions: BF, JC, PL, AM, and KR were responsible for the
conception and design of this study. BF, JC, and KR drafted the quality
control guidelines and the structured implementation in clinical
routine. Data acquisition was conducted by BF and AH. BF, JC, AM, PL,
and KR were responsible for data interpretation. Statistical analysis
was conducted by BF and KR. BF, JC, AM, PL, and KR drafted the
manuscript and all authors revised and approved the manuscript for
intellectual content before submission.
Abstract
Background: Multiple breath washout (MBW) is increasingly used in the
clinical assessment of patients with cystic fibrosis (CF). Guidelines
for MBW quality control (QC) were developed primarily for retrospective
assessment and central overreading. We assessed whether real-time QC of
MBW data during the measurement improves test acceptability in the
clinical setting.
Methods: We implemented standardized real-time QC and reporting of MBW
data at the time of the measurement in the clinical pediatric lung
function laboratory in Bern, Switzerland in children with CF aged 4-18
years. We assessed MBW test acceptability before (31 tests; 89 trials)
and after (32 tests; 97 trials) implementation of real-time QC and
compared agreement between reviewers. Further, we assessed the
implementation of real-time QC at a secondary center in Zurich,
Switzerland.
Results: Before implementation of real-time QC in Bern, only 68% of
clinical MBW tests were deemed acceptable following retrospective QC by
an experienced reviewer. After implementation of real-time QC, MBW test
acceptability improved to 84% in Bern. In Zurich, after implementation
of real-time QC, test acceptability improved from 50% to 90%. Further,
the agreement between MBW operators and an experienced reviewer for test
acceptability was 97% in Bern and 100% in Zurich.
Conclusion: Real-time QC of MBW data at the time of measurement is
feasible in the clinical setting and results in improved test
acceptability.
1.
Introduction
The lung clearance index (LCI) derived from the multiple breath washout
technique (MBW) is sensitive to detect early lung disease in patients
with cystic fibrosis (CF) 1-4. With the availability
of commercial MBW devices, LCI is increasingly being used as an outcome
in routine clinical surveillance 5-10. While MBW
testing requires minimal cooperation from the subject, an acceptable
test requires relaxed tidal breathing and a leak-free
system11, which can be challenging in young children
and individuals with respiratory disease12. Besides,
prospective quality control (QC) of MBW measurements can be challenging
in the busy clinical setting.
Quality control guidelines for MBW focus primarily on retrospective
analysis and central overreading of MBW measurements by experienced
users for research studies and clinical trials11,13-15. The 2013 European Respiratory Society (ERS)
and American Thoracic Society (ATS) consensus statement for inert gas
washout measurements proposed initial recommendations for testing
procedure and technical acceptability criteria 11.
Further to this, ATS published additional guidelines for the preschool
age group 14. Jensen et al. proposed
comprehensive guidelines for retrospective quality control of MBW
measurements, which involved both qualitative and quantitative criteria
for trial grading and acceptability13. These
guidelines were further implemented in a standardized MBW training and
quality control platform for central overreading in clinical trials15. However, for LCI to be used as a clinical outcome,
prospective reporting of acceptability and test results is required for
clinical decision making.
Therefore, we aimed to implement prospective, real-time quality control
of MBW measurements in the clinical pediatric lung function laboratory
in Bern, Switzerland. The first aim was to evaluate the acceptability of
clinical MBW measurements in children with CF before and after the
implementation of real-time quality control. The second aim was to
assess the implementation of real-time quality control of MBW
measurements in a pediatric lung function laboratory with less
experience in MBW testing, Zurich, Switzerland. The third aim was to
evaluate agreement in MBW test acceptability between the operator and a
retrospective reviewer.
2.
Methods
2.1
Development of MBW quality control criteria
The quality control criteria used in this study was based on the ATS/ERS
consensus statement guidelines, ATS pre-school MBW technical statement,
and the publications by Jensen et al and Saunders et al.11,13-15. We used these guidelines to create a
simplified matrix for qualitative assessment of clinical Nitrogen
(N2) MBW measurements that can be applied at the time of
the measurement and did not require any further retrospective
assessment.
Our quality control criteria are presented in Table 1 and details of how
our criteria differ from the ERS/ATS consensus statement are provided in
Supplemental Table E1. Detailed instructions on how to apply the
guidelines are presented in the online supplemental. An A grade
represents a perfect trial with relaxed, regular tidal breathing
throughout the measurement, a B grade represents a good quality trial
with only minimal deviations, and a C grade represents an acceptable
trial with moderate deviations but no highly abnormal breaths during the
pre-phase or start of washout. A, B and C grade trials are considered
acceptable for outcome reporting. D grade represents trials with
questionable quality due to variable breathing patterns, abnormal
breaths, or evidence for hypo- or hyper-ventilation. D grade trials have
no signs of leak and satisfy both the start and end of test criteria.
Generally, D grade trials should be rejected and not used for reporting,
however, sometimes the deviations in tidal breathing in a D trial do not
significantly impact the primary outcomes. Therefore, we propose that D
grade trials can be accepted if the primary outcomes (LCI and FRC) are
within 10% of an acceptable trial (A, B or C grade). An F grade
represents trials that need to be rejected due to not meeting the
technical acceptability criteria for MBW: 1) Start of test criteria not
met (last three breaths of pre-phase N2-concentration
with normalized N2 concentration ≥ 77%); 2) End of test
criteria not met (three consecutive tidal breaths with normalized
N2 concentration < 2.5%; 3) No evidence of
leaks (for detailed instruction see online supplemental).
The overall test occasion is classified as acceptable when (i) at least
two trials are graded as acceptable (A, B or C), or (ii) one trial was
graded as acceptable (A, B, or C) and one trial was graded as
questionable (D) given that both FRC and LCI are within 10% of the A-C
grade trial when there are only two trials or 10% of the median when
there are three or more trials for this test occasion. A test occasion
with only D trials should be rejected. We used the overall test
repeatability criteria described in the consensus document (i.e. FRC
variability within 25%)11. MBW outcomes from
acceptable and repeatable test occasions are reported as the mean from
all acceptable trials.
2.2 MBW data collection and study
population
The N2MBW measurements were collected using the
Exhalyzer D device (Ecomedics, Duernten, Switzerland) with Spiroware
software (version 3.2.1) and were performed according to international
guidelines 11 in both centers. We approached all
pediatric patients with CF attending their regular three monthly
outpatient clinic visits aged 4 to 18 years. Approval was obtained from
the local ethics committee in Bern. Patients and caregivers gave
informed consent.
2.3 Test acceptability before
implementation of real-time MBW quality control in
Bern
Before implementation of real-time MBW quality control criteria into
clinical routine in our centre, MBW operators were trained in data
collection and general test acceptability. However, due to time
restrictions, they were not required to perform a detailed assessment of
test quality during the measurement. They also did not routinely mark
trial classification on the lung function reports. To assess the quality
of these MBW measurements, 31 clinic visits from children with CF aged
five to 18 years were evaluated. The visits were randomly selected by an
independent person not involved in this study and only one visit per
patient was included in the analysis. Retrospective quality control was
performed by an experienced reviewer who was involved in the development
of the criteria and was blinded to any test comments by the MBW
operator. The reviewer graded each trial individually and then assessed
overall test acceptability.
2.4 Test acceptability after
implementation of real-time MBW quality control in
Bern
To implement real-time quality control of MBW measurements, operators in
our center received instruction on how to perform quality control. A
printed copy of the quality control criteria matrix was provided and
operators were given a presentation on how to use the matrix, grade
individual trials, determine test acceptability and repeatability, and
report outcomes (detailed information provided in online supplement).
All MBW operators were required to perform real-time quality control of
each MBW measurement. The operators reported a grade for each trial and
provided a standardized comment regarding the acceptability of the test
occasion in the clinical report (example provided in the online
supplement).
To assess the quality of MBW measurements after implementation of
real-time quality control in routine clinical testing, 32 clinic visits
from the same population of children with CF were evaluated. The visits
were randomly selected by an independent person not involved in this
study and only one visit per child was assessed. The experienced
reviewer performed retrospective quality control of these measurements
while being blinded to the real-time quality control assessment of the
operator.