Abstract Background A low-cost real-time Respirator Seal Integrity Monitor (ReSIM) was recently developed to monitor a respirator’s actual performance at a workplace. The objective of this study was to evaluate the capability of the new ReSIM prototype in manikin-based laboratory experiments to rapidly detect induced leakage of a half-mask elastomeric respirator. Methods Two phases of testing were conducted in this study. First, the accuracy of ReSIM measuring an aerosol concentration was assessed by comparing the outputs of ReSIM against a reference optical aerosol spectrometer (OAS) in a flow-through set-up. Second, the capability to detect a leak was tested using a manikin-based set-up to simulate leaks into a functional respirator. Results: The regression curve of ReSIM versus OAS had an R2 of 0.936, indicating its high accuracy within the targeted particle size range of 0.5–2 µm. The ReSIM provided a leak detection sensitivity (probability of correctly identifying intervals with the true leak) of 98.4% when challenged with a combustion aerosol, compared to 71.8% when challenged with a NaCl aerosol. Its specificity (probability of identifying intervals without a leak) was 99.8% after adjusting for persistent false positives for both types of challenge aerosol. Conclusion The ReSIM prototype not only can estimate the particle concentration with high accuracy but also can rapidly detect respirator faceseal leakage in real time with sufficient sensitivity and specificity. In addition, it can trigger an alarm when the faceseal integrity is compromised. aerosol, elastomeric respirator, faceseal leakage, low cost, manikin, real-time monitoring, sensitivity, sensor, specificity Introduction The US National Institute for Occupational Safety and Health (NIOSH)-certified respirators are required for workers by the US Occupational Safety and Health Administration (OSHA). Effective respiratory protection can prevent or reduce exposure to airborne contaminants in a workplace when effective engineering controls are infeasible or while they are not yet implemented (OSHA, 1999). To ensure adequate protection for wearers, respirators are certified by meeting NIOSH certification requirements. NIOSH classifies particulate respirator filters (not the entire protection device) by their minimum filtration efficiency against a specified test aerosol. Filter designations of 95, 99, and 100 correspond to minimum filtration efficiencies of 95, 99, and 99.97%, respectively (NIOSH, 1995). The filter performance alone does not determine the respirator’s total inward leakage, which accounts for penetrations from all paths: filter (PFilter), faceseal leakage (PLeakage), and secondary paths, e.g. an exhalation valve (NIOSH, 2004). Furthermore, it is widely recognized that the particle penetration through faceseal leakage may be greater than penetration through the filter medium (Coffey et al., 1998; Zhuang et al., 1998; Grinshpun et al., 2009; Rengasamy and Eimer, 2012; He et al., 2013). Therefore, the protection efficiency of a respirator is largely dependent on how well it fits on the wearer’s face. This is particularly true for tight-fitting respirators such as elastomeric half-mask and full-face respirator because they are equipped with highly efficient filters (e.g. P100). For a tight-fitting respirator wearer, the fit (determined by the tightness of the facepiece seal) is verified through OSHA fit testing standard (29 CFR 1910.134) (OSHA, 1999). In quantitative fit testing, e.g. if using the PortaCount Respirator Fit Tester (TSI Inc., Shoreview, MN, USA), the test outcome is a fit factor (FF) calculated as the ratio of aerosol concentrations measured outside and inside of the respirator donned on a worker. While both the filter efficiency test and the fit test are important for evaluating a respirator before use, however, neither of those is able to predict a respirator’s actual performance at a workplace. The FF value is, perhaps, more relevant, but it is based on a set of specific exercises and utilizes ambient or NaCl particles as challenge aerosol; thus, a fit test may not adequately represent the true respiratory protection for a worker performing actual work activities (the latter is defined as a workplace protection factor, WPF). Studies have demonstrated either weak or insignificant correlation between FFs and WPFs with correlation coefficients (R) lying between 0.38 and 0.55 (Han, 2002; Zhuang et al., 2003). A true WPF is often difficult and costly to measure. Nevertheless, several investigators have generated meaningful WPF databases or determined simulated WPFs. For instance, Dietrich et al. (2015) and Vo et al. (2015) measured real-time simulated WPFs using condensation particle counters while other investigators quantified the real-time fit using two TSI PortaCount units in a simulated workplace setting (Duling et al., 2007; Hauge et al., 2012). The quoted studies, however, utilized expensive and complex instruments that have a limited usage in routine field assessment efforts. The National Academies called for ‘incorporating sensors into PPE to detect breaches and notify users of the end of service life and other protection information’ (IOM, 2008). Interest in a low-cost method for real-time monitoring of respirator performance under workplace conditions has been increasing recently. Such a method would be very useful for rapid detection of respirator failures and alerting the wearer. Low cost is essential for widespread adoption. A novel Respirator Seal Integrity Monitor (ReSIM) capable of detecting the respirator performance failure in real time has been recently developed and pilot-tested (Liu et al., 2017). The objective of this study was to evaluate the newly developed ReSIM prototype under controlled laboratory conditions. ReSIM concept Elastomeric half-mask and full-face respirators are widely used by firefighters during fire overhaul operations; other workers populations also extensively use these respirators (Tannahill et al., 1990; Bolstad-Johnson et al., 2000; Janssen and Bidwell, 2007). These respirators generally provide a high level of protection against various aerosol particles. Previous studies have suggested that the collection efficiency of a respirator filter increases as the particle size increases for aerosol particles above a certain size (varying from 0.1 to 0.3 µm, depending on several parameters), (Stevens and Moyer, 1989; Hinds, 1999). Penetration of relatively large particles (≳0.5 µm) through a typical high-efficiency filter such as P100 is extraordinarily low and usually undetectable (Martin and Moyer, 2000; Rengasamy et al., 2009). If these large particles penetrate an elastomeric respirator in any significant quantity, one could reasonably expect that this occurred due to a failure in respirator fit. Even if a worker passed fit testing prior to entering an occupational environment, the faceseal may be compromised during workplace activities. Therefore, it would be useful to monitor the aerosol concentration inside the respirator during work activities with a portable sensor capable of detecting relatively large aerosol particles (≳0.5 µm). If these particles are detected above an established threshold, the integrity of the respirator faceseal is considered to be compromised. Particles in excess of ~0.5 µm can be successfully detected and enumerated utilizing optical particle measurement principles. Fortunately, recent developments in aerosol instrumentation have led to low-cost, ultra-small optical particle sensors that can be integrated for in-respirator monitoring. It is important for a sensor to have an appropriately low particle concentration detection threshold, which allows detecting particles that penetrated through the P100 filter. This threshold can be established as a reasonable reference point determined by PFilter to represent the case of a perfectly fit respirator. Additionally, the sensor should provide accurate measurements in a wide range of particle concentrations up to the level at which a faceseal leakage presents, PLeakage. This represents the case of a compromised faceseal (improper fit) respirator. Besides the sensor (which is placed outside of a respirator), a leak-monitoring device should have an air sampling inlet and sampling line for aspirating and transporting particles from the respirator cavity. The sampling flow rate should be chosen to minimize its interference with the wearer’s breathing, provide adequate aspiration conditions, make the particle losses inside the sampling line negligible, and minimize the time between drawing sample air from within the respirator and analyzing it in the ReSIM sensor. The sampling tube diameter and length, together with the ReSIM’s air pump flow rate, combine to control the time required to draw a sample from the mask to the ReSIM sensor. Decreasing the sampling tube diameter and length or increasing the sampling air flow rate decreases the time between particles entering the respirator and being sampled by ReSIM. If the sampling delay is allowed to become too large, particle concentrations could grow to unacceptable levels before being detected. Other requirements include small size, low cost, and ease of use. Materials and methodology ReSIM prototype The ReSIM prototype built in our laboratories weighs 0.8 kg; all the components are assembled in a plastic box (16 × 12 × 6 cm), including a particle sensor PPD60PV-T2 produced by Shinyei, Kobe, Japan (Shinyei Technology, 2017), a data acquisition and processing core, an air pump with a flow rate of 1 l min−1, a sampling tube, supporting electronics, and an alarm system (Fig. 1). The battery can last for an 8-h work shift. Detailed components descriptions can be found elsewhere (Liu et al., 2017). The PPD60PV-T2 utilizes optical light scattering principles and is able to detect particles of ~0.5 µm and larger. It was proven to consistently provide measurement with high accuracy and sensitivity to different types of particles, but particle-specific calibrations are generally needed to convert the sensor output to particle concentration (Wang et al., 2015; Sousana et al., 2017). Using an inexpensive commercially available sensor and low-cost peripheral parts (~$500 for the current prototype, production unit is estimated to be below $200), ReSIM has a great potential for commercialization. Figure 1. View largeDownload slide ReSIM prototype: (A) front view and (B) back view. Figure 1. View largeDownload slide ReSIM prototype: (A) front view and (B) back view. ReSIM data processing The PPD60PV-T2 sensor does not output a direct count of particles. Instead, while detecting particles, the sensor generates a low electrical voltage pulse as particles pass through the sensing unit. The particle count is determined by counting the number of pulses generated during a given time period. It is possible for more than one particle to be in the detection zone simultaneously, low voltage pulses correspond more closely to particle counts at low concentrations. As the particle concentration increases, the correlation between pulse counts and concentration rapidly decreases. Consequently, it is clearer to express findings using the percentage of time the sensor detects particles during a reporting interval (30 s), which will be referred as pulse occupancy ratio (%) (also referred as occupancy ratio) from here on. Over a 30-s sampling period, the ratio of the total duration of all such low voltage pulses to the overall sampling time (the ‘occupancy ratio’) serves as a measure of the number of particles detected. Therefore, the output of ReSIM, occupancy ratio (%), is directly correlated with the aerosol particle concentration. The direct correlation between occupancy ratio and particle concentration will only apply when particle concentrations are sufficiently low. A 3-step leak detection algorithm was developed in the prototype development phase. There are three components in this algorithm. First, the aerosol concentration inside a ‘perfectly fit’ (fully sealed) operating respirator is defined as a background level. Then, threshold values for each ReSIM data processing interval are calculated based on 1.2 times the rolling average of the previous five background intervals not flagged as leaks or a lower limit occupancy ratio value of 2%, whichever is higher. Last, any concentration above the threshold constitutes a leak and that interval is not included in the rolling average. Note that the threshold formula is optimized based on the actual test results and can be modified when test conditions change. The entire calculation algorithm is programmed to the data processing component of ReSIM. The data acquisition core records these outputs and processes the data (writing data to storage, end of interval processing, and the beginning of interval processing) in real time and indicates leak detection via an LED warning light. While the data set is processed, ReSIM stops detecting particles for approximately a second at the end of every 30-s interval. The interval data can be downloaded to a computer at any time. Determination of ReSIM accuracy The ReSIM accuracy level was determined by comparing the outputs of ReSIM against a reference stationary optical aerosol spectrometer (OAS) using a polydisperse sodium chloride (NaCl) aerosol in a flow-through set-up shown in Fig. 2. Accuracy is assessed by how close the concentration predicted by ReSIM’s occupancy ratio output (using regression-based approach) follows the actual particle concentration measured with the reference instrument, OAS. Figure 2. View largeDownload slide Schematic diagram of flow-through experimental set-up. Figure 2. View largeDownload slide Schematic diagram of flow-through experimental set-up. Experimental design and set-up In this phase of the study, ReSIM was challenged with NaCl aerosol generated by a 6-jet Collison nebulizer. The pressure of the nebulizer was adjusted to generate different levels of aerosol concentration. A solution of 2% NaCl by weight was prepared by dissolving 1 g of NaCl in 50 ml of deionized water. Ambient air was driven by a pump (U21, Dayton Electric Manufacturing Co., Chicago, IL, USA), cleaned with high-efficiency particulate air (HEPA) filter and then mixed with the challenge aerosol. The diluted aerosol was driven by a second pump of the same model and passed through the horizontal flow chamber (L = 80 cm, D = 11.5 cm) with a flow rate of 6 l min−1. The aerosol particle number concentration was measured with ReSIM and a Grimm portable OAS (model 1.108, Grimm Technologies, Inc., Douglasville, GA, USA) as the reference instrument. The two operated in parallel. The Grimm spectrometer measures fractional particle number concentrations within the ‘optical size range’ from 0.3 to 20 µm in 15 channels. The operational range of the ReSIM is approximately from 0.5 to 2 µm. To perform an appropriate comparison, the Grimm aerosol concentration data were recorded only for channels corresponding to particle diameters in the range of 0.5–2 µm. Test protocol The aerosol concentration was controlled by adjusting the air flow rate of the two pumps and the adjustable valve at the outlet of the Collison nebulizer input. In the comparison effort, we generated the aerosol at eight different concentration levels and managed to maintain a steady state aerosol flow for at least 5 min at each level. Measurements were conducted 1–2 min after a steady aerosol concentration in a particle size range of 0.5–2 µm was achieved. Evaluation of ReSIM for detecting respirator faceseal leakage To evaluate the ReSIM capability to rapidly detect respirator performance failures, the device was tested in a manikin-based set-up that enabled simulating faceseal leaks. While ReSIM performed continuous air sampling inside a respirator which was donned on a manikin headform, an operator opened a specially designed solenoid-controlled valve intermittently, thus creating a small faceseal leak for predetermined time intervals, and then closed it. To mimic workplace leakage scenarios, leaks of variable durations ranging from 5 to 20 s were simulated. Since leaks longer than 20 s have been proved to be easily detected in an earlier stage of the prototype development and pilot testing, here we focused only on shorter leak durations. Experimental design and set-up Figure 3 shows the manikin-based experimental set-up housed inside a 24.3 m3 exposure chamber. The main elements include an aerosol generation system, a respirator donned on a manikin headform, a breathing simulator connected with a tube fitted through the manikin headform, an orifice and a solenoid valve to simulate a respirator faceseal leakage, a reference OAS, and a ReSIM device. Figure 3. View largeDownload slide Schematic diagram of manikin-based experimental set-up. Figure 3. View largeDownload slide Schematic diagram of manikin-based experimental set-up. In these experiments, two types of challenge aerosols were generated in the chamber: NaCl aerosol was generated using a model 8026 particle generator (TSI Inc., Shoreview, MN, USA), and combustion aerosol was generated by burning sheets of paper towel (23 × 24 cm brown multifold paper towel, 2.1 ± 0.2 g). NaCl was chosen because it is a standard, non-toxic, challenge aerosol widely used for the evaluation of respirators under laboratory conditions (NIOSH, 1995) and for respirator fit testing (OSHA, 1999). Combustion aerosol was chosen to represent a real workplace exposure scenario, e.g. when firefighters are exposed to high concentrations of combustion particles at different stages of firefighting, primarily during fire overhaul (Baxter et al., 2010) when firefighters switch to elastomeric half-mask or full-face respirators from their self-contained breathing apparatus (Bolstad-Johnson et al., 2000; Burgess et al., 2001). An elastomeric half-mask respirator (model: 6800, 3M Corp., MN, USA) equipped with two NIOSH-certified P100 filters (model: 2091, 3M Corp., MN, USA) was fully sealed on a manikin headform using acrylic latex and silicone sealant. The manikin was connected to a breathing simulator through a stainless-steel pipe passing through the manikin’s oral cavity. Cyclic breathing was simulated by a breathing recording and simulation system (BRSS, Koken Ltd, Tokyo, Japan). A HEPA filter and a flow meter were placed between the headform and BRSS. The HEPA filter kept particles from re-entering the respirator during the exhalation cycle, while the flow meter assured a stable air flow rate. The tests were performed at mean inspiratory flow (MIF) rates of 30, 60, and 85 l min−1. Those flow rates were selected to represent breathing at low, moderate, and heavy workloads, similar to numerous previous investigations (Haruta et al., 2008; Rengasamy and Eimer, 2011; Gao et al., 2015; Mahdavi et al., 2015). A 5-mm diameter orifice on the faceseal was utilized to induce a leak. It was controlled by a solenoid valve (AB21012, CKD Corporation, Komaki, Aichi, Japan). Opening the valve allowed outside air to enter the respirator during a specific time interval (5, 10, 15, or 20 s) to simulate faceseal leaks of different durations. The respirator remained fully sealed when the valve was closed. Both ReSIM and the reference aerosol spectrometer were connected to the same sampling probe drawing from inside the respirator. The spectrometer was used to confirm that a faceseal leakage was successfully established and maintained over a predetermined period and, on the other hand, that no large particles (within the optical range) penetrated while the respirator was fully sealed. Test protocol The test order is shown in Fig. 4. The challenge aerosols were generated in the chamber for at least 15 min before starting a new test session to achieve a spatially and temporally uniform distribution. A fan was placed inside the chamber to mix the particles and help to achieve a uniform spatial distribution. Aerosol generation continued during testing. The aerosol concentration in the targeted particle size range (0.5–2 µm) was measured in the chamber with the OAS three times: 2 to 5 min before, in the middle of testing, and immediately after testing. While the OAS measured the aerosol concentration in the chamber, we disconnected the in-mask sampling tube. After the in-mask sampling tube was reconnected to the mask, the OAS was operated at least 1 min to clean the residual particles before in-mask concentration was recorded. Each duration of the leak (5, 10, 15, and 20 s) was simulated at least six replicates in random order. Since a continuous leak brings a high number of particles to the respirator cavity and therefore can be easily detected, only intermittent leaks were simulated to determine the lower concentration threshold of the ReSIM. There was at least a 2-min interval between any two test runs with a specific leak duration. The same set of tests was conducted under the three chosen flow rates (30, 60, and 85 l min−1). A timer was used to monitor at which data processing interval the leak was induced. Additional sets of 10-s leaks were intentionally timed to split the 10-s leak across two ReSIM 30-s data processing intervals. This condition is referred as the 10-s cross leak. The purpose of this test was to assess the chance of a small scale faceseal leakage to be detected by ReSIM when the leak occurred over two consecutive intervals. Figure 4. View largeDownload slide Diagram of the leak detection test protocol. The testing was repeated at three flow rates. Figure 4. View largeDownload slide Diagram of the leak detection test protocol. The testing was repeated at three flow rates. A cross leak presents a challenging case for ReSIM to detect the leak because detected particles are divided into two consecutive intervals. Under these conditions, each interval has fewer particles to detect than normal and uneven splits of particles between intervals can easily reduce pulse occupancy below detection thresholds. In addition, the monitor does not measure particles for the entire 10 s of the cross leak, it loses about 1 s for data processing in each interval as indicated above. Therefore, the likelihood of a true cross leak exceeding the leak threshold of ReSIM is further reduced. Data analysis The data collected on the ReSIM accuracy were analyzed by using a simple regression model. The dependent variable was ReSIM’s occupancy ratio output, whereas the independent variable was the actual particle concentration measured with the reference instrument, OAS. Signal detection theory (SDT), which was developed and widely applied in the fields of mathematical statistics and electronic communications (Marcum, 1948; Peterson et al., 1954), was used to assess the performance of ReSIM as an electronic alarm-type detector. According to SDT, there are two ways of evaluating the ReSIM with respect to its ability to detect leaks: to determine its sensitivity and specificity as defined below. An interval of ReSIM during which a true leak is induced is defined as true positive, whereas an interval without any leak is defined as true negative. For the latter, measurements with the Grimm OAS provided confirmation that (i) an interval is free of induced leaks and (ii) no leak occurs at any point in the sampling train. First, false negatives may occur when ReSIM’s response is negative (particles are not detected) while a leak is present. The probability of ReSIM correctly identifying its intervals with true leaks (i.e. true positives) is defined as statistical sensitivity: Sensitivity=True Leak (TL) TL+False negative (FN)=True Positive (TP) TP+False negative (FN)×100% Second, false positives may occur when ReSIM indicates a positive response (particles are detected) in the absence of a leak. The probability of ReSIM correctly identifying an interval without leaks (i.e. true negative), e.g. with a fully sealed respirator, is defined as statistical specificity: Specificity=True negative (TN)TN+False postive (FP)×100% Thus, the sensitivity and false negatives were calculated for all ReSIM intervals with true leaks, and specificity and false positives were calculated for all ReSIM intervals without a leak. The calculations were carried out for each combination of the challenge aerosol type and simulated breathing flow rate. Results and discussion ReSIM accuracy The data obtained in the flow-through set-up are presented in Fig. 5. The outputs of ReSIM are plotted against the aerosol concentration measured with the reference OAS for particles of 0.5–2 µm. First, the ReSIM prototype was found to operate across a broad particle concentration range, from low levels (<1 particles cm−3, expected inside of a well-fit respirator) to moderately high ones (>100 particles cm−3, contaminated air). Moreover, the regression results showed that the ReSIM outputs correlated with the aerosol concentration data generated by the reference instrument, with a regression function of y = 3.86 × ln(x) + 8.35 (y: the outputs of ReSIM, x: the OAS-measured aerosol concentration), with R2 being 0.936. The findings suggest that ReSIM provides high accuracy estimates of the aerosol concentrations within the specified particle size range. Lastly, the logarithmic relationship between the ReSIM output and the OAS-measured aerosol concentration suggests that ReSIM is more responsive at lower aerosol concentration levels, i.e. the ReSIM output increases more rapidly as aerosol concentration increases. Figure 5. View largeDownload slide ReSIM output plotted against the aerosol concentration measured with the reference OAS (the latter is plotted in logarithmic scale). Figure 5. View largeDownload slide ReSIM output plotted against the aerosol concentration measured with the reference OAS (the latter is plotted in logarithmic scale). The lowest aerosol concentration generated in the accuracy assessment phase was ~0.3 particles cm−3. This limit reflects difficulties encountered with generating stable particle concentration at levels lower than 0.3 particles cm−3. The highest concentration of ReSIM accuracy assessment was ~200 particles cm−3. The range 0.3–200 particles cm−3 was deemed to be sufficient for differentiating clean air inside a functioning respirator and contaminated air from outside the respirator. In principle, the true operating range of ReSIM may expand to the values lower than 0.3 particles cm−3 and higher than 200 particles cm−3. Since the particle sensor of the ReSIM prototype is based on light scattering (the output is dependent on the refractive index of particles), the ReSIM’s performance, in terms of accurately quantifying the aerosol concentration, depends on the particle composition. It is acknowledged that the accuracy assessment was conducted solely with NaCl particles in this phase of the study. While the coefficients in the regression equation presented in Fig. 5 may vary for particles of different composition (Wang et al., 2015; Sousana et al., 2017), the important finding here is that a relationship between outputs of ReSIM and actual particle concentration was established. Detection of leaks Statistical sensitivity of ReSIM The ReSIM sensitivity data obtained in the manikin-based tests are presented in Table 1. When testing with combustion aerosols, an overall of 98.4% (124 out 126) intervals with true faceseal leaks were correctly identified, which corresponded to a sensitivity as high as 98.4%. The sensitivities for specific flow rates and leak durations were close to 100% except for two false negatives, both were identified for 5-s leaks at MIF = 60 l min−1. Table 1. ReSIM sensitivity. Challenge aerosol Flow rate (l min−1) Leak duration (s) Number of intervals with Sensitivity True leaks False negativesa Combustion 30 5 11 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% 60 5 8 2 75.0% 10-crossb 12 0 100% 10 6 0 100% 15 6 0 100% 20 6 0 100% 85 5 9 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 126 2 98.4% NaCl 30 5 8 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 5 0 100% 20 6 0 100% 60 5 9 9 0% 10-crossb 12 8 33.3% 10 8 0 100% 15 6 0 100% 20 6 0 100% 85 5 8 8 0% 10-crossb 12 10 16.7% 10 10 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 124 35 71.8% Challenge aerosol Flow rate (l min−1) Leak duration (s) Number of intervals with Sensitivity True leaks False negativesa Combustion 30 5 11 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% 60 5 8 2 75.0% 10-crossb 12 0 100% 10 6 0 100% 15 6 0 100% 20 6 0 100% 85 5 9 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 126 2 98.4% NaCl 30 5 8 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 5 0 100% 20 6 0 100% 60 5 9 9 0% 10-crossb 12 8 33.3% 10 8 0 100% 15 6 0 100% 20 6 0 100% 85 5 8 8 0% 10-crossb 12 10 16.7% 10 10 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 124 35 71.8% aFalse negatives occur when ReSIM’s response is negative while a leak is present. b10-cross: a 10-s leak is intentionally timed to split the leak time across two ReSIM processing intervals. For 10-s cross, a false negative is defined as neither of the two intervals being identified as a leak. View Large Table 1. ReSIM sensitivity. Challenge aerosol Flow rate (l min−1) Leak duration (s) Number of intervals with Sensitivity True leaks False negativesa Combustion 30 5 11 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% 60 5 8 2 75.0% 10-crossb 12 0 100% 10 6 0 100% 15 6 0 100% 20 6 0 100% 85 5 9 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 126 2 98.4% NaCl 30 5 8 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 5 0 100% 20 6 0 100% 60 5 9 9 0% 10-crossb 12 8 33.3% 10 8 0 100% 15 6 0 100% 20 6 0 100% 85 5 8 8 0% 10-crossb 12 10 16.7% 10 10 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 124 35 71.8% Challenge aerosol Flow rate (l min−1) Leak duration (s) Number of intervals with Sensitivity True leaks False negativesa Combustion 30 5 11 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% 60 5 8 2 75.0% 10-crossb 12 0 100% 10 6 0 100% 15 6 0 100% 20 6 0 100% 85 5 9 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 126 2 98.4% NaCl 30 5 8 0 100% 10-crossb 14 0 100% 10 8 0 100% 15 5 0 100% 20 6 0 100% 60 5 9 9 0% 10-crossb 12 8 33.3% 10 8 0 100% 15 6 0 100% 20 6 0 100% 85 5 8 8 0% 10-crossb 12 10 16.7% 10 10 0 100% 15 6 0 100% 20 6 0 100% All tests Overall 124 35 71.8% aFalse negatives occur when ReSIM’s response is negative while a leak is present. b10-cross: a 10-s leak is intentionally timed to split the leak time across two ReSIM processing intervals. For 10-s cross, a false negative is defined as neither of the two intervals being identified as a leak. View Large When NaCl was used as a challenge aerosol, the overall sensitivity was 71.8% (89 out 124). The flow-rate-specific sensitivity values were 100% (41 out of 41) at MIF = 30 l min−1, 58.5% (24 out of 41) at MIF = 60 l min−1, and 57.1% (24 out of 42) at MIF = 85 l min−1. Thirty-five false negatives were identified all from 5-s or 10-s cross leaks at higher flow rates of 60 and 85 l min−1. Given that a higher ambient particle concentration results in more particles drawn into the respirator cavity through the faceseal leak, it is explicable why the sensitivity was greater for combustion aerosol as compared to NaCl. It is acknowledged that both challenge aerosols generated in these tests primarily cover ultra-fine and lower-range fine particle fractions and have relatively few large particles. At the same time, there were substantial numbers of larger particles of the targeted size range in the ambient environment. The median number concentration of combustion particles within the targeted size range was 1049 particles cm−3 compared to 566 particles cm−3 for NaCl. Note that, in principle, a higher sensitivity to leaks could be achieved as compared to the one found in this study, for instance, for NaCl aerosol, by lowering the leak detection threshold level. Initially, the occupancy ratio of 2% was set as the leak threshold level. The average output of false negatives (true leaks that failed to be identified) was found to be 1.40% (it ranged from 0.48 to 1.96%). Detection thresholds could be decreased to enable correct identification of such intervals close to 2%; however, the number of false positives would increase. In addition, setting a lower threshold would require moving significantly below the calibrated thresholds. Optimizing ReSIM for other detection thresholds will require analysis of the consistency of the environment to determine whether noise will cause unacceptable false positives and negatives. Ultimately, ReSIM can then be built with various thresholds for leak detection and consequently selected for utilization at different workplaces with different particle compositions. All false negatives were generated at short leak durations (<10 s) and at higher breathing flow rates. Leaks of shorter duration can be difficult to detect when the detection time is averaged across a relatively long 30-s interval. This difficulty is compounded by breathing impacts where for 2 s of every 4 s, the breathing simulator either draws all the particles inside or breathes out the particle-free filtered air. This creates a situation where the short-duration leaks can be significantly impacted by the breathing pattern. As the leak duration decreases, the chance of the leak occurring entirely during exhalation increases. Also, there should be fewer aerosol particles entering the respirator through the leak when the seal is opened for a shorter period. At higher MIF rates, the repeated cycling and high tidal volumes can pass smaller particles through the filters (Mahdavi et al., 2014; Gao et al., 2015). This effect is more pronounced for combustion aerosol with its higher particle concentration, in general, and the higher count of smaller particles, in particular. Even a P-100 filter at 99.97% efficiency allows 0.03% of particles through, which is treated in this investigation as a low but measurable level. Statistical specificity of ReSIM Overall, ReSIM presented the same specificity (response to the absence of leaks) for the two types of challenge aerosol. The specificity was 96.9% for both challenge aerosols (either 495 out of 511 for combustion aerosol or 505 out of 521 for NaCl), see Table 2. Table 2. ReSIM specificity. Challenge aerosol Flow rate (l min−1) Number of intervals with Specificity Number of intervals with Specificity No leaka False positivesb Persistent false positivesc Adjusted false positivesd Combustion 30 200 6 97.0% 6 0 100% 60 124 5 96.0% 5 0 100% 85 187 5 97.3% 4 1 99.5% Overall 511 16 96.9% 15 1 99.8% NaCl 30 183 14 92.3% 14 0 100% 60 142 1 99.3% 1 0 100% 85 196 1 99.5% 0 1 99.5% Overall 521 16 96.9% 15 1 99.8% Challenge aerosol Flow rate (l min−1) Number of intervals with Specificity Number of intervals with Specificity No leaka False positivesb Persistent false positivesc Adjusted false positivesd Combustion 30 200 6 97.0% 6 0 100% 60 124 5 96.0% 5 0 100% 85 187 5 97.3% 4 1 99.5% Overall 511 16 96.9% 15 1 99.8% NaCl 30 183 14 92.3% 14 0 100% 60 142 1 99.3% 1 0 100% 85 196 1 99.5% 0 1 99.5% Overall 521 16 96.9% 15 1 99.8% aNo leak means the respirator facepiece is fully sealed at 30-s intervals. bFalse positives may occur when ReSIM indicates a positive response (particles are detected) in the absence of a leak. cA persistent false positive is a false positive occurring in the interval immediately after an interval with a correctly identified leak. dPersistent false positives are excluded because the wearer would already be alerted by the preceding leak. View Large Table 2. ReSIM specificity. Challenge aerosol Flow rate (l min−1) Number of intervals with Specificity Number of intervals with Specificity No leaka False positivesb Persistent false positivesc Adjusted false positivesd Combustion 30 200 6 97.0% 6 0 100% 60 124 5 96.0% 5 0 100% 85 187 5 97.3% 4 1 99.5% Overall 511 16 96.9% 15 1 99.8% NaCl 30 183 14 92.3% 14 0 100% 60 142 1 99.3% 1 0 100% 85 196 1 99.5% 0 1 99.5% Overall 521 16 96.9% 15 1 99.8% Challenge aerosol Flow rate (l min−1) Number of intervals with Specificity Number of intervals with Specificity No leaka False positivesb Persistent false positivesc Adjusted false positivesd Combustion 30 200 6 97.0% 6 0 100% 60 124 5 96.0% 5 0 100% 85 187 5 97.3% 4 1 99.5% Overall 511 16 96.9% 15 1 99.8% NaCl 30 183 14 92.3% 14 0 100% 60 142 1 99.3% 1 0 100% 85 196 1 99.5% 0 1 99.5% Overall 521 16 96.9% 15 1 99.8% aNo leak means the respirator facepiece is fully sealed at 30-s intervals. bFalse positives may occur when ReSIM indicates a positive response (particles are detected) in the absence of a leak. cA persistent false positive is a false positive occurring in the interval immediately after an interval with a correctly identified leak. dPersistent false positives are excluded because the wearer would already be alerted by the preceding leak. View Large A persistent detection of particles inside the respirator over one or more intervals trailing a true leak was observed, particularly when testing at MIF = 30 l min−1. It generated 15 out of 16 false positive leak detections for both challenge aerosols. Persistent false positive occurs in the interval immediately following the one with a correctly identified leak. While this identification of trailing intervals as continued links is a false positive, the impact of this type of false positive from a user standpoint is simply an extended alert to a leak. This is less concerning than random false positive alerts would be. The data are presented with an adjusted false positive column indicating how many other false positive events were observed and excluding persistent false positives. Only one random false positive was observed at MIF = 85 l min−1 for both challenge aerosols used in this study. Note that the recent pilot testing of the ReSIM unit showed similar leak detection results (Liu et al., 2017). There are some differences between the prototypes of ReSIM used in the pilot study and the present effort. In the pilot testing, ReSIM prototype 1 was equipped with an external SKC AirChek TOUCH personal air sampling pump (SKC Inc., Eighty Four, PA, USA) and an external electrical power source, whereas prototype 2 evaluated in this study utilized a new particle sensor PPD60PV-T2, a built-in air pump, and battery. The present version of ReSIM equipped with new particle sensor is more sensitive than that in prototype 1, resulting in a higher level of response to the same aerosol concentration level. It suggested that the sensor in ReSIM needs periodic cleaning to ensure that the optical properties necessary for sensor operation are maintained. Also, a modified leak detection algorithm was deployed for prototype 2 based on a larger data set generated in this effort. Limitations Although the manufacturer claims that the operating relative humidity range of the particle sensor (PPD60PV-T2) extends to about 95% with temperatures from 0 to 45°C (Shinyei Technology, 2017), previous studies (Wang et al., 2015) suggested the performance of this type of particle sensor was highly affected by the air humidity. The temperature and relative humidity inside of chamber during the tests were 21–26°C and 39–46%, respectively. Therefore, the performance of ReSIM under hot and humid environment needs to be further evaluated. It is acknowledged that the leak algorithm is not yet perfect. More data are needed to optimize it. Nevertheless, the present laboratory evaluation study has demonstrated the ReSIM’s capability to perform a real-time monitoring of the performance of elastomeric respirators by detecting the seal leakage. This study demonstrated the ReSIM can detect a small-scale leakage introduced through a 5-mm diameter hole on faceseal. Any larger scale of leakage leading to higher particle penetration can be detected. The threshold of leakage alarm can be re-adjusted to a level suitable for a specific application. (Since the data from all tests conducted in this study are stored and available for further analysis, an optimum detection threshold can always be re-visited.) In this investigation, the leaks were artificially created by opening a small part of the faceseal. The leak’s shape was close to circular. In the real world, the faceseal leakage formed along the peripheral area around the face, it does not have a circular shape; instead, it is highly non-isometric. Furthermore, considering the body activities and facial muscle movement, a real faceseal leakage does not have a definitive shape. Further studies involving human subjects should address this limitation. Future direction The next phase of the study is to evaluate the ReSIM performance with human subjects under laboratory conditions using a simulation of routine workplace activities. In addition to qualifying the leak presence by ReSIM, the true real-time simulated workplace protection factor (SWPF) should be quantified by measuring the ratio of the aerosol concentrations outside (Cout) and inside (Cin) of the respirator donned on a subject. The SWPF data will create sufficient foundation to set the threshold for alarming the wearer about the respirator performance failure. Conclusions In this laboratory study, ReSIM demonstrated its ability to assess the real-time respirator performance during actual use. The findings showed that the ReSIM prototype could not only estimate the aerosol particle concentration with high accuracy, but—for a high-efficiency respirator—is also capable of responding rapidly in real time, with sufficient sensitivity and specificity, in the event when the faceseal integrity of this respirator is compromised. With an integrated alarm system, this response would trigger an alarm signaling the wearer that he/she can be overexposed to aerosol hazards during work activities. Although only an elastomeric half-mask respirator was used for the ReSIM evaluation in this study, the concept can be applied to some other types of particulate respirators, various workers’ populations, and various challenge aerosols. In addition to firefighters, the ReSIM can be used by respirator wearers engaging in the asbestos stripping, sandblasting, spray painting, and other activities. Acknowledgement This work was supported by the US Ohio Bureau of Workers’ Compensation. The authors declare no conflict of interest relating to the material presented in this article. Its contents, including any opinions and/or conclusions expressed, are solely those of the authors. References Baxter CS, Ross CS, Fabian T, et al. ( 2010) Ultrafine particle exposure during fire suppression–is it an important contributory factor for coronary heart disease in firefighters? J Occup Environ Med ; 52: 791– 6. Google Scholar CrossRef Search ADS PubMed Bolstad-Johnson DM, Burgess JL, Crutchfield CD, et al. ( 2000) Characterization of firefighter exposures during fire overhaul. Am Ind Hyg Asso J ; 61: 636– 41. Google Scholar CrossRef Search ADS Burgess JL, Nanson CJ, Bolstad-Johnson DM, et al. ( 2001) Adverse respiratory effects following overhaul in firefighters. J Occup Environ Med ; 43: 467– 73. Google Scholar CrossRef Search ADS PubMed Coffey C, Zhuang Z, Campbell D, et al. ( 1998) Quantitative fit-testing of N95 respirators: part II--results, effect of filter penetration, fit-test, and pass/fail criteria on respirator performance. J Int Soc Respir Prot ; 16: 25– 36. Dietrich J, Yermakov M, Reponen T, et al. ( 2015) Protection of firefighters against combustion aerosol particles: simulated workplace protection factor of a half-mask respirator (pilot study). J Occup Environ Hyg ; 12: 415– 20. Google Scholar CrossRef Search ADS PubMed Duling MG, Lawrence RB, Slaven JE, et al. ( 2007) Simulated workplace protection factors for half-facepiece respiratory protective devices. J Occup Environ Hyg ; 4: 420– 31. Google Scholar CrossRef Search ADS PubMed Gao S, Kim J, Yermakov M, et al. ( 2015) Penetration of combustion aerosol particles through filters of NIOSH-certified filtering facepiece respirators (FFRs). J Occup Environ Hyg ; 12: 678– 85. Google Scholar CrossRef Search ADS PubMed Grinshpun SA, Haruta H, Eninger RM, et al. ( 2009) Performance of an N95 filtering facepiece particulate respirator and a surgical mask during human breathing: two pathways for particle penetration. J Occup Environ Hyg ; 6: 593– 603. Google Scholar CrossRef Search ADS PubMed Han DH. ( 2002) Correlations between workplace protection factors and fit factors for filtering facepieces in the welding workplace. Ind Health ; 40: 328– 34. Google Scholar CrossRef Search ADS PubMed Haruta H, Honda T, Eninger RM. ( 2008) Experimental and theoretical investigation of the performance of N95 respirator filters against ultrafine aerosol particles tested at constant and cyclic flows. J Int Soc Respir Prot ; 25: 75– 88. Hauge J, Roe M, Brosseau LM, et al. ( 2012) Real-time fit of a respirator during simulated health care tasks. J Occup Environ Hyg ; 9: 563– 71. Google Scholar CrossRef Search ADS PubMed He X, Yermakov M, Reponen T, et al. ( 2013) Manikin-based performance evaluation of elastomeric respirators against combustion particles. J Occup Environ Hyg ; 10: 203– 12. Google Scholar CrossRef Search ADS PubMed Hinds WC. ( 1999) Aerosol technology: properties, behaviour, and measurements of airborne particles . 2nd edn. New York: John Willey & Sons. Institute of Medicine (IOM). ( 2008) Preparing for an influenza pandemic: personal protective equipment for healthcare workers . Washington, DC: The National Academies Press. Available at https://doi.org/10.17226/11980. Accessed 10 April 2018. Janssen L, Bidwell J. ( 2007) Performance of a full facepiece, air-purifying respirator against lead aerosols in a workplace environment. J Occup Environ Hyg ; 4: 123– 8. Google Scholar CrossRef Search ADS PubMed Liu Y, Corey J, Yermakov M, Wu B, and Grinshpun S. ( 2017) Preliminary development of a real-time respirator seal integrity monitor with low-cost particle sensor. IEEE Trans Ind Appl ; doi:10.1109/TIA.2018.2816907. Mahdavi A, Bahloul A, Haghighat F, et al. ( 2014) Contribution of breathing frequency and inhalation flow rate on performance of N95 filtering facepiece respirators. Ann Occup Hyg ; 58: 195– 205. Google Scholar PubMed Mahdavi A, Haghighat F, Bahloul A, et al. ( 2015) Particle loading time and humidity effects on the efficiency of an N95 filtering facepiece respirator model under constant and inhalation cyclic flows. Ann Occup Hyg ; 59: 629– 40. Google Scholar PubMed Marcum JI. ( 1948) A statistical theory of target detection by pulsed radar: mathematical appendix . Santa Monica, CA: RAND Corporation. Martin SBJr, Moyer ES. ( 2000) Electrostatic respirator filter media: filter efficiency and most penetrating particle size effects. Appl Occup Environ Hyg ; 15: 609– 17. Google Scholar CrossRef Search ADS PubMed NIOSH. ( 1995) Respiratory protective devices. Title 42 Code of Federal Regulation, Part 84, 30335-98 . Washington, DC: U.S. Government Printing Office, Register, Office of the Federal. NIOSH. ( 2004) Standards development: program concept for total inward leakage (TIL) performance requirements and test methods. Available at https://www.cdc.gov/niosh/npptl/standardsdev/til/. Accessed 10 April 2018. OSHA. ( 1999) Respiratory protection. Title 29 Code of Federal Regulation, Part 1910.134. U.S . Available at https://www.osha.gov/pls/oshaweb/owadisp.show_document?p_table=STANDARDS&p_id=12716. Accessed 10 August 2017. Peterson WW, Birdsall TG, Fox WC. ( 1954) The theory of signal detectability. IEEE Trans Inf Theory, 4: 171– 212. Rengasamy S, Eimer BC. ( 2011) Total inward leakage of nanoparticles through filtering facepiece respirators. Ann Occup Hyg ; 55: 253– 63. Google Scholar PubMed Rengasamy S, Eimer BC. ( 2012) Nanoparticle penetration through filter media and leakage through face seal interface of N95 filtering facepiece respirators. Ann Occup Hyg ; 56: 568– 80. Google Scholar PubMed Rengasamy S, Eimer BC, Shaffer RE. ( 2009) Comparison of nanoparticle filtration performance of NIOSH-approved and CE-marked particulate filtering facepiece respirators. Ann Occup Hyg ; 53: 117– 28. Google Scholar CrossRef Search ADS PubMed Shinyei Technology. ( 2017) PPD60PV-T2 particle sensor unit. Available at http://www.shinyei.co.jp/stc/eng/optical/main_ppd60pv.html. Accessed 10 April 2018. Sousana S, Koehler K, Halletta Let al. , editors. ( 2017) Evaluation of consumer monitors to measure particulate matter. J Aero Sci ; 107: 123– 33. Google Scholar CrossRef Search ADS Stevens GA, Moyer ES. ( 1989) “Worst case” aerosol testing parameters: I. Sodium chloride and dioctyl phthalate aerosol filter efficiency as a function of particle size and flow rate. Am Ind Hyg Assoc J ; 50: 257– 64. Google Scholar CrossRef Search ADS PubMed Tannahill SN, Willey RJ, Jackson MH. ( 1990) Workplace protection factors of HSE approved negative pressure full-facepiece dust respirators during asbestos stripping: preliminary findings. Ann Occup Hyg ; 34: 547– 52. Google Scholar PubMed Vo E, Zhuang Z, Horvatin M, et al. ( 2015) Respirator performance against nanoparticles under simulated workplace activities. Ann Occup Hyg ; 59: 1012– 21. Google Scholar CrossRef Search ADS PubMed Wang Y, Li J, Jing H, et al. ( 2015) Laboratory evaluation and calibration of three low-cost particle sensors for particulate matter measurement. Aero Sci Tech ; 49: 1063– 77. Google Scholar CrossRef Search ADS Zhuang Z, Coffey CC, Jensen PA, et al. ( 2003) Correlation between quantitative fit factors and workplace protection factors measured in actual workplace environments at a steel foundry. Am Ind Hyg Assoc J ; 64: 730– 8. Google Scholar CrossRef Search ADS Zhuang Z, Coffey C, Myers W, et al. ( 1998) Quantitative fit-testing of N95 respirators: part I-method development. J Int Soc Respir Prot ; 16: 11– 24. © The Author(s) 2018. Published by Oxford University Press on behalf of the British Occupational Hygiene Society. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)
Annals of Work Exposures and Health (formerly Annals Of Occupational Hygiene) – Oxford University Press
Published: Apr 21, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera