EPA 747-R-95-002a May 1995 A FIELD TEST OF LEAD-BASED PAINT TESTING TECHNOLOGIES: SUMMARY REPORT Technical Programs Branch Chemical Management Division Office of Pollution Prevention and Toxics Office of Prevention, Pesticides, and Toxic Substances U.S. Environmental Protection Agency 401 M Street, S.W. Washington, D.C. 20460 The material in this document has been subject to Agency technical and policy review and approved for publication as an EPA report. Mention of trade names, products, or services does not convey, and should not be interpreted as conveying, official EPA approval, endorsement, or recommendation. This report is copied on recycled paper. CONTRIBUTING ORGANIZATIONS The study described in this report was funded by the U.S. Environmental Protection Agency and the U.S. Department of Housing and Urban Development. The study was managed by the U.S. Environmental Protection Agency. The study was conducted collaboratively by two organizations under contract to the Environmental Protection Agency, Midwest Research Institute and QuanTech. Each organization's responsibilities are listed below. Midwest Research Institute Midwest Research Institute (MRI) was responsible for initiating the pilot study on schedule, for overall production of the Quality Assurance Project Plan for both the pilot and the full study, for providing input to the design of the study, for planning and supervising the field work, for collecting paint samples, for the laboratory analysis of paint chip samples, and for writing sections of the technical report. QuanTech QuanTech (formerly David C. Cox & Associates) was responsible for the design of the study and contributions to the Quality Assurance Project Plan for the pilot and full studies, for participation in field work, for data management and statistical analysis, and for overall production of the technical and summary reports. U.S. Environmental Protection Agency The U.S. Environmental Protection Agency (EPA) co-funded the study and was responsible for managing the study, for reviewing study documents, and for arranging for the peer review of the final report. The EPA Project Leader was John Schwemberger. The EPA Work Assignment Managers were John Scalera and John Schwemberger. The EPA Project Officers were Jill Hacker, Samuel Brown, and Janet Remmers. Cindy Stroup was the Branch Chief of the Technical Programs Branch and initiated this study. U.S. Department of Housing and Development The Department of Housing and Urban Development (HUD) co-funded the study and identified sources of housing for the study. Bill Wisner was the key HUD staff member. ACKNOWLEDGEMENTS The study could not have been done without the assistance and cooperation of the Housing Authority of Louisville, the Denver Housing Authority, and the Philadelphia Housing Authority. Special thanks are due to Mike Godfrey and George Adams of the Housing Authority of Louisville, Mark Ward and Ben Roybal of the Denver Housing Authority, and John Peduto, Cynthia Jones, and Bill Zollicoffer of the Philadelphia Housing Authority. TABLE OF CONTENTS EXECUTIVE SUMMARY. . . . . . . . . . . . . . . . . xiii BACKGROUND. . . . . . . . . . . . . . . . . . xiii TECHNOLOGIES EVALUATED. . . . . . . . . . . . xiii STUDY OBJECTIVES. . . . . . . . . . . . . . . .xiv FIELD TESTING . . . . . . . . . . . . . . . . .xiv STUDY RESULTS . . . . . . . . . . . . . . . . .xvi Laboratory Analysis Results. . . . . . . xvi Chemical Test Kit Results. . . . . . . . xvi XRF Results. . . . . . . . . . . . . . . xvii OVERALL RECOMMENDATIONS FOR TESTING . . . . .xviii XRF Instrument Conclusions . . . . . . xviii Chemical Test Kit Conclusions. . . . . . xix 1 DESCRIPTION OF THE STUDY. . . . . . . . . . . . . 1 1.1 BACKGROUND . . . . . . . . . . . . . . . . 1 1.2 STUDY OBJECTIVES . . . . . . . . . . . . . 2 1.3 APPROACH . . . . . . . . . . . . . . . . . 3 1.4 TECHNOLOGIES . . . . . . . . . . . . . . . 4 1.5 FIELD TESTING. . . . . . . . . . . . . . . 6 1.6 PEER REVIEW. . . . . . . . . . . . . . . . 8 2 STUDY CONCLUSIONS, TESTING RECOMMENDATIONS, AND SUMMARY OF STUDY RESULTS. . . . . . . . . . 11 2.1 CONCLUSIONS AND RECOMMENDATIONS FOR TESTING. . . . . . . . . . . . . . . . . . 11 2.1.1 XRF Instrument Conclusions . . . 11 2.1.2 Chemical Test Kit Conclusions. . 11 2.2 RESULTS FOR STUDY OBJECTIVES . . . . . . . 12 2.2.1 Precision and Accuracy of XRF Instruments. . . . . . . . . . . 12 2.2.2 Substrate Interference . . . . . 12 2.2.3 Large XRF Errors . . . . . . . . 13 2.2.4 Field Quality Assurance and Quality Control Methods. . . . . 13 2.2.5 Operating Characteristic Curves for Test Kits. . . . . . . . . . 13 2.2.6 Variability of Lead Levels in Paint. . . . . . . . . . . . . . 14 3 DETAILED STUDY RESULTS. . . . . . . . . . . . . . 15 3.1 LEAD LEVELS IN THE STUDY SAMPLES . . . . . 15 3.2 XRF INSTRUMENTS. . . . . . . . . . . . . . 16 3.3 CHEMICAL TEST KITS . . . . . . . . . . . . 28 3.4 PAINT CHIP SAMPLING AND ANALYSIS . . . . . 33 LIST OF TABLES Table 1. Cross-Tabulation of Paint Sample Lead Levels in mg/cm2 Lead and Percent Lead by Weight . . . . . . . . . . . . . . . . . . 15 Table 2. Estimated Standard Deviation at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Paint Reading for K-Shell XRF Instruments, by Substrate. . . . . . . . . 17 Table 3. Estimated Standard Deviation at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading on Control Blocks for K-Shell XRF Instruments, by Substrate. . . 18 Table 4. Bias at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading for K-Shell XRF Instruments, by Substrate. . . . . . . 19 Table 5. Bias at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading on Control Blocks for K-Shell XRF Instruments, by Substrate. . . . . . . . . . . . . . . . . 20 Table 6. Bias at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading for L-Shell XRF Instruments, by Substrate. . . . . . . 21 Table 7. False Positive, False Negative and Inconclusive Percentages for K-Shell XRF Instruments, Based on One Nominal 15-Second Reading With an INCONCLUSIVE RANGE OF 0.4 - 1.6 mg/cm2 (1.0 mg/cm2 Threshold). . . . . . . . . . . . . 24 Table 8. False Positive, False Negative and Inconclusive Percentages for K-Shell XRF Instruments, Based on One Nominal 15-Second Reading With an INCONCLUSIVE RANGE OF 0.7 - 1.3 mg/cm2 (1.0 mg/cm2 Threshold). . . . . . . . . . . . . 25 Table 9. False Positive and False Negative Percentages for K-Shell XRF Instruments, Based on One Nominal 15-Second Reading With NO INCONCLUSIVE RANGE (1.0 mg/cm2 Threshold). . . . . . . . . . . . . 26 Table 10. False Positive, False Negative and Inconclusive Percentages for L-Shell XRF Instruments, Based on One Nominal 15-Second Reading with an INCONCLUSIVE RANGE OF 0.4 - 1.6 mg/cm2 (1.0 mg/cm2 Threshold). . . . . . . . . . . . . 26 Table 11. Overall False Positive and False Negative Rates for Test Kits Compared to Laboratory Analytical Results Using the 1.0 mg/cm2 Threshold . . . . . . . . . . . 29 Table 12. Overall False Positive and False Negative Rates for Test Kits Compared to Laboratory Analytical Results Using the 0.5% Threshold . . . . . . . . . . . . . . 29 Table 13. Probability of a Positive Test Kit Result at 1.0 mg/cm2 Lead.. . . . . . . . . . . . 31 Table 14. Probability of a Positive Test Kit Result at 0.5% Lead.. . . . . . . . . . . . . . . 32 Table 15. Lead Level in mg/cm2 at Which There is a 50% Probability of a Positive Test Kit Result . . . . . . . . . . . . . . . . . . 33 Table 16. Lead Level in Percent Lead by Weight at Which There is a 50% Probability of a Positive Test Kit Result . . . . . . . . . 33 LIST OF FIGURES Figure 1. Full study template. . . . . . . . . . . . 7 EXECUTIVE SUMMARY BACKGROUND This study was undertaken by the U.S. Environmental Protection Agency (EPA) and the U.S. Department of Housing and Urban Development (HUD) to collect information needed for the development of federal guidance on testing paint for lead. Prior to this study, lead testing information was inadequate as little formal evaluation had been done of the various field testing methodologies. The impetus for this study came from the passage of Title X (Section 1017 of the Residential Lead-Based Paint Hazard Reduction Act of 1992), which mandated that the federal government establish guidelines for lead-based paint hazard evaluation and reduction. This study was designed to produce the type of detailed information EPA and HUD needed in order to respond to that mandate, and focused on two field technologies that are used for testing for lead in paint: portable X-ray fluorescence (XRF) instruments and chemical test kits. A pilot study was conducted during March and April 1993 in Louisville, Kentucky. The full study was conducted from July through October 1993 in Denver, Colorado and Philadelphia, Pennsylvania. This is a summary report of the study. For readers that are interested in more technical detail on the study, there is also a comprehensive technical report available: A Field Test of Lead-Based Paint Testing Technologies: Technical Report (EPA 747-R-95- 002b). Both reports are available from the National Lead Information Center Clearinghouse (1-800-424-LEAD). TECHNOLOGIES EVALUATED This study evaluated XRF instruments and chemical test kits. XRF instruments measure lead in paint by directing high energy X-rays and gamma rays into the paint, causing the lead atoms in the paint to emit X- rays which are detected by the instrument and converted to a measurement of the amount of lead in the paint. Chemical test kits detect the presence of lead in paint by a chemical reaction that occurs when chemicals in the kit are exposed to lead. This reaction causes a color change to occur if lead is present in the paint. Laboratory spectroscopic analysis of paint samples was conducted to determine the actual levels of lead in the paint. The laboratory results were used as a benchmark for comparison to the XRF and test kit results. STUDY OBJECTIVES The overall study goal was to collect information about field measurement methodologies sufficient to allow EPA and HUD to establish guidance and protocols for lead hazard identification and evaluation. In order to achieve that goal, the study had to be designed and conducted with sufficient rigor and appropriate quality assurance. To ensure adequacy of the resulting data, six specific study objectives were developed: three primary and three secondary. The results are presented in this report in two ways: overall conclusions and testing recommendations are made in light of the overall study goal, and results are provided in terms of the specific study objectives. The three primary study objectives were: (1) to characterize the performance (precision and accuracy) of portable XRF instruments under field conditions; (2) to evaluate the effect on XRF performance of interference from the material (the substrate) underlying the paint; and (3) to characterize the relationship between test kit results and the actual lead level in the paint (operating characteristic curves). The three secondary study objectives were: (4) to understand XRF behavior in the field through the investigation of XRF measurements that were very different than their corresponding lab result; (5) to evaluate field quality assurance and control methods; and (6) to investigate the variability of lead levels in the paint within the study sampling locations. FIELD TESTING Three primary concerns of the field testing portion of the study were consistency, real world comparability, and quality control. Due to the differences among the three measurement methods: XRF, test kits, and laboratory analysis, field testing approaches necessarily varied somewhat. In order to ensure consistency, testing was standardized as much as possible. A template was designed for test locations throughout the study housing units, and the different measurement methods were systematically assigned to consistent test locations within the template. This approach ensured results could be compared across different test locations and measurement methods. At each test location, chemical test kits were tested first. The individuals who did the field testing of the test kits were selected to represent typical homeowners who might purchase test kits for their personal use. That is, they did not have any specific scientific background nor prior training. To further replicate þreal worldþ use, the test kits were rotated among the testers during the study. One of the test kits was an exception to this. It was a kit which is only used by state-certified inspectors. For that kit, a state-certified inspector was brought in and that particular kit was not included in the kit rotation. After each tester completed a test location, the used area of the template was covered to prevent subsequent testers from observing the results obtained by prior testers. Once test kit testing was finished, paint samples were taken. Paint was removed from a specified location on the template and sent to a laboratory for spectroscopic analysis. A modified NIOSH method 7082 was followed with all appropriate quality control samples including laboratory and field duplicates. XRF testing was the final step in the field portion of the study. It was conducted by trained and licensed XRF instrument operators employed by independent testing companies. XRF testing was carried out on the portions of the templates designated for this purpose. A number of quality control procedures were employed, including the use of National Institute of Standards and Technology (NIST) Standard Reference Material (SRM) paint films. The NIST SRM paint film is a thin layer of paint with a known level of lead enclosed between two layers of plastic. A portion of the template was scraped bare of paint, revealing the material underneath the paint, the substrate, which was either brick, concrete, drywall, metal, plaster or wood. The NIST SRM paint film was placed on the bare substrate and a reading was taken in order to determine if the substrate interfered with the XRF reading. In addition, blocks of known substrate materials, called control blocks, were utilized in the field. The NIST SRM paint film was placed on the appropriate block and XRF readings taken in order to determine if control block substrates could be surrogates for the substrates underlying the painted areas tested. STUDY RESULTS: Laboratory Analysis Results There were two key results of the laboratory analyses. First, laboratory analysis results exhibited a wide range of lead levels with a distribution similar to that reported in the 1990 HUD National Survey of Lead-Based Paint in Housing. Second, lead levels appear to vary significantly across the same painted surface. Two federal thresholds have been established to define lead-based paint on painted architectural components. If paint is found to contain lead equal to or greater than these thresholds, it is characterized as lead-based paint. The federal threshold in milligrams lead per unit area is 1.0 mg/cm2. The federal threshold in percent lead by weight is 0.5%. Approximately 20% of the samples analyzed in this study were equal to or greater than the federal threshold of 1.0 mg/cm2, while 29% were equal to or greater than the federal threshold of 0.5% lead. A rough numerical equivalence between results reported as mass of lead per unit area (mg/cm2) and as percent lead by weight (%) was found in the study data. That is, 1.0 mg/cm2 lead was found to be roughly equivalent to 1% lead by weight. The variability of a set of test results is the extent to which the results in the set differ from one another. The standard deviation is a statistical measure of the extent that actual test results tend to spread about an average value. The typical relative standard deviation for laboratory analytical measurements in the study samples was 13%. Variability between field duplicate samples, taken nine inches apart at a subset of test locations, was much larger, between 30% - 60%, indicating significant variability in lead levels across the same painted surface. The statistical analysis of the data took variability in lead levels into account. Chemical Test Kit Results The primary result of the test kit evaluation is that they varied widely in their performance in classifying paint against either the 1.0 mg/cm2 or 0.5% threshold. No single kit achieved a low rate of both false positive and false negative results and their performance varied across substrates. A false negative result occurs when the kit fails to detect the presence of lead in paint equal to or greater than the federal threshold, but in fact, the paint is shown by laboratory analysis to contain lead equal to or greater than the threshold. Similarly, a false positive result occurs when the kit detects lead equal to or greater than the federal threshold, but laboratory analysis shows that the paint does not contain lead equal to or greater than the threshold. No kit in the study achieved low rates of both false positive and false negative results. Two out of six kits were prone to false negative results. Negative test results obtained with these two kits do not necessarily indicate the absence of lead. The other four kits had a tendency to produce false positive results, even at levels of lead well below the federal thresholds. Further, the performance of the test kits varied with different types of substrates. Most kits usually produced a positive result on at least one substrate, even for very low lead levels. This suggests positive interferences with the chemicals in the kits. On the other hand, some test kits demonstrated negative interferences on some substrates, as indicated by not always giving a positive result for high levels of lead. XRF Results The primary result of the XRF testing is that K- shell instruments were often effective in classifying paint samples against the federal threshold of 1.0 mg/cm2, when using an inconclusive classification range, laboratory confirmation, and substrate correction, as needed. Generally, L-shell instruments had extremely high false negative rates, making them ineffective in classifying paint against the 1.0 mg/cm2 threshold. In this study, measurement bias, or bias, is the tendency of a set of test results to be either greater or less than the laboratory measurements of the lead content of the paint. If test results tend to be greater than the laboratory results, they are said to exhibit positive bias. If the test results tend to be less than the laboratory results, they exhibit negative bias. Results of tests using XRF instruments showed both positive and negative bias. Biases of the K-shell XRF instruments were strongly dependent on the underlying substrate. One K-shell instrument exhibited much less bias than the other XRF instruments. L-shell instruments generally had large negative biases at the 1.0 mg/cm2 threshold that were usually independent of the substrate. Substrate correction, using NIST SRM readings on either the scraped substrates or the control blocks, did not reduce bias for L-shell instruments. For K- shell instruments, results were mixed. Control block correction reduced bias for two instruments on some substrates. Correction using NIST SRM readings on the scraped substrate was effective for two instruments on most substrates, and for another instrument on some substrates. The variability of the results from each XRF instrument was estimated by calculating a standard deviation. The results of most K-shell instruments exhibited high variability at the federal threshold of 1.0 mg/cm2. The variability in the results from the L- shell instruments was significantly lower than that of K-shell instruments. Despite their generally high variability and bias, K-shell instruments were often effective in classifying the paint samples in this study against the federal threshold of 1.0 mg/cm2 when using an inconclusive classification range of 0.4 to 1.6 mg/cm2 with mandatory laboratory confirmation. Without using an inconclusive range and laboratory confirmation, only two of the K-shell instruments had both false positive and false negative rates below 10%. Generally, L-shell instruments had extremely high false negative rates. One L-shell instrument had moderate to high false negative rates, depending on the width of the inconclusive range, but still gave low readings on some samples with high levels of lead. OVERALL RECOMMENDATIONS FOR TESTING XRF Instrument Conclusions The primary XRF conclusion is that testing by K- shell XRF instruments, with laboratory confirmation of inconclusive XRF results, and with substrate correction in cases where this is effective in reducing bias, is a viable way to test for lead-based paint. This approach can produce satisfactory results for classifying the paint on architectural components using the federal threshold of 1.0 mg/cm2. Further, the variability found in paint samples located approximately nine inches apart supports the conclusion that the most effective method of XRF testing of a single architectural component, such as a window sill, wall, or door, is to obtain readings at different points on the component, and compute their average. This would replace the current practice which is to average a number of XRF readings taken at a single point. Chemical Test Kit Conclusions The conclusion of this study is that test kits should not be used for lead paint testing. Test kits cannot determine the extent of lead-based paint in a home and the need for protecting the occupants, especially when repairs or renovations are carried out. Homeowners and renters cannot be confident that test kits will discriminate accurately between lead-based paint and non-lead based paint. They should not make decisions on repairs, renovations or abatements based on test kit results. 1 DESCRIPTION OF THE STUDY 1.1 BACKGROUND Lead-based paint (LBP) in older housing, especially lead-based paint in poor condition, is recognized as a major cause, both direct and indirect, of elevated blood lead levels in children between 1 and 6 years old. Exposure to lead in paint can come from the paint chips themselves, from dust caused by abrasion of paint on friction surfaces, or from chalking of exterior paint. The Lead-Based Paint Poisoning Prevention Act of 1971, as amended by the Housing and Community Development Act of 1987, established 1.0 mg/cm2 as the federal threshold requiring abatement of lead-based paint in public and Indian housing developments nationwide. To implement this legislation, Congress required the U.S. Department of Housing and Urban Development (HUD) to complete testing for lead-based paint in all public and Indian housing by December, 1994. In response to this requirement, HUD, with substantial input from the Environmental Protection Agency (EPA), published interim guidelines for testing and abatement of LBP in public and Indian housing in April, 1990. At the time the HUD Guidelines were published, the research conducted to evaluate the performance of X-ray fluorescence (XRF) instruments and chemical test kits in detecting LBP at or above the federal threshold was limited. The recommended approach was to perform XRF testing, with laboratory confirmation of inconclusive results. The Guidelines recommended that test kits should not be used as a primary testing method. Federal guidance documents available from the National Lead Information Center Clearinghouse also did not recommend the use of test kits by homeowners or renters. The Residential Lead-Based Paint Hazard Reduction Act of 1992 ("Title X") mandated the evaluation and reduction of lead-based paint hazards in the nation's existing housing. Title X also established 0.5% lead as an alternative to the 1.0 mg/cm2 threshold. Section 1017 of Title X required HUD to develop guidelines for federally-supported lead-based paint hazard evaluation and reduction activities. HUD is complying with this requirement by preparing a major revision and expansion of the 1990 Guidelines. To support the testing and inspection portion of the revised Guidelines, EPA and HUD funded this field study of technologies used to detect and measure lead in paint. It is the first comprehensive evaluation of XRF instruments and test kits under field conditions. This report is a summary of the study procedures and provides results, conclusions and recommendations for testing for lead-based paint. Study conclusions and testing recommendations, and a summary of study results are in chapter 2. Chapter 3 contains detailed study results. Further information on all aspects of the study can be found in the detailed report entitled A Field Test of Lead-Based Paint Testing Technologies: Technical Report (EPA 747-R-95-002b). 1.2 STUDY OBJECTIVES The overall study goal was to collect information about field measurement methodologies sufficient to allow EPA and HUD to establish guidance and protocols for lead hazard identification and evaluation. In order to achieve that goal, the study had to be designed and conducted with sufficient rigor and appropriate quality assurance. To ensure adequacy of the resulting data, six specific study objectives were developed: three primary and three secondary. The results are presented in this report in two ways: overall conclusions and testing recommendations are made in light of the overall study goal, and results are provided in terms of the specific study objectives. The three primary study objectives were: (1) to characterize the performance (precision and accuracy) of portable XRF instruments under field conditions; (2) to evaluate the effect on XRF performance of interference from the material (the substrate) underlying the paint; and (3) to characterize the relationship between test kit results and the actual lead level in the paint (operating characteristic curves). The three secondary study objectives were: (4) to understand XRF behavior in the field through the investigation of XRF measurements that were very different than their corresponding lab result; (5) to evaluate field quality assurance and control methods; and (6) to investigate the variability of lead levels in the paint within the study sampling locations. This study differs from previous studies conducted to measure lead in paint because the study included a larger number of samples and more diverse testing locations, and was designed so that test results obtained at different locations could be compared. Paint from a total of 1,290 locations in 22 housing units in three cities was tested. The tested locations were free from identifiable biases and represent a variety of paint types, substrates, architectural designs, and lead levels in paint. The study was designed to evaluate field testing technologies used to identify lead-based paint that were commercially available or were working prototypes as of June, 1993. These technologies included six types of XRF instruments and six chemical test kits. Spectroscopic laboratory analysis was used to verify results obtained by the XRF instruments and chemical test kits. 1.3 APPROACH The study began in March 1993 in Louisville, Kentucky, with a pilot conducted at a vacant public housing development built in 1937. Testing was conducted at 100 locations in 4 units in 2 buildings. The pilot had several objectives. First, it was important to determine the feasibility of collecting large numbers of paint samples in the field while ensuring the quality of the samples, and to develop and test a system for labelling and tracking the samples. Removal of paint with a heat gun and paint scraper proved to be a successful technique. A barcode system that labelled and tracked samples was developed and tested. A working system for selecting and marking test locations was developed. The field practicality of the test kits for large testing programs was evaluated. Procedures for monitoring XRF testing and recording of data were developed. Field testing sequences to minimize the potential for variability in XRF results caused by frequent substrate changes were used. Time estimates for all aspects of sample collection and testing were made. The schedule and logistics for the full study were based on these time estimates. A database structure was developed for storing and retrieving study data. The full study was conducted in two cities, Denver in July and August 1993 and Philadelphia in September and October 1993. Denver and Philadelphia were specifically chosen because housing was available that met study criteria and because the public housing authorities in those cities were willing to work closely with EPA and its contractors. The study tested units from both multifamily housing, where units tend to be quite similar to each other, and from single- family homes. A total of 10 scattered-site single- family homes were tested in Denver; eight were built between 1943 and 1952, while two were older, dating from 1890 and 1905. In Philadelphia, eight units in two buildings in a single multifamily development built in 1942 were tested. Including those in the pilot study, a total of 1,290 individual test locations on 6 substrate types in the 22 housing units were tested. There were 100 test locations in Louisville, 750 in Denver and 440 in Philadelphia. The breakdown of testing locations by substrate was: 93 brick, 226 concrete, 124 drywall, 217 metal, 242 plaster, and 388 wood substrates. 1.4 TECHNOLOGIES Chemical test kits detect the presence of lead in paint by a chemical reaction that occurs when chemicals in the kit are exposed to lead. This reaction causes a color change to occur if lead is present in the paint. The test kits in the study represented the range of kits available at the time the study was conducted. Test kits from five different manufacturers were examined in this study: three rhodizonate based kits, two sodium sulfide based kits, and one proprietary kit. Both of the most common types of chemical test kits, rhodizonate based kits and sodium sulfide based kits, were used in the pilot study. The rhodizonate kits included were LeadCheck (also called LeadCheck II) and the sanding and coring versions of Lead Alert; the sodium sulfide kits were Lead Detective and the Massachusetts state-approved kit. The pilot study also included the Lead Zone kit, which utilizes proprietary chemistry. It was expected that the results of the pilot study would be similar for kits based on similar chemistry, that is, rhodizonate or sodium sulfide, so that fewer kits would need to be included in the full study. However, the test results were not similar for kits utilizing similar chemistry, so the same six kits were included in the full study. Portable XRF instruments direct high energy X-rays and gamma rays into paint. These high-energy rays strike lead atoms, causing electrons to be ejected from their electron orbits, or shells. In a process called fluorescence, other electrons refill the voids left by the ejected electrons, producing X-rays. These X-rays have specific frequencies based on differences in energy between the electron shells which contained the emitted electrons and the electron shells which received the electrons. The amount of X-ray energy emitted at several specific frequencies, in this case called K-shell or L-shell X-ray energy, is measured by detectors on XRF instruments and used to calculate the amount of lead in paint. XRF instruments are classified by the type of X-ray energy that they detect, K-shell X-rays, L-shell X-rays, or both. K-shell X-rays are more highly penetrating than L-shell X-rays since L-shell X-rays have lower energy. For this reason, K-shell X-rays are more useful for detecting lead in deeper layers of paint. Two of the XRF instruments in this study detected K-shell X-rays, two XRF instruments detected L-shell X-rays, and two instruments detected both K-shell and L-shell X-rays. Efforts were made to include a representative example of every XRF instrument available at the time of the study. Six types of XRF instruments were in the study. The MAP-3, the Microlead I, and the XK-3 were included because they were the most commonly used instruments for LBP testing when the study began. The X-MET 880 was included because it performed successfully in the pilot study. After completion of the pilot study, all other known manufacturers of XRF instruments or working prototypes were invited to participate in a day of ruggedness testing to determine whether the instruments were portable and could function reliably throughout a full day of field testing. As a result, two additional instruments, the Lead Analyzer and a prototype of the XL, were included in the full study. Since the conclusion of the field portion of the study, new XRF instruments and modified versions of some tested instruments have become commercially available. The third type of technology in the study was laboratory analysis which was used to verify results obtained by the two field technologies: chemical test kits and XRF instruments. For this study, the laboratory instrument used was an atomic emission spectrophotometer. The laboratory procedure involved dissolving paint samples in acid, then filtering and diluting them. A portion of the dissolved sample was placed in the spectrophotometer and heated to extremely high temperatures by a device inside the spectrophotometer called a high temperature atomizer. At very high temperatures, most of the sample is broken down into individual atoms. Individual atoms absorb and re-emit energy produced by the atomizer. Atoms of different chemical elements re-emit energy at different energy levels. A detector in the spectrophotometer sorts and measures the energy re-emitted by the atoms of different chemical elements. In this way, the amount of energy re-emitted by lead atoms is measured and then used to calculate the amount of lead in the sample. The particular type of spectrophotometer used in this study was an inductively coupled plasma atomic emission spectrophotometer (ICP). The analytical laboratory results were continually evaluated by using reference materials to assure the accuracy of the laboratory analysis of field samples. Chemical test kit results were reported as either negative or positive indicating the absence of lead or presence of lead, respectively. XRF instruments and laboratory analysis results were reported as quantitative measures of lead. XRF instruments report their results as mass of lead per unit area (mg/cm2). Laboratory analysis results were reported both as mass of lead per unit area (mg/cm2) and percent lead by weight (%). 1.5 FIELD TESTING Templates were designed for marking test locations in the study housing units so that results could be compared for different test technologies and locations. The most commonly used template, shown in Figure 1, was a rectangle 14 inches long and 4 inches wide. For certain locations such as door frames, a thin version of the template, 2 inches by 14 inches, was needed. On the left of the most commonly used template was a square 4 x 4 inches; in the center, a second 4 x 4 inch square was divided into four 2 x 2 inch subsquares; the remaining 6 x 4 inch rectangle on the right of the template was divided into six vertical strips each 1 x 4 inches. One of the 2 x 2 inch subsquares was randomly selected as the location for paint sampling for laboratory analysis. At 10% of locations in the full study, a duplicate paint sample was taken adjacent to the right end of the template for use in assessing variability in the paint lead levels. Following paint sampling, the remainder of the center 4 x 4 inch square was scraped to remove all remaining paint. It was then used for taking XRF measurements on bare substrates both with and without the standard reference material paint films (SRM 2579) developed by the National Institute of Standards and Technology (NIST). The NIST SRM paint film is a thin layer of paint with a known level of lead enclosed between two layers of plastic. The 4 x 4 inch square on the left of the template was used for XRF measurements on paint. The six 1 x 4 inch strips were randomly assigned as testing locations for the six chemical test kits. Each of the testing locations in the study was selected and marked by the field statisticians using the template and an indelible ink marker. Each test location was numbered for identification and sample tracking. Figure 1. Full Study Template (MISSING) The first step in the full study was to test the six chemical test kits. Testers for five of the six test kits were individuals without any special scientific background or prior training. They were selected to represent typical homeowners who might purchase kits for their personal use. The testers were trained by field supervisors to ensure that study protocols were followed. The training did not provide the testers with knowledge about test kit operation beyond the information contained in the manufacturer's instructions. These five kits were rotated among the testers during the study. The sixth kit, tested by a state-certified inspector, was not part of the kit rotation. After each tester had completed the testing at a location, the strip of the test location where the color change could be observed was taped over to prevent subsequent testers from knowing the result of the test. After test kit testing was completed, paint chip samples were taken and sent to the laboratory for ICP spectroscopic analysis. Paint samples were homogenized by grinding to a powder, and, if necessary, subsampled prior to analysis. Subsampling was necessary because the total mass of many samples was too large for a single laboratory analysis. The third and final step in the field study was XRF testing. It was conducted by trained and licensed XRF instrument operators employed by independent testing companies. Within each unit, test locations from each substrate type were tested as a group. For example, all locations on metal substrates were tested, then all locations on wood substrates were tested, etc. This was done to minimize the potential for XRF variability caused by repeated substrate changes. However, the order of substrates tested within a unit was varied. Quality control checks were also performed on six control blocks, each composed of a different substrate, combined with the NIST SRM paint films. To ensure that the testing protocol was followed exactly, and to ensure accurate recording of data, during testing each XRF instrument operator was observed by a full-time monitor who recorded the results and reported to a field supervisor. 1.6 PEER REVIEW The technical report on this study was reviewed independently by members of a peer review panel. Comments which are important for interpreting the study results or which had an important impact on the report are discussed below. A comment from a number of reviewers related to the representativeness of the study paint samples and the fact that the sample was not selected randomly from the national housing stock. Although the sample was not randomly selected, the sample did include different substrate materials, housing components, paint thicknesses, and lead levels. The housing in the study included both single-family homes and multifamily housing. The distribution of lead levels in the study is similar to the distribution in the HUD National Survey of pre-1980 housing. A comment from the reviewers related to the training received by the individuals who, as representative homeowners or renters, applied the test kits. There were concerns that it would have been more appropriate to have no training to better simulate what a homeowner or renter would encounter. However, the training did not give the individuals in the study any more information beyond what could have been obtained from a careful reading of the kit instructions. The kits were rotated among the testers to reduce the chance of an individual becoming an expert with a single kit. Nevertheless, it is probably fair to say that the training, the availability of on-site supervisors, and the large number of tests performed by the individual testers provided conditions that exceeded what would be typical for a homeowner or renter who purchased a test kit. A comment was made concerning the impact of spatial variation and laboratory measurement error on the false positive and false negative rates calculated from the study data. A simulation study was conducted to address this comment and the results included in the final technical report. The simulation study demonstrated that the false positive and false negative rates were robust, and therefore accurately portrayed performance of the technologies in the study. Another reviewer comment on the statistical analysis of paint samples with lead below detection levels led to an improvement in the approach for estimating model parameters. A number of reviewers commented on the length of the technical report. In response to those comments, a summary report was developed from the technical report to make the information in the technical report accessible to a wider audience. EPA has established a public record for the peer review under administrative record 142. The record is available in the TSCA Nonconfidential Information Center, which is open from noon to 4 PM Monday through Friday, except legal holidays. The TSCA Nonconfidential Information Center is located in Room NE-B607, Northeast Mall, 401 M Street SW, Washington, D.C. 2 STUDY CONCLUSIONS, TESTING RECOMMENDATIONS, AND SUMMARY OF STUDY RESULTS This section provides conclusions and recommendations for testing as well as a summary of results from the study. The section is divided into two subsections. Section 2.1 contains conclusions and recommendations for testing for lead-based paint and section 2.2 contains a summary of results organized by study objectives. The conclusions, recommendations, and results are based on the samples and data collected in this study, and are specific to the laboratory analysis method, chemical test kits, and XRF instruments used. 2.1 CONCLUSIONS AND RECOMMENDATIONS FOR TESTING 2.1.1 XRF Instrument Conclusions The primary XRF conclusion is that testing using K-shell XRF instruments, with laboratory confirmation of inconclusive XRF results, and with substrate correction in cases where this is effective in reducing bias, is a viable way to test for lead-based paint. This approach can be expected to produce satisfactory results for classifying the paint on architectural components as either above or below the federal threshold of 1.0 mg/cm2. Currently, a common practice is to average a number of readings taken at a single point on an architectural component. The study demonstrated that the most effective method of XRF testing is to obtain readings at different points on the component and compute their average. This recommendation is supported by the variability found in paint samples located approximately nine inches apart, and evidence that a single XRF reading at one point provided almost as much information as an average of three XRF readings at the same point. 2.1.2 Chemical Test Kit Conclusions The conclusion of this study is that test kits should not be used for lead paint testing. Test kits cannot determine the extent of lead-based paint in a home and the need for protecting the occupants, especially when repairs or renovations are carried out. Homeowners and renters cannot be confident that test kits will discriminate accurately between lead-based paint and non-lead based paint. They should not make decisions on repairs, renovations or abatements based on test kit results. 2.2 RESULTS FOR STUDY OBJECTIVES 2.2.1 Precision and Accuracy of XRF Instruments The first primary objective of this study was to characterize the precision and accuracy of XRF instruments on common substrates under field conditions. The results of the study showed that most K-shell instruments exhibited relatively high variability and a high degree of bias at lead levels close to the federal threshold of 1.0 mg/cm2. Nevertheless, K-shell XRF instruments reliably classified the paint samples in this study vis-a-vis the federal threshold of 1.0 mg/cm2, provided a suitable inconclusive range and substrate correction (where appropriate) were used. Test results using L-shell instruments generally exhibited large negative biases which increased with the lead level in the paint. Bias for L-shell instruments was usually substantial at 1.0 mg/cm2 lead. L-shell instruments were less variable than K-shell instruments. As a consequence of the large negative biases, L-shell instruments exhibited a high rate of false negative results when classifying paint using the 1.0 mg/cm2 threshold. When an inconclusive range was added, L-shell instruments, with one exception, still had high rates of false negatives. The one exception exhibited reductions in the rate of false negatives as the inconclusive range was lengthened. 2.2.2 Substrate Interference The second primary objective of the study was to evaluate the effect on the performance of XRF instruments of interference or bias attributable to the underlying substrate and, hence, to evaluate the utility of different approaches for adjusting XRF readings for this bias. The results of the study showed that biases of most K-shell instruments were strongly substrate dependent. Test results using L-shell instruments generally exhibited large negative biases at the 1.0 mg/cm2 threshold that were usually independent of the substrate. Substrate correction obtained using readings on NIST SRM paint films placed on test location areas scraped bare of paint reduced bias for two of the K- shell instruments, and for a third on metal and wood substrates. The already low bias of the fourth K-shell instrument's results was unchanged. Substrate correction using NIST SRM paint films over control blocks was effective in reducing bias for one K-shell instrument, and somewhat effective for a second on plaster, concrete and metal. No method of substrate correction reduced the bias of L-shell readings. 2.2.3 Large XRF Errors A secondary objective of the study was to investigate large errors in the XRF measurements, i.e., measurements that were very different than their corresponding lab results. The results of the study showed that the incidence of large XRF errors was very low (0.6%). Moreover, many of the large errors occurred for several instruments at the same test location. This suggests a common cause other than mere erratic behavior on the part on any single XRF instrument. 2.2.4 Field Quality Assurance and Quality Control Methods Another secondary objective of the study was to evaluate field quality assurance and quality control methods. The study results showed that NIST SRM readings on control blocks were unable to predict XRF instrument performance on painted components in most cases. In particular, the study results showed that erratic behavior in XRF readings taken on control blocks was not necessarily predictive of similarly erratic behavior on actual paint samples. Finally, with the exception of two K-shell instruments used on some substrates, substrate correction using readings on NIST SRM paint films placed on control blocks of substrate materials brought to the site was not effective in reducing biases of readings attributable to substrate interference. 2.2.5 Operating Characteristic Curves for Test Kits The third primary objective of the study was to estimate the operating characteristic curve for each test kit under field conditions. The results of the study showed that the probability of a positive classification when the sample's lead level was equal to the federal thresholds varied depending on the kit and substrate and that high levels of lead would not always be detected by some test kits. Furthermore, there were numerous cases of positive test results at lead levels well below the federal thresholds. None of the test kits used in this study demonstrated low rates of both false positive and false negative results when compared to laboratory analytical results using the federal thresholds, 1.0 mg/cm2 and 0.5%. 2.2.6 Variability of Lead Levels in Paint The third secondary objective of the study was to investigate the variability of lead levels in paint using laboratory measurements of field duplicate samples. The study results showed that the typical relative standard deviation for laboratory analytical measurements in the study samples was 13%. Variability between field duplicate samples was much larger, between 30% - 60% at one standard deviation, indicating significant variability in lead levels between paint samples approximately 9 inches apart. This variability in lead levels within single architectural components, called spatial variability, was the primary cause of variability in the paint samples. 3 DETAILED STUDY RESULTS This section contains details of the study results. 3.1 LEAD LEVELS IN THE STUDY SAMPLES 1. Of the 1,290 paint samples collected and analyzed in the laboratory in this study, approximately 20% contained lead at a level equal to or greater than 1.0 mg/cm2, one of the federal thresholds for defining LBP on painted surfaces. Approximately 29% of the samples contained lead equal to or greater than 0.5% by weight, the other federal threshold for LBP on painted surfaces. Lead levels in the samples were reported by the laboratory as mass per unit area (mg/cm2 lead) and percent lead by weight (%). Table 1 presents a cross-tabulation of lead levels expressed in mg/cm2 and percent lead by weight. The arithmetic mean lead level in the study samples was 1.17 mg/cm2 (1.12%). The median lead level of the study samples was 0.20 mg/cm2 (0.20%). The 25th and 75th percentiles were 0.03 mg/cm2 (0.05%) and 0.62 mg/cm2 (0.72%). The minimum and maximum values were 0.0001 mg/cm2 (0.0004%) and 37.29 mg/cm2 (34.56%). Table 1. Cross-Tabulation of Paint Sample Lead Levels in mg/cm2 Lead and Percent Lead by Weight. ------------------------------------------------------ | Percent | mg/cm2 Lead | | Lead |----------------------------------------- | by Weight | LT 0.5 | 0.5 - 1.0 | GE 1.0 | Totals | |-------------|----------------------------------------- | LT 0.5 | 874 | 42 | 2 | 918 | | 0.5 - 1.0 | 36 | 44 | 14 | 94 | | GE 1.0 | 16 | 25 | 237 | 278 | |-------------|----------------------------------------- | Totals | 926 | 111 | 253 | 1,290 | ------------------------------------------------------ 2. For the paint samples, lead levels expressed in mg/cm2 and lead levels expressed in percent lead by weight were roughly equivalent, as shown by the distribution in Table 1. A level of 1.0 mg/cm2 was roughly equivalent to 1.0% by weight and a level of 0.5% by weight was roughly equivalent to 0.5 mg/cm2. The overall average ratio between the two types of measurement units for the 1,290 primary paint samples analyzed in the laboratory was 1.00. In 80% of the samples, the ratio was between 0.25 and 2.34. A regression plot of results expressed in percent lead by weight (%) versus mass of lead per unit area (mg/cm2) using a logarithmic scale showed good agreement between the two types of measurement units (R2 = 0.91), with the following relationship: PERCENT LEAD = 0.96 x (AREA LEAD)0.85, where PERCENT LEAD = percent lead by weight (%) and AREA LEAD = mass of lead per unit area (mg/cm2). This relationship suggests that 0.5% lead is roughly equivalent to 0.5 mg/cm2 lead, while 1.0 mg/cm2 lead is roughly equivalent to 1.0% lead. This demonstrates that the threshold of 1.0 mg/cm2 lead is typically less stringent than 0.5% lead. 3.2 XRF INSTRUMENTS 1. Most K-shell instruments exhibited relatively high variability, even for paint with low levels of lead. The amount of variability was sometimes related to the level of lead in the sample. Table 2 shows estimated standard deviations for each substrate for results using the K-shell XRF instruments at lead levels of 0.0 mg/cm2 and 1.0 mg/cm2, for a single 15-second (nominal) reading taken on the painted surface of each test location. For XRF instrument results that showed significant variation between instruments and/or cities in the study, a range of values for the standard deviation is also presented. In these cases, the single value in the table represents the single instrument, or a group of similar instruments, with the largest number of readings taken. These estimated standard deviations take into account several sources of variability in addition to instrumental variation. These include site-specific factors such as the substrate composition and the age and thickness of the paint. The MAP-3, Microlead I, and XK-3 results exhibited similar high levels of variability. The Lead Analyzer's results were significantly less variable than the other three. Generally, the instruments' results showedhigher variability at 1.0 mg/cm2 lead than at 0.0 mg/cm2. The difference in variability at the two levels was greatest for the Lead Analyzer's results and least for the MAP-3's results. Variability of control block quality control test results was significantly lower than results for field test locations. Table 3 is the companion to Table 2 for control block test results. The standard deviation at 0.0 mg/cm2 was estimated using XRF test results on the bare control blocks. The standard deviation at 1.0 mg/cm2 was estimated using XRF test results from control blocks covered with the NIST SRM 2579 paint film that has a lead level of 1.02 mg/cm2. As in Table 2, the Lead Analyzer's results were less variable than the results of the other three instruments. For tests on control blocks, the Lead Analyzer's results were more variable at 1.0 mg/cm2 than at 0.0 mg/cm2. However, the other three instruments' results showed similar variability on the control blocks at the two levels, 0.0 and 1.0 mg/cm2. Table 2. Estimated Standard Deviation at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Paint Reading for K-Shell XRF Instruments, by Substrate. ______________________________________________________________________________________ | | LEAD | | | | | | ANALYZER | MAP-3 | MICROLEAD I | XK-3 | | | K-SHELL | K-SHELL | | | |SUBSTRATE |-------------|-------------|-----------------------|-----------------------| | | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | | |mg/cm2|mg/cm2|mg/cm2|mg/cm2| mg/cm2 | mg/cm2 | mg/cm2 | mg/cm2 | |----------|------|------|------|------|-----------|-----------|-----------|-----------| | Brick | 0.17 | 0.23 | 0.93 | 0.93 | 0.59 | 0.55 | 0.60 | 0.60 | | | | | | | | | | | | Concrete | 0.11 | 0.37 | 0.90 | 1.00 | 0.61 | 0.72 | 0.64 | 0.64 | | | | | | |(0.48-1.24)|(0.48-1.31)|(0.51-0.85)|(0.51-0.85)| | Drywall | 0.08 | 0.35 | 0.38 | 0.38 | 0.34 | 0.34 | 0.36 | 0.56 | | | | | | |(0.34-0.53)|(0.34-0.53)|(0.21-0.36)|(0.55-0.56)| | Metal | 0.18 | 0.41 | 0.37 | 0.55 | 0.62 | 0.68 | 0.52 | 1.06 | | | | | | |(0.37-0.81)|(0.55-0.81)|(0.34-0.70)|(0.49-1.63)| | Plaster | 0.14 | 0.24 | 0.81 | 0.87 | 0.55 | 0.64 | 0.55 | 0.63 | | | | | | |(0.37-1.01)|(0.46-1.01)|(0.40-0.55)|(0.40-0.81)| | Wood | 0.08 | 0.43 | 0.49 | 0.67 | 0.62 | 0.92 | 0.49 | 0.69 | | | | | | |(0.50-1.06)|(0.55-1.06)|(0.25-0.51)|(0.44-1.15)| |--------------------------------------------------------------------------------------| | Ranges presented for XRFs demonstrating significant variability between different | | instruments. | -------------------------------------------------------------------------------------- Table 3. Estimated Standard Deviation at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading on Control Blocks for K-Shell XRF Instruments, by Substrate. ____________________________________________________________________________ | | LEAD | | | | | | ANALYZER | MAP-3 | MICROLEAD I | XK-3 | | | K-SHELL | K-SHELL | | | |SUBSTRATE |-------------|-------------|-----------------------|-------------| | | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | | |mg/cm2|mg/cm2|mg/cm2|mg/cm2| mg/cm2 | mg/cm2 |mg/cm2|mg/cm2| |----------|------|------|------|------|-----------|-----------|------|------| | Brick | 0.11 | 0.24 | 0.72 | 0.61 | 0.48 | 0.40 | 0.33 | 0.41 | | | | | | |(0.28-0.76)|(0.26-0.61)| | | | Concrete | 0.11 | 0.24 | 0.64 | 0.67 | 0.38 | 0.50 | 0.41 | 0.50 | | | | | | |(0.31-0.51)|(0.41-0.68)| | | | Drywall | 0.07 | 0.19 | 0.28 | 0.34 | 0.29 | 0.29 | 0.32 | 0.45 | | | | | | |(0.21-0.42)|(0.25-0.44)| | | | Metal | 0.15 | 0.22 | 0.21 | 0.25 | 0.27 | 0.36 | 0.38 | 0.47 | | | | | | |(0.22-2.39)|(0.21-2.14)| | | | Plaster | 0.09 | 0.20 | 0.69 | 0.57 | 0.49 | 0.47 | 0.50 | 0.70 | | | | | | |(0.33-0.66)|(0.30-0.69)| | | | Wood | 0.03 | 0.18 | 0.24 | 0.24 | 0.26 | 0.33 | 0.39 | 0.43 | | | | | | |(0.24-2.08)|(0.23-2.25)| | | |----------------------------------------------------------------------------| | Ranges presented for XRFs demonstrating significant variability between | | different instruments. | ----------------------------------------------------------------------------- 2. Biases of most K-shell instruments were strongly substrate dependent. Bias of an XRF instrument is defined as the average difference between XRF readings and the true lead level in the paint. Table 4 shows biases of the K-shell XRF instruments on the field samples. The results of the Lead Analyzer exhibited low bias on all substrates. The MAP-3's results showed negative bias on brick, concrete, and plaster; positive bias on metal; and low bias on wood and drywall with the exception of wood at 1.0 mg/cm2. The Microlead I's results were mostly positively biased, but with large differences between individual instruments. The XK-3's results showed large positive biases except on wood and drywall, and also exhibited substantial variation between individual instruments. Table 5 shows biases for the K-shell instruments' results, estimated using control block readings. For the Lead Analyzer, control block biases were very small. For the MAP-3, the control block result biases were generally of the same sign, positive or negative, as the field sample result biases, but the magnitudes were very different. For the Microlead I, sporadic agreement existed between control block and field sample result biases. For example, the control block results showed negative bias on metal, while the field sample results showed a positive bias on the same substrate. For the XK-3, the control block result biases usually tracked the field sample result biases. Table 4. Bias at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading for K-Shell XRF Instruments, by Substrate. ______________________________________________________________________________________ | | LEAD | | | | | | ANALYZER | MAP-3 | MICROLEAD I | XK-3 | | | K-SHELL | K-SHELL | | | |SUBSTRATE |-------------|-------------|-----------------------|-----------------------| | | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | | |mg/cm2|mg/cm2|mg/cm2|mg/cm2| mg/cm2 | mg/cm2 | mg/cm2 | mg/cm2 | |----------|------|------|------|------|-----------|-----------|-----------|-----------| | Brick | 0.08 |-0.21 |-0.60 |-0.80 | 0.10 | -0.33 | 0.86 | 0.88 | | | | | | | | | | | | Concrete | 0.02 |-0.01 |-0.66 |-0.45 | 0.28 | 0.38 | 1.08 | 1.75 | | | | | | |(-.03-0.89)|(0.01-1.23)|(0.66-1.84)|(0.23-2.57)| | Drywall |-0.02 | 0.18 | 0.01 |-0.12 | 0.02 | 0.22 | -0.33 | -0.09 | | | | | | |(0.00-0.66)|(0.16-1.79)|(-.33-0.25)|(-.09-0.18)| | Metal | 0.06 | 0.02 | 0.33 | 0.42 | 0.35 | 0.45 | 0.45 | 0.86 | | | | | | |(-.42-1.08)|(-.17-1.36)|(0.26-1.48)|(0.81-1.69)| | Plaster | 0.03 |-0.11 |-0.68 |-0.55 | 0.01 | 0.06 | 0.54 | 0.57 | | | | | | |(-.09-0.22)|(-.32-0.18)|(0.38-1.68)|(0.18-1.63)| | Wood | 0.01 | 0.28 |-0.05 | 0.36 | 0.00 | 0.43 | -0.07 | 0.35 | | | | | | |(0.00-0.60)|(0.18-0.90)|(-.07-0.93)|(0.31-1.23)| |--------------------------------------------------------------------------------------| | Ranges presented for XRFs demonstrating significant variability between different | | instruments. | ---------------------------------------------------------------------------------------- Table 5. Bias at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading on Control Blocks for K-Shell XRF Instruments, by Substrate. ____________________________________________________________________________ | | LEAD | | | | | | ANALYZER | MAP-3 | MICROLEAD I | XK-3 | | | K-SHELL | K-SHELL | | | |SUBSTRATE |-------------|-------------|-----------------------|-------------| | | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | | |mg/cm2|mg/cm2|mg/cm2|mg/cm2| mg/cm2 | mg/cm2 |mg/cm2|mg/cm2| |----------|------|------|------|------|-----------|-----------|------|------| | Brick | 0.05 | 0.08 |-1.18 |-0.05 | 0.47 | 0.45 | 0.97 | 1.10 | | | | | | |(-.10-0.51)|(-.31-0.54)| | | | Concrete |-0.01 | 0.06 |-1.20 |-0.18 | 0.57 | 0.70 | 0.89 | 1.00 | | | | | | |(0.15-1.43)|(0.25-1.59)| | | | Drywall |-0.01 | 0.06 |-0.10 | 0.04 | 0.03 | 0.12 | 0.17 | 0.48 | | | | | | |(-.62-0.14)|(-.56-0.18)| | | | Metal |-0.01 | 0.11 | 0.23 | 0.18 | -0.34 | -0.35 | 1.10 | 1.34 | | | | | | |(-.82-2.25)|(-.84-2.00)| | | | Plaster |-0.03 | 0.05 |-1.38 |-0.64 | 0.45 | 0.40 | 0.83 | 0.83 | | | | | | |(0.06-1.13)|(0.09-1.02)| | | | Wood |-0.00 | 0.04 |-0.27 |-0.14 | 0.15 | 0.18 | 0.25 | 0.49 | | | | | | |(-.22-1.57)|(-.05-1.47)| | | |----------------------------------------------------------------------------| | Ranges presented for XRFs demonstrating significant variability between | | different instruments. | ----------------------------------------------------------------------------- 3. With the exception of the XL prototype, test results using L-shell instruments exhibited large negative biases at the 1.0 mg/cm2 threshold. However, test results using L-shell instruments were less variable than results obtained using K-shell instruments. Table 6 shows estimated biases of field sample results using L-shell instruments at 0.0 mg/cm2 and 1.0 mg/cm2. The instruments' results show little bias at 0.0 mg/cm2. However, large negative biases, typically between -0.7 and -0.9 mg/cm2, at 1.0 mg/cm2 lead, are shown for all L-shell instruments' results except those obtained using the XL. Standard deviations were usually 0.2 mg/cm2 or less for field sample test results at both 0.0 and 1.0 mg/cm2 lead, although the MAP-3's L-shell results showed slightly higher variability than this on metal. Variability of control block results was significantly lower for all L-shell instruments compared to K-shell instruments' results. Table 6. Bias at 0.0 mg/cm2 and 1.0 mg/cm2 Lead for One Nominal 15-Second Reading for L-Shell XRF Instruments, by Substrate. __________________________________________________________________ | | LEAD | | | | | | ANALYZER | MAP-3 | MICROLEAD I | XK-3 | | | K-SHELL | K-SHELL | | | |SUBSTRATE |-------------|-------------|-------------|-------------| | | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | 0.0 | 1.0 | | |mg/cm2|mg/cm2|mg/cm2|mg/cm2|mg/cm2|mg/cm2|mg/cm2|mg/cm2| |----------|------|------|------|------|------|------|------|------| | Brick | 0.01 |-0.77 | 0.01 |-0.88 | 0.11 |-0.40 | 0.03 |-0.74 | | Concrete | 0.01 |-0.84 |-0.14 |-0.94 | 0.07 |-0.15 | 0.05 |-0.89 | | Drywall |-0.01 |-0.70 |-0.12 |-0.62 | 0.08 |-0.63 | 0.04 |-0.74 | | Metal | 0.01 |-0.79 | 0.04 |-0.69 | 0.07 |-0.10 | 0.11 |-0.77 | | Plaster | 0.002|-0.80 |-0.12 |-0.96 | 0.08 |-0.26 | 0.05 |-0.88 | | Wood |-0.02 |-0.74 |-0.08 |-0.65 | 0.06 |-0.30 | 0.04 |-0.70 | ------------------------------------------------------------------- 4. The XL results showed smaller biases at 1.0 mg/cm2 than results of the other L-shell instruments, but still showed large negative biases at higher lead levels. Biases of the XL's results at 1.0 mg/cm2 lead range from -0.10 to -0.63 mg/cm2. There was some variation in bias between different XL machines on metal and wood at 1.0 mg/cm2. The instrument's results showed large negative biases at higher lead levels. For example, it read 1.0 mg/cm2 or less on 26% of the samples with lead levels of 10.0 mg/cm2or greater. The XL instruments used in this study were prototype models. 5. Substrate correction obtained using readings for NIST SRM paint films placed on test location areas scraped bare of paint reduced bias for results using the Microlead I and the XK-3, and for the MAP-3 K-shell instrument results on metal and wood substrates. The already low bias of the Lead Analyzer's K-shell results was unchanged. Two methods of substrate correction using NIST SRM paint films placed on the bare substrate were analyzed. In the first method, called "full" correction, readings were taken at each individual test location after the NIST SRM paint film was placed on the bare area of the substrate. These readings provided an offset value used to correct the paint sample readings taken at that location. The second method, called "average" correction, used the average of all readings taken after the NIST SRM paint film was placed on the bare area at test locations of the same substrate in the entire dwelling unit. These average readings provided an offset value used to correct paint sample readings taken on the same substrate in a dwelling unit. Full correction is not a practical method, while average correction approximates the method recommended in the 1990 HUD Guidelines. The two methods were found to give approximately the same results. 6. With the exception of the XK-3 and the MAP-3 on some substrates, substrate correction using readings for NIST SRM paint films placed on control blocks of substrate materials brought to the site was not effective in reducing biases of K-shell readings attributable to substrates. A third method of correcting for bias attributable to substrates, called "control block" correction, used the average of readings taken on control blocks after the SRM paint film was placed on the control block. These average readings provided an offset value used to correct paint sample readings taken on the same substrate. Control block correction was not a generally effective technique to detect location-dependent substrate characteristics which cause the results to show bias. An exception was the XK-3 instrument. This instrument's results typically exhibited positive bias which was reduced significantly by control block correction. For the MAP-3, control block correction was somewhat effective in reducing bias for plaster, concrete, and metal. For the Microlead I, control block correction actually increased bias for metal and plaster. 7. No method of substrate correction reduced the bias of L-shell readings. Neither the use of control blocks nor readings taken after placing NIST SRM paint films on scraped substrates was effective in reducing the biases in L-shell readings. This is because L-shell result bias is caused by difficulty in detecting lead in deeper layers of paint, which was not simulated by usage of the NIST SRM paint films. 8. Despite the generally high variability and bias of their results, K-shell XRF instruments reliably classified the paint samples in this study using the federal threshold of 1.0 mg/cm2, with laboratory confirmation of XRF readings between 0.4 and 1.6 mg/cm2 and correction of biases attributable to substrates as needed. Classify a paint sample as positive if the first 15-second (nominal) K-shell XRF reading (substrate corrected as appropriate) taken on paint is 1.6 mg/cm2 or greater, as negative if the reading is 0.4 mg/cm2 or less; otherwise the paint sample is classified as inconclusive. Inconclusive readings are to be resolved by laboratory analysis. Using the ICP spectroscopic analysis of the paint sample to determine whether the lead level was actually greater than or equal to 1.0 mg/cm2, the overall false positive, false negative and inconclusive rates for the K-shell XRF instruments are shown in Table 7. With the exception of the XK-3 false positive rate, all error rates were below 10%. The false positive rate for the XK-3 was dramatically reduced by either method of substrate correction. For each substrate type, most error rates were still below 10%. The exceptions were MAP-3 false negative rates on concrete and plaster, the Microlead I false positive rate on wood, and the XK-3 false negative rate on metal. It is important to remember that these classification results apply strictly only to the set of samples and instruments in this study. Classification results for a different set of samples or instruments could be different. Table 7. False Positive, False Negative and Inconclusive Percentages for K-Shell XRF Instruments, Based on One Nominal 15-Second Reading With an INCONCLUSIVE RANGE OF 0.4 - 1.6 mg/cm2 (1.0 mg/cm2 Threshold). _________________________________________________________________________ | |FALSE POSITIVE|FALSE NEGATIVE|INCONCLUSIVE| | INSTRUMENT | PERCENTAGE | PERCENTAGE | PERCENTAGE | |------------------------------|--------------|--------------|------------| |Lead Analyzer K-shell | 0.5% | 1.4% | 18% | |MAP-3 K-shell | 2.3% | 3.7% | 23% | |Microlead I | 7.5% | 1.1% | 30% | |XK-3 | 22% | 1.1% | 35% | |XK-3 (Average Corrected) | 2.3% | 4.2% | 25% | |XK-3 (Control Block Corrected)| 3.5% | 4.0% | 25% | --------------------------------------------------------------------------- 9. When the laboratory confirmation range was narrowed to 0.7 to 1.3 mg/cm2, thereby substantially reducing the inconclusive percentages, the K-shell instruments continued to reliably classify paint samples in this study. Table 8 shows similar data to Table 7 with the narrower inconclusive range. Results of the Microlead I and the XK-3 both needed substrate correction to achieve satisfactory false positive rates. For each substrate type, error rates were generally below 10%. The exceptions were MAP-3 false negative rates on concrete and plaster, the Microlead I false negative rate on concrete, XK-3 false negative rates on metal and plaster, and the XK-3 false positive rate on concrete. Inconclusive percentages are reduced by at least 50% for all XRF instruments compared to the inconclusive percentages when classifying paint samples using the 0.4 - 1.6 mg/cm2 inconclusive range. Table 8. False Positive, False Negative and Inconclusive Percentages for K-Shell XRF Instruments, Based on One Nominal 15-Second Reading With an INCONCLUSIVE RANGE OF 0.7 - 1.3 mg/cm2 (1.0 mg/cm2 Threshold). __________________________________________________________________________ | |FALSE POSITIVE|FALSE NEGATIVE|INCONCLUSIVE| | INSTRUMENT | PERCENTAGE | PERCENTAGE | PERCENTAGE | |-------------------------------|--------------|--------------|------------| |Lead Analyzer K-shell | 1.2% | 2.7% | 6% | |MAP-3 K-shell | 4.1% | 4.6% | 11% | |Microlead I | 12% | 2.1% | 15% | |Microlead I (Average Corrected)| 4.9% | 5.3% | 12% | |XK-3 | 30% | 1.7% | 17% | |XK-3 (Average Corrected) | 5.5% | 6.6% | 12% | |XK-3 (Control Block Corrected) | 6.5% | 6.8% | 12% | ---------------------------------------------------------------------------- 10. Without a laboratory confimation range, the K-shell instruments' performance differed when classifying paint samples in this study using the federal threshold of 1.0 mg/cm2. Based on readings obtained using the K-shell instruments, paint samples were classified as positive if the XRF reading was 1.0 mg/cm2 or higher and negative otherwise. There was no inconclusive range. False positive and false negative rates for the K-shell instruments' results are shown in Table 9. As expected, these rates are higher than when inconclusive ranges were used, but still no greater than 11% overall when substrate correction methods are employed as needed. False positive and false negative rates for readings on particular substrates were substantially higher than the overall rates as exemplified by the following ranges. For all of the K-shell instruments, the lowest false positive or false negative rate on a particular substrate was less than 2.0%. However, on the high end, the Lead Analyzer's false negative rate on concrete was 11%, the MAP-3's false negative rate on concrete was 24%, the Microlead I's false positive rate on wood was 26%, and the XK-3's false positive rate on concrete was 66%. Table 9. False Positive and False Negative Percentages for K-Shell XRF Instruments, Based on One Nominal 15-Second Reading With NO INCONCLUSIVE RANGE (1.0 mg/cm2 Threshold). _____________________________________________________________ | |FALSE POSITIVE|FALSE NEGATIVE| | INSTRUMENT | PERCENTAGE | PERCENTAGE | |-------------------------------|--------------|--------------| |Lead Analyzer K-shell | 3.1% | 5.9% | |MAP-3 K-shell | 8.0% | 8.3% | |Microlead I | 20% | 3.8% | |Microlead I (Average Corrected)| 9% | 9% | |XK-3 | 40% | 3.6% | |XK-3 (Average Corrected) | 11% | 10% | |XK-3 (Control Block Corrected) | 11% | 11% | --------------------------------------------------------------- 11. With the exception of the XL, L-shell instruments performed poorly when classifying paint using the 1.0 mg/cm2 threshold, because of a high rate of false negative results. Table 10 shows false positive, false negative and inconclusive percentages for tests using L-shell instruments and an inconclusive range of 0.4 to 1.6 mg/cm2. With the exception of the XL, the false negative rates for the L-shell instruments' results were very high, due to the large negative biases shown in the results using these instruments. False positive rates were very low for all L-shell instruments' results. Table 10. False Positive, False Negative and Inconclusive Percentages for L-Shell XRF Instruments, Based on One Nominal 15-Second Reading with an INCONCLUSIVE RANGE OF 0.4 - 1.6 mg/cm2 (1.0 mg/cm2 Threshold). ___________________________________________________________________ | |FALSE POSITIVE|FALSE NEGATIVE|INCONCLUSIVE| | INSTRUMENT | PERCENTAGE | PERCENTAGE | PERCENTAGE | |------------------------|--------------|--------------|------------| |Lead Analyzer L-shell | 0.0% | 66% | 6% | |MAP-3 L-shell | 0.0% | 37% | 12% | |XL | 0.1% | 11% | 15% | |X-MET 880 | 0.0% | 66% | 7% | --------------------------------------------------------------------- 12. Although the XL prototype had a lower rate of false negative results than the other L-shell instruments, it still exhibited false negative results at very high lead levels. As shown in Table 10, the XL had a false negative rate of approximately 11% and a false positive rate of 0.1% using an inconclusive range of 0.4 to 1.6 mg/cm2. However, of the 38 instances where the ICP measurement exceeded 10 mg/cm2, 2 of the XL readings were below 0.4 mg/cm2 and one was equal to 0.4 mg/cm2. In all 3 cases, a paint sample with an ICP result above 10 mg/cm2 was classified as negative for lead-based paint. With a narrower inconclusive range of 0.7 to 1.3 mg/cm2, the XL had an overall false negative rate of 24.1% and a 0.2% false positive rate. Classifying the XL results without an inconclusive range yielded a 41.8% false negative rate and a 0.5% false positive rate. 13. Generally, a single XRF reading at one point of an architectural component provided almost as much accuracy as an average of three XRF readings at the same point. When paint samples were classified as positive for XRF results 1.6 mg/cm2 or greater, negative for XRF results 0.4 mg/cm2 or less, or inconclusive, otherwise, and the results were compared to the lead level obtained from the ICP spectroscopic analysis of the paint sample, there was very little difference in the false positive and false negative rates for the average of three 15-second readings versus a single 15-second reading. The small improvement in classification accuracy did not justify the additional time and expense of taking three readings at the same point. This remained true when substrate correction and different inconclusive ranges were employed. A similar conclusion was reached when the precision of the average of three 15-second readings, as measured by its standard deviation, was compared to that of a single reading. If the three readings were statistically independent, one would expect the standard deviation of the average to be 58% of the standard deviation of a single reading. However, it was found that the standard deviation of the average was much greater than this. For L-shell instruments, the standard deviation of the average was typically at least 95% of the standard deviation of a single reading. For K-shell instruments, the standard deviation of the average was typically between 76% and 93% of the standard deviation of a single reading. There are two reasons why taking the average of three readings did not produce the expected gains in precision. First, with the exception of the MAP-3 K-shell instrument's readings, successive readings at the same point were positively correlated. Thus, the reduction in variability from averaging repeat readings was less than would be achieved if successive readings had been statistically independent. The second reason why the average produced a smaller reduction in variability than expected is that repeated readings reduced only the component of variability due solely to the performance of the instrument. The study data clearly demonstrated that there were additional sources of variability that were generally at least as large as the component due to the performance of the XRF instrument. Taking repeated readings does not reduce the variability due to these other sources. The additional variability was due to location-specific factors, such as paint and substrate composition. 3.3 CHEMICAL TEST KITS 1. None of the test kits used in this study demonstrated low rates of both false positive and false negative results when compared to laboratory analytical results using the federal thresholds, 1.0 mg/cm2 and 0.5%. Table 11 shows overall false positive and false negative rates for the test kits compared to laboratory analytical results using the 1.0 mg/cm2 threshold. Table 12 shows the corresponding rates for the 0.5% threshold. Rates for the Lead Alert kits exclude results of tests on painted plaster substrates since the manufacturer does not recommend use of these kits on plaster. For the 1.0 mg/cm2 threshold, State Sodium Sulfide and LeadCheck had low false negative rates, but high false positive rates. Lead Alert: Sanding had a low false positive rate, but a high false negative rate. The other three kits tested, Lead Zone, Lead Detective, and Lead Alert: Coring, had moderate to high rates of both false positive and false negative results. For the 0.5% threshold, State Sodium Sulfide had a low false negative rate and Lead Alert: Sanding had a low false positive rate. False negative rates for LeadCheck and false positive rates for Lead Alert: Coring were slightly above 10%. Lead Zone and Lead Detective had high rates of both false positive and false negative results. As was pointed out for XRFs, it is important to remember that these classification results apply strictly only to the set of samples and kits in this study. Classification results for a different set of samples or kits could be different. Table 11. Overall False Positive and False Negative Rates for Test Kits Compared to Laboratory Analytical Results Using the 1.0 mg/cm2 Threshold. ____________________________________________________ | |FALSE POSITIVE|FALSE NEGATIVE| | TEST KIT | PERCENTAGE | PERCENTAGE | |----------------------|--------------|--------------| |LeadCheck | 46% | 6% | |Lead Alert: Coring | 15% | 24% | |Lead Alert: Sanding | 9% | 53% | |Lead Detective | 36% | 23% | |Lead Zone | 28% | 14% | |State Sodium Sulfide | 65% | 1% | ---------------------------------------------------- Table 12. Overall False Positive and False Negative Rates for Test Kits Compared to Laboratory Analytical Results Using the 0.5% Threshold. ____________________________________________________ | |FALSE POSITIVE|FALSE NEGATIVE| | TEST KIT | PERCENTAGE | PERCENTAGE | |----------------------|--------------|--------------| |LeadCheck | 42% | 11% | |Lead Alert: Coring | 11% | 36% | |Lead Alert: Sanding | 10% | 67% | |Lead Detective | 32% | 27% | |Lead Zone | 25% | 25% | |State Sodium Sulfide | 62% | 6% | ---------------------------------------------------- 2. The substrate underlying the paint sometimes affected false positive and false negative rates for test kits. LeadCheck: For both federal thresholds, the false positive rate on drywall was considerably lower than on the other five substrates. False negative rates in mg/cm2 on concrete and plaster were higher than on the other substrates. For percent by weight, false negative rates were higher on concrete, drywall, metal, and plaster than on brick and wood. Some of these differences in false negative rates may be caused by sulfates found in plaster dust, gypsum and stucco. The kit includes a confirmation procedure to guard against false negative results caused by sulfates. Lead Alert: Coring: The manufacturer states that this kit is prone to negative interferences from gypsum and plaster dust. High false negative rates were observed on plaster and drywall for percent lead by weight measurements and on plaster for mg/cm2 measurements. However, the sample size for drywall was very small. False negative rates on brick were much lower than on the other substrates for both types of measurements. For mg/cm2 measurements, false positive rates were lowest on plaster and drywall substrates, and highest on brick. For percent lead by weight measurements, false positive rates were lowest on drywall, plaster, and wood substrates, and highest on brick. Lead Alert: Sanding: This kit had a very similar pattern to Lead Alert: Coring with high false negative rates on plaster and drywall, and the highest false positive rate on brick. Lead Detective: The manufacturer does not recommend use on metal, but does recommend application on wood, drywall, and plaster. False positive rates were consistent for both types of measurements on all substrates except brick, which had a higher false positive rate. False negative rates were lowest on wood and highest on brick and concrete substrates. (Results were observed showing that drywall had the highest false negative rate for percent lead by weight units, but the sample size was very small.) Thus, this kit did not perform much better on wood, plaster, and drywall than on metal so that the manufacturer's recommendations were not borne out by the study data. Lead Zone: The manufacturer's instructions only mention testing on wood and metal. False positive rates were the same on all substrates for both types of measurements. False negative rates were lower on brick, wood, and concrete, and higher on the other substrates. The false negative rate on metal was the highest of all substrates using percent lead by weight measurements. The manufacturer's instructions do not include mention of using this kit on substrates where it performed similarly to its performance on wood, but do mention its use on metal, where its false negative rate was substantially larger than its false negative rate on wood. State Sodium Sulfide: The instructions contain a caution not to test directly on metal. For metal substrates, a paint chip can be removed and tested separate from the substrate. This kit had very high false positive rates for both types of measurements on all substrates except drywall. False negative rates were low on all substrates for mg/cm2 measurements. For percent lead by weight measurements, this kit had higher false negative rates on metal, plaster, and drywall than on the other substrates. 3. The probability of a positive classification when the sample's lead level was equal to the federal thresholds varied depending on the kit and substrate. High levels of lead would not always be detected using test kits alone. Table 13 shows the probability of a positive result using a test kit on paint with a lead level equal to the 1.0 mg/cm2 federal threshold, as estimated from the statistical model developed in this study. Table 14 is the companion table for the other federal threshold of 0.5% by weight. Considerable variation among results for each kit and each substrate is seen in the tables. High levels of lead were not always detected with complete certainty using test kits. The statistical model estimated the limiting probability of a positive test kit result at high levels of lead using the laboratory ICP spectroscopic results reported in mg/cm2 units. In a number of cases, the limiting probability was much lower than the desired value of 100%. This occurred for four of the six kits: Lead Alert: Coring on metal; Lead Alert: Sanding on concrete, metal, and wood; Lead Detective on concrete, metal, and plaster; and Lead Zone on plaster. Table 13. Probability of a Positive Test Kit Result at 1.0 mg/cm2 Lead. -------------------------------------------------------------------------- | TEST KIT | Brick | Concrete | Drywall | Metal | Plaster | Wood | |--------------------|-------|----------|---------|-------|---------|------| |LeadCheck | 0.95 | 0.69 | 0.49 | 0.93 | 0.69 | 0.91 | |Lead Alert: Coring | 0.93 | 0.27 | N/A | 0.66 | N/A | 0.57 | |Lead Alert: Sanding| N/A | 0.50 | N/A | 0.39 | N/A | 0.02 | |Lead Detective | 0.81 | 0.58 | 0.34 | 0.74 | 0.51 | 0.78 | |Lead Zone | 0.82 | 0.27 | 0.64 | 0.59 | 0.55 | 0.80 | |State Sodium Sulfide| 0.99 | 0.95 | 0.68 | 0.94 | 0.95 | 0.95 | -------------------------------------------------------------------------- Table 14. Probability of a Positive Test Kit Result at 0.5% Lead. -------------------------------------------------------------------------- | TEST KIT | Brick | Concrete | Drywall | Metal | Plaster | Wood | |--------------------|-------|----------|---------|-------|---------|------| |LeadCheck | 0.95 | 0.68 | 0.48 | 0.62 | 0.68 | 0.83 | |Lead Alert: Coring | 0.73 | 0.23 | N/A | 0.26 | N/A | 0.28 | |Lead Alert: Sanding| N/A | 0.13 | N/A | 0.05 | N/A | 0.03 | |Lead Detective | 0.80 | 0.55 | 0.31 | 0.43 | 0.46 | 0.58 | |Lead Zone | 0.81 | 0.51 | 0.55 | 0.19 | 0.53 | 0.62 | |State Sodium Sulfide| 0.93 | 0.93 | 0.59 | 0.83 | 0.91 | 0.87 | -------------------------------------------------------------------------- 4. The lead level at which there was a 50% chance of the occurrence of a positive test kit result varied depending on the kit and substrate. In many cases, positive results occurred even when paint with very low lead levels was tested. Table 15 shows the lead level in mg/cm2 at which each kit had an estimated 50% probability of a positive result, by substrate. Table 16 is the companion table in percent lead by weight measurements. There was significant variation in 50% probability levels for different kits used on the same substrate. There was also significant variation in the 50% probability levels for the same kit used on different substrates. One exception, the State Sodium Sulfide kit, reached a 50% probability of a positive result at low lead levels on all substrates for both types of measurements. The statistical model used to analyze the test kit data also provided estimates of the limiting probability of a positive result as the lead level in the paint sample approached zero using the laboratory ICP spectroscopic results reported in mg/cm2 units. It is desirable that this limiting probability be zero; otherwise, the kit will produce some positive results even for paint samples with very low lead levels. However, every kit exhibited a non-zero limiting probability of a positive result on at least one substrate. This occurred on metal substrates for all six kits. With the sodium sulfide kits, Lead Detective and State Sodium Sulfide, most substrates had a non-zero limiting probability of a positive result. For the other 4 test kits, limiting probabilities of a positive result equaled or exceeded 20% for LeadCheck on metal and plaster, Lead Alert: Coring on brick, and Lead Zone on concrete. For LeadCheck, Lead Detective and State Sodium Sulfide, limiting probabilities for the wood substrate were positive. Table 15. Lead Level in mg/cm2 at Which There is a 50% Probability of a Positive Test Kit Result. -------------------------------------------------------------------------- | TEST KIT | Brick | Concrete | Drywall | Metal | Plaster | Wood | |--------------------|-------|----------|---------|-------|---------|------| |LeadCheck | 0.02 | 0.19 | 1.14 | 0.34 | 0.13 | 0.03 | |Lead Alert: Coring | 0.33 | 1.84 | N/A | 0.65 | N/A | 0.77 | |Lead Alert: Sanding| N/A | N/A | N/A | N/A | N/A | 1.24 | |Lead Detective | 0.05 | 0.60 | N/A | 0.55 | 0.98 | 0.20 | |Lead Zone | 0.08 | 1.38 | 0.31 | 0.82 | 0.71 | 0.15 | |State Sodium Sulfide| 0.01 | 0.01 | 0.08 | 0.08 | 0.02 | 0.04 | -------------------------------------------------------------------------- Table 16. Lead Level in Percent Lead by Weight at Which There is a 50% Probability of a Positive Test Kit Result. -------------------------------------------------------------------------- | TEST KIT | Brick | Concrete | Drywall | Metal | Plaster | Wood | |--------------------|-------|----------|---------|-------|---------|------| |LeadCheck | 0.02 | 0.16 | 0.56 | 0.32 | 0.14 | 0.07 | |Lead Alert: Coring | 0.13 | 1.14 | N/A | 1.09 | N/A | 0.97 | |Lead Alert: Sanding| N/A | 0.88 | N/A | N/A | N/A | 1.68 | |Lead Detective | 0.01 | 0.33 | N/A | 0.63 | 0.58 | 0.36 | |Lead Zone | 0.07 | 0.49 | 0.35 | 1.03 | 0.44 | 0.26 | |State Sodium Sulfide| 0.01 | 0.01 | 0.13 | 0.08 | 0.02 | 0.09 | -------------------------------------------------------------------------- 3.4 PAINT CHIP SAMPLING AND ANALYSIS 1. Lead levels in paint showed significant variation within individual architectural components such as doors, walls, and baseboards. Duplicate paint samples were taken approximately 9 inches apart on the same component at 10% of the test locations in the full study in Denver and Philadelphia. Duplicate paint samples taken from the same component were called duplicate pairs. The estimated median ratio of the larger to the smaller ICP spectroscopic result, measured in mg/cm2, for duplicate pairs was 1.6 in Denver and 1.3 in Philadelphia. The corresponding median ratios for percent lead by weight units were 1.5 and 1.2. The estimated 95th percentile for the ratio in mg/cm2 was 3.7 in Denver and 2.1 in Philadelphia. The corresponding 95th percentile ratios for percent lead by weight units were 3.1 and 1.9. There was slightly greater variability in lead levels within architectural components when measured in mg/cm2 than in percent lead by weight. The extent to which greater variability would be observed between samples taken farther apart than 9 inches is not addressed by the study data. Variability in duplicate samples could result in different classification of paint depending on which member of the pair was compared to the federal threshold. If the lead level of a paint sample was equal to or greater than the federal threshold, it was classified as positive for lead- based paint. Likewise, if the sample was less than the federal threshold, then it was classified as negative. Of 128 total duplicate pairs in the study, 10 (8%) had different classifications, one sample positive and the other negative for lead, compared to the 1.0 mg/cm2 threshold, while 8 (6%) had different classifications compared to the 0.5%threshold. Spatial variation in lead levels within single architectural components complicated the statistical analysis of XRF and test kit performance data in the study. Complex statistical models were needed to account for the impact of spatial variation on estimates of XRF measurement bias and standard deviation. Spatial variation had a smaller impact on the test kit data analysis. 2. Variation between members of laboratory duplicate subsample pairs was much smaller than variation between members of duplicate samples obtained in the field. Laboratory analytical measurement error for ICP spectroscopic analysis of 2 x 2 inch paint chip samples, including homogenization, subsampling and instrumental error, can be quantified using the ratio of the larger to the smaller ICP measurement for a pair of subsamples of the same sample. The estimated median for this error ratio was 1.13 for samples taken from smooth substrates with no unusual difficulty in paint removal. The estimated 95th percentile for the error ratio was 1.4. These ratios apply to laboratory results reported in both in mg/cm2 and percent lead by weight units. Laboratory measurement error was approximately constant across metal, wood, plaster, and drywall substrates, across cities, and across samples within a substrate or within a city. For samples taken on rough substrates such as brick or concrete, total laboratory analytical measurement error was higher: the estimated median ratio was 1.2 and the estimated 95th percentile ratio was 1.8. Only two laboratory duplicate pairs out of a total of 171 (1%) had different classifications, one of the pair positive and one negative, with respect to the 1.0 mg/cm2 threshold. For the 0.5% threshold, three subsample pairs out of 171 (2%) had different classifications. 50272-101 -------------------------------------------------------------------------- |REPORT DOCUMENTATION|1. REPORT NO. |2. |3. Recipient's Accession No.| |PAGE |EPA 747-R-95-002a| | | |--------------------------------------------------------------------------| |4. Title and Subtitle |5. Report Date | | | May 1995 | | A FIELD TEST OF LEAD-BASED PAINT TESTING TECHNOLOGIES: |----------------| | SUMMARY REPORT |6. | | | | |--------------------------------------------------------------------------| |7. Author(s) |8. Performing Organization | | | Rept. No. | | Cox, D.C.; Dewalt, F.G.; Haugen, M.M.; | | | Koyak, R.A.; Schmehl, R.L. | | |--------------------------------------------------------------------------| |9. Performing Organization Name and Address |10. | | | | | QuanTech, Inc. Midwest Research Institute|-----------------| | 1911 North Fort Myer Drive 425 Volker Boulevard |11. Contract(C) | | Suite 1000 & Kansas City, Missouri |or Grant (G) No. | | Rosslyn, Virginia 22209 64110 | 68-DO-0137 | |--------------------------------------------------------------------------| |12. Sponsoring Organization Name and Address |13. Type of Report &| | | Period Covered | | U.S. Environmental Protection Agency | Summary Report | | Office of Pollution, Pesticides and Toxic Substances|--------------------| | Washington, DC 20460 |14. | | | | |--------------------------------------------------------------------------| |15. Supplementary Notes | | | | In addition to the authors listed above, the following key staff members | | were major contributors to the study: Paul Constant, Donna Nichols, | | Jack Balsinger, Nancy Friederich, and John Jones of Midwest Research | | Institute; and Connie Reese of QuanTech. | | | |--------------------------------------------------------------------------| |16. Abstract (Limit: 200 words) | | | | A large field study was conducted to compare three methods commonly used | | to test for lead in paint: portable X-ray fluorescence (XRF) | | instruments, lead paint test kits, and laboratory analysis of paint chip | | samples. Laboratory analysis is considered to be the most accurate of | | the three methods and was the benchmark for comparisons. The study | | concludes that use of K-shell XRFs, with laboratory confirmation of | | readings designated as inconclusive and with correction of substrate | | biases where appropriate, is an acceptable way to classify painted | | architectural components versus the federal threshold of 1.0 mg/cm2. | | The study concludes that test kits should not be used to test for lead | | in paint. No test kit in the study achieved low rates of both false | | positive and false negative results. Some kits yielded a positive result| | at low levels of lead. Other kits were prone to a negative result when | | lead in paint was above the federal thresholds of 1.0 mg/cm2 and 0.5% by | | weight. | | | |--------------------------------------------------------------------------| | 7. Document Analysis a. Descriptors | | | | Lead-based paint, lead-based paint testing, comparability study, field | | evaluation, recommendations for testing for lead in paint | | | | b. Identifiers/Open-Ended Terms | | | | X-ray fluorescence instrument, XRF instrument, portable XRF, lead | | paint test kit, chemical test kit, test kit, inductively coupled | | plasma-atomic emission spectrometry, ICP-AES, ICP | | | | c. COSATI Field/Group | | | |--------------------------------------------------------------------------| |18. Availability Statement|19. Security Class (This Report)|21.No of Pages| | | Unclassified | 53 | | |--------------------------------|--------------| | |20. Security Class (This Page) |22. Price | | | Unclassified | | --------------------------------------------------------------------------- (See ANSI-Z39.18) OPTIONAL FORM 272 (4-77) be used to test for lead | | in paint. No test kit in the study achieved low rates of both false | | positive and false negative results. Some kits yielded a positive result| | at low levels of lead. Other kits were prone to a negative result when | | lead in paint was above the federal thresholds of 1.0 mg/cm2 and 0.5% by | | weight. | | | |--------------------------------------------------------------------------| | 7. Document Analysis a. Descriptors | | | | Lead-based paint, lead-based paint testing, comparability study, field | | evaluation, recommendations for testing for lead in paint | | | | b. Identifiers/Open-Ended Terms | | | | X-ray fluorescence instrument, XRF instrument, portable XRF, lead | | paint test kit, chemical test kit, test kit, inductively coupled | | plasma-atomic emission spectrometry, ICP-AES, ICP | | | | c. COSATI Field/Group | | | |--------------------------------------------------------------------------| |18. Availability Statement|19. Security Class (This Report)|21.No of Pages| | | Unclassified | 53 | | |--------------------------------|--------------| | |20. Security Class (This Page) |22. Price | | | Unclassified fied | | --------------------------------------------------------------------------- (See ANSI-Z39.18) OPTIONAL FORM 272 (4-77) (Formerly NTIS-35) Department of Commerce