Validation Guidelines for Laboratories Performing Forensic Analysis of Chemical Terrorism
Scientific Working Group on Forensic Analysis of Chemical Terrorism (SWGFACT)
Preface
The Scientific Working Group on Forensic Analysis of Chemical Terrorism has developed the following guidelines for laboratories engaged in the forensic analysis of chemical evidence associated with terrorism. This document provides a baseline framework and guidance for a validation program for forensic analytical procedures. Consideration may be given to alternative approaches of achieving the intent of these guidelines.
Introduction
The Scientific Working Group on Forensic Analysis of Chemical
Terrorism's mission is "to develop guidelines for the forensic identification,
characterization, and attribution of evidence in planned, threatened,
or actual acts of chemical terrorism." The Validation Guidelines
for Laboratories Performing Forensic Analysis of Chemical Terrorism
may be used by laboratories to structure or enhance their validation
of procedures used in the forensic analysis of chemical terrorism.
Analytical procedure validation is often a complex, iterative process that requires scientific judgment. The goal of these guidelines is to provide the laboratory with information on steps that are commonly used when validating an analytical procedure. This guidance is meant to complement the experience and professional judgment of the laboratory personnel.
Chemicals associated with terrorism may present acute hazards not normally encountered during routine operations by laboratory personnel. Appropriate caution and safety should be exercised when dealing with these types of hazardous materials.
Reference
Scientific Working Group on Forensic Analysis of Chemical Terrorism,
Quality assurance guidelines for laboratories performing forensic
analysis of chemical terrorism, Forensic Science Communications
[Online] (2004). Available: http://www.fbi.gov/hq/lab/fsc/backissu/april2004/standards/
2004_02_standards01.htm.
Suggested Reading
EURACHEM. Fitness for Purpose of Analytical Methods: A Laboratory
Guide to Method Validation and Related Topics, EURACHEM, 1998.
Available: http://www.eurachem.ul.pt/guides/valid.pdf.
Wernimont, G. Use of Statistics to Develop and Evaluate Analytical
Methods, AOAC International, 1985.
Peters, F. T. and Maurer, H. H. Bioanalytical method validation
and its implications for forensic and clinical toxicology: A review,
Accredited Quality Assurance (2002) 7:441-449.
Thompson, M., Ellison, S. L. R., and Wood, R. Harmonized guidelines
for single-laboratory validation of methods of analysis, Pure
and Applied Chemistry (2002) 74(5):835-855. Available: http://www.iupac.org/publications/pac/2002/pdf/7405x0835.pdf.
1. Scope
These guidelines describe validation practices that a laboratory
should follow to ensure that its analytical procedures produce data
that are fit for the intended purpose. It is the responsibility
of each laboratory to select the appropriate analytical procedure
that will meet the intended data quality objectives. A laboratory,
in the context of these guidelines, is defined as a facility in
which analysis associated with chemical terrorism is performed.
It is assumed that the laboratory is operating under the quality
assurance system described in the Quality Assurance Guidelines
for Laboratories Performing Forensic Analysis of Chemical Terrorism
(http://www.fbi.gov/hq/lab/fsc/backissu/april2004/standards/
2004_02_standards01.htm).
This guidance may be useful in the following situations:
- When a standardized analytical procedure is applied in the laboratory for the first time.
- When a standardized analytical procedure is modified.
- When a published but nonstandardized analytical procedure is used in the laboratory.
- When an analytical procedure is developed in-house for subsequent routine use.
- When an analytical procedure is developed in-house as a solution to a unique analytical problem.
2. Definitions
As used in these guidelines, the following terms shall have the meanings specified. (See also Appendix A and Appendix B.)
Accuracy is the extent to which an analytical result
approaches the true value. Accuracy includes a combination of random
error (precision) and systematic error (bias).
Analyte is the chemical entity whose presence and/or
concentration is being established.
Analytical procedure is an orderly step-by-step instruction
designed to ensure operational uniformity and to minimize uncertainty.
Laboratory is a facility where chemical forensic testing
is performed.
Limit of detection is the lowest concentration or smallest
amount of analyte that can be statistically differentiated from
the analyte-free sample matrix.
Limit of quantitation is the lowest concentration or
smallest amount of analyte that can be measured at a specified accuracy.
Linearity is the extent to which an analytical procedure
produces a signal directly proportional to the concentration or
mass of the analyte of interest.
Matrix is the medium or material that may contain the
target analyte(s).
Reference material is a material for which component
identities, types, or values are certified by technically valid
procedures and is accompanied by or traceable to a certificate or
other documentation that is issued by a certifying body.
Sample is the subject of an analytical procedure. The
sample consists of potential target analyte(s) that may exist in
a matrix.
Selectivity is the extent to which an analytical procedure
is free from interferences arising from nonanalytes, including matrix
components.
Sensitivity is the level of instrumental signal obtained
per unit amount of analyte. Sensitivity is not interchangeable with
limit of detection.
Spike is a laboratory-prepared sample to which a known
amount of the analyte of interest is added, either to a portion
of the original sample or similar matrix.
Standard is a substance of known identity and purity
and/or concentration.
Standardized analytical procedure is one that has been
documented, validated, and approved by a recognized technical organization
(e.g., AOAC International, ASTM International, U.S. Environmental
Protection Agency, U.S. Pharmacopeial Convention Incorporated).
Validation is the process by which an analytical procedure
is evaluated to determine its fitness for its intended use.
Working range is the concentration range over which the
analytical procedure has been validated.
3. Validation Process
3.1. The laboratory should establish and maintain
a documented validation process for the forensic analytical procedures
performed by the laboratory.
3.2. The validation process should consist of
the following:
- Define the scope of the analytical procedure.
- Identify the analytical procedure performance characteristic(s) that must be validated.
- Select experiments to determine the required performance characteristic(s).
- Conduct experiments to determine required performance characteristics.
- Document, review, and maintain analytical procedure validation results.
4. Define the Scope of the Analytical Procedure
4.1. The laboratory should develop a clear and
unambiguous statement defining the scope, which includes the matrix,
target analyte(s), analytical technique, and intended purpose of
the analytical procedure.
4.2. The purpose of an analytical procedure may
include the following:
- Identification of unknown component(s).
- Establishment of the presence or absence of specified analyte(s) and/or classes.
- Quantitation of specified analyte(s).
- Determination of a physical and/or chemical property (e.g., mass, color, viscosity, flash point, particle morphology, crystalline structure).
5. Identify the Analytical Procedure Performance Characteristic(s) That Should be Validated
The performance characteristics that should be validated will vary depending on the purpose of the procedure and may include accuracy (precision and bias), limit of detection, limit of quantitation, linearity, working range, and selectivity. Selecting performance characteristics requires professional judgment. The following performance characteristics may be relevant to the intended purposes stated above:
5.1. Identification of unknown component(s).
Due to the nature of unknown component identification, the validation may be conducted either at the time of analysis or immediately following. The performance characteristic that should be validated in this case is selectivity.
5.2. Establishment of presence and/or absence
of specified analyte(s) or classes including the following:
- Selectivity.
- Limit of detection.
5.3. Quantification of specified analyte(s) including
the following:
- Selectivity.
- Linearity (or other calibration model).
- Working range.
- Limit of detection.
- Limit of quantitation.
- Accuracy.
5.4. Measurement of a physical property including
the following:
- Linearity (or other calibration model).
- Working range.
- Accuracy.
6. Select Experiments to Determine the Required Performance Characteristic(s)
The laboratory should evaluate the existing documentation relating to the analytical procedure to determine if it fulfills the performance characteristics relevant for the intended purpose described in Section 5. Existing validation data will vary based on the analytical procedure category, as described below, and may determine the need for additional experiments. In evaluating existing data, the laboratory should consider issues related to false positives and false negatives as described in Appendix B.
- A standardized analytical procedure has been documented, validated, and endorsed by a recognized technical organization (e.g., AOAC International, ASTM International, U.S. Environmental Protection Agency, U.S. Pharmacopeial Convention Incorporated). In this case, the laboratory should demonstrate its capability to obtain similar performance characteristics.
- A modified standardized analytical procedure is one that has been modified outside the specifications of the standardized procedure. In this case, the laboratory should verify that the modifications do not alter the performance characteristics such that the data are no longer fit for the intended purpose.
- A nonstandardized analytical procedure has been developed externally but has not been previously endorsed by a recognized technical organization (e.g., an analytical procedure published in a technical journal). In this case, the laboratory should determine the performance characteristics applicable to the intended purpose and conduct any necessary validation experiments.
- An in-house analytical procedure is developed in the laboratory for subsequent routine use or as a solution to a unique analytical problem. In this case, the laboratory should determine the performance characteristics applicable to the intended purpose and conduct any necessary validation experiments.
7. Conduct Experiments to Determine the Required Performance Characteristic(s)
The experiments required for validation depend on the analytical procedure scope, its intended use, the analytical procedure category, and the quality of the existing validation efforts. Validation requirements are often project-specific and may be adjusted depending on specific needs (e.g., safety considerations, limited quantity of available sample, differences in sample matrices, reference materials).
Laboratories should refer to Appendix A when conducting validation
experiments for the various performance characteristic(s) and to
Appendix B for considerations relating to false positives and false
negatives. In addition, laboratories should refer to the Quality
Assurance Guidelines for Laboratories Performing Forensic Analysis
of Chemical Terrorism (http://www.fbi.gov/hq/lab/fsc/backissu/april2004/standards/
2004_02_standards01.htm) prior to conducting validation experiments.
8. Document, Review, and Maintain Analytical Procedure Validation Results
All documentation related to validation studies should be logically
and consistently maintained in accordance with the Quality Assurance
Guidelines for Laboratories Performing Forensic Analysis of Chemical
Terrorism (http://www.fbi.gov/hq/lab/fsc/backissu/april2004/standards/
2004_02_standards01.htm). This documentation includes references,
analytical procedures, and experimental data. All known limitations
of the procedure should be included in the documentation. Appendix
C may be used as a template for summarizing analytical procedure
validation data.
The validation documentation and data should be reviewed by a technically qualified person to ensure that the validation approach produces performance characteristics and that results are fit for the intended use of the analytical procedure. Evidence of the technical review should be maintained for future reference.
Appendix A: Analytical Performance Characteristics
Accuracy
Accuracy is the extent to which an analytical result
approaches the true value. The accuracy of an analytical measurement
is related to the random error (precision) and the systematic error
(bias).
Precision determination is made by repeating a measurement
over a specified time frame appropriate for the intended analytical
procedure use. The measure of precision will depend on the range
of conditions (e.g., analyte concentrations, matrices, instrumental
parameters) over which the analytical procedure is applied. The
measurements should be made using the entire analytical procedure
including all preparation and analysis steps.
Because the true value need not be known, a wide variety of materials may be used to assess precision, including reference materials, in-house quality control materials, and the actual samples of interest. Precision is typically expressed as the percent relative standard deviation (% RSD):
% RSD = standard deviation of measurements * 100% mean of measurements
Bias in an analytical procedure is determined by comparing
the measurement result with the true value. Bias can be estimated
by measuring materials of known composition, such as reference materials.
Matrix matched reference materials are considered the preferred
materials for estimating bias. When a suitable reference material
is not available, bias may be estimated by the analysis of spiked
samples. The behavior of the added analyte may differ from that
of the native analyte, but spiking attempts to achieve the goal
of matrix matching. Spike recovery is calculated as follows:
% spike recovery = (measured concentration spiked sample - measured concentration unspiked sample) * 100% concentration of spike contribution
Bias can also be estimated by comparing results obtained for the same samples using another analytical procedure with a known bias (i.e., a reference method).
Limit of Detection
The limit of detection is the lowest concentration or
smallest amount of analyte that can be statistically differentiated
from the analyte-free sample matrix. The limit of detection depends
not only on the sensitivity but also on the instrumental noise and/or
blank variability.
The instrumental limit of detection is a measure of instrument
performance and is not sample matrix specific. It is a measure of
either the instrumental signal-to-noise level or the variability
of a standard blank. Of greater importance is the analytical procedure
limit of detection. It incorporates not only the instrumental sensitivity
and noise but also the variability induced by components of the
sample matrix.
There are many approaches used to calculate the limit of detection; therefore, the laboratory should define its approach for determining a limit of detection.
The limit of detection may or may not be relevant in a validation study, depending on the concentration range and the intended purpose of the analytical procedure. For example, for analytical procedures when the analyte measured is always in the calibration range of the assay and well above the true limit of detection, it may be sufficient to indicate that the detection limit is "less than" the value of the lowest nonzero calibration standard.
Limit of Quantitation
The limit of quantitation is the lowest concentration
or smallest amount of analyte that can be measured at a specified
accuracy. There are many approaches used to calculate the limit
of quantitation; therefore, the laboratory should define its approach
for determining a limit of quantitation.
The limit of quantitation may or may not be relevant in a validation study, depending on the concentration range and the intended purpose of the analytical procedure.
Linearity (or other calibration model)
Linearity is the extent to which an analytical procedure
produces a signal directly proportional to the concentration or
mass of the analyte of interest. Linearity (or other calibration
model) is assessed by constructing a calibration curve (response
versus analyte concentration) from known standards. Linear calibration
models are frequently used, although various analytical procedures
may yield acceptable nonlinear calibrations. The analyst can evaluate
the linearity of the calibration curve by visual inspection or by
using appropriate statistical methodology. The magnitude of the
linear correlation coefficient, whereas sometimes used as a linearity
measure, can be misleading. Depending on the number and spacing
of calibration points, a visually nonlinear plot can lead to a correlation
coefficient very close to one.
Working Range
Working range is the concentration or measurement range
over which the analytical procedure has been validated. The concentration
of the analyte of interest will have an effect on most analytical
performance characteristics. Therefore, the analytical procedure
should be validated for a working range consistent with its intended
purpose.
The low end of the working range depends on the purpose of the analytical procedure. For example, if the purpose of the analysis at low concentrations is to simply indicate presence or absence of analyte, then the limit of detection may mark the low end of the working range. When accurate concentration values are needed, then the limit of quantitation may become the low end of the analytical procedure's working range.
If the analyte response exceeds the working range, the working range should be reestablished. A more common approach is to dilute the sample into the working range.
Selectivity
Selectivity is the extent to which an analytical procedure
is free from interferences arising from nonanalytes, including matrix
components.
Although it is possible to establish that an interference exists, it is more difficult to state that no interferences exist. Matrix interferences are usually sample specific and should be addressed on a matrix-by-matrix basis. Many instrumental analytical procedures have specific approaches that can be used to detect (and possibly circumvent) lack of selectivity. Some examples are the use of an alternate column in a chromatography method or the use of alternate emission lines in emission spectroscopy.
Another approach to assessing selectivity is using an alternate analytical procedure for reanalysis of the samples. This assessment is the most convincing when an independent or orthogonal analytical technique is employed. Orthogonal techniques respond to distinct characteristics of a particular analyte. Infrared spectroscopy and mass spectrometry are orthogonal to each other, whereas infrared and Raman spectroscopies are not orthogonal to each other.
Appendix B: False Positives and False Negatives
Several quality control considerations are especially important when applying an existing method to a matrix or analyte for which the analytical approach has not been validated. They should also be incorporated in a regular quality control program.
The negative control (matrix blank) is a sample that
closely matches the samples being analyzed with regard to matrix
components and is collected to establish the background level (presence
and/or absence), of the analyte(s) of interest. It incorporates
all the reagents employed in treating the samples of interest and
is subjected to all sample-processing operations. Its role is to
verify that the normal sample matrix does not interfere with or
affect the analytical signal. The negative control may be difficult
to obtain because many matrices cannot be closely matched or guaranteed
to be free from analytes.
The method blank is a quality control sample that incorporates
all the reagents employed in treating the samples of interest and
is subjected to all sample processing operations. A method blank
serves to verify that an identified component does not originate
in the reagents, by cross contamination, or from the analytical
process.
A positive control is a quality control sample containing
the target analyte(s) and is subjected to all sample-processing
operations. The positive control may be a spiked matrix similar
to the one being analyzed, or it may be a reference material. The
positive control serves to demonstrate that the analyte of interest
would have been detected, if present, at or above a particular concentration.
Carryover is the addition of analyte from
a sample or standard to subsequent samples in a series of analyses.
Carryover should be evaluated as a part of the validation effort.
Carryover can often occur during instrumental analysis. The incorporation
of appropriate blanks at key points in the analytical workflow (i.e.,
after analyzing the standards) could demonstrate the absence of
analyte carryover.
Appendix C
Table 1: Summary Guidelines for Validating an Analytical
Procedure
Procedure Purpose |
Describe the purpose, which may include identifying unknown
component(s); establishing presence and/or absence of specified
analyte(s) and/or classes; quantifying specified analyte(s);
and/or determining a physical property (e.g., mass, color, viscosity,
flash point, particle morphology, crystalline structure). |
Procedure Scope |
Develop a clear and unambiguous statement defining the scope,
which includes the matrix, target analyte(s), analytical technique,
and intended purpose of the analytical procedure. |
Selectivity |
Describe the requirements of the analytical procedure with
regard to selectivity. Describe experiments that should be performed
to achieve the required selectivity. Consider the availability
of reference materials, standards, matrix blanks. |
Bias |
Describe approaches used to assess bias, such as reference
materials, alternate analytical procedures, or spike recovery. |
Precision |
Describe precision estimates, and state the range of conditions
(e.g., analyte concentrations, matrices, instrumental parameters)
over which the analytical procedure was validated. |
Limit of Detection |
Define the approach for estimating the limit of detection
or provide a statement when this performance characteristic
is not relevant to a validation. |
Limit of Quantitation |
Define the approach for determining the limit of quantitation.
|
Working Range |
Describe the working range over which the analytical procedure
was validated. |
Calibration Model |
Describe the type of calibration model acceptable to validate
the analytical procedure. Assess linearity if applicable. |
Critical Step |
Provide any information about any steps that may be critical
to the successful application of the analytical procedure. |
Limitations |
Identify all known limitations of the analytical procedure
and its use. References list any previously documented analytical
procedures referred to during the validation process. Include
in validation documentation copies of analytical procedures
that are not readily available. |
Contributors
The following SGWFACT members contributed to this document:
Dean D. Fetterolf
Federal Bureau of Investigation
Laboratory
Quantico, Virginia
Armando Alcaraz
Lawrence Livermore National Laboratory
Forensic Science Center
Livermore, California
Shauna Darby
Battelle
Arlington, Virginia
Mary Drummond
Research, Development and Engineering Command
Forensic Analytical Center
Aberdeen Proving Ground, Maryland
Martin Harper
Centers for Disease Control and Prevention
National Institute for Occupational Safety and Health
Health Effects Laboratory Division
Morgantown, West Virginia
Charles Hooper
U.S. Environmental Protection Agency
Athens, Georgia
Eugene Kennedy
Centers for Disease Control and Prevention
National Institute for Occupational Safety and Health
Chemical Exposure and Monitoring Branch
Cincinnati, Ohio
Mathew Magnuson
U.S. Environmental Protection Agency
National Risk Management Research Laboratory
Cincinnati, Ohio
Madeline A. Montgomery
Federal Bureau of Investigation
Laboratory
Quantico, Virginia
Eric Nottingham
U.S. Environmental Protection Agency
National Enforcement Investigations Center
Denver, Colorado
James Peterson
Batelle
Aberdeen Proving Ground, Maryland
Charles F. Quenzer
Federal Bureau of Investigation
Laboratory
Quantico, Virginia
R. Duane Satzger
U.S. Food and Drug Administration
Forensic Chemistry Center
Cincinnati, Ohio
James Seidel
U.S. Environmental Protection Agency
Office of Criminal Enforcement
Forensics and Training
Denver, Colorado
Karen Wolnik
U.S. Food and Drug Administration
Forensic Chemistry Center
Cincinnati, Ohio
|