Best Practices for Soil Sampling Design

Module 1
Best Practices for Efficient Soil Sampling Designs
The Point of this Workshop

is not to teach you to be statisticians.
Rather, teach you basic concepts so able to ask for what you need & detect issues with statisticallybased soil sampling program designs.
is not to say statistics are wrong, never appropriate, or cannot be used.

Rather, to show that statistics cannot be used blindly or as a black box for data collection design/interpretation And identify pitfalls common to sampling programs, the problems they cause, and how to avoid them.
10June2008 Triad Investigations: New Approaches and Innovative Strategies 2
Workshop Agenda
Welcome Where does decision uncertainty come from? You cant find the answer if you dont know the question! Beware of statistics bearing assumptions The cure for sampling dilemmas: use emerging best practices
Instructors
Deana Crumbling, crumbling.deana@epa.gov Office of Superfund Remediation &Technology Innovation U.S. Environmental Protection Agency Washington, D.C. (703) 603-0643 Robert Johnson, rlj@anl.gov Environmental Science Division Argonne National Laboratory Argonne, Illinois (630) 252-7004
Software Resources and Disclaimer

Several software packages are referenced. References do not constitute an endorsement. For more information:
Visual Sampling Plan (http://dqo.pnl.gov/) ProUCL (http://www.epa.gov/esd/tsc/software.htm) BAASS (http://web.ead.anl.gov/baass/register2/)
10June2008
Triad Investigations: New Approaches and Innovative Strategies
Take Away Points

Statistics often are used to address cleanup decision-making uncertainty Care must be taken when applying traditional statistical approaches to soil sampling programs Be extremely cautious with black-box software Systematic planning, dynamic work plans, and real-time measurement techniques (the Triad) can greatly improve sampling program designs
10June2008
Module 2
Where Does Uncertainty Come from When Making Restoration Decisions?
As we know, there are known knowns. There are things we know we know. We also know there are known unknowns. That is to say we know there are some things we do not know. But there are also unknown unknowns, the ones we don't know we don't know. Donald Rumsfeld, Feb. 12, 2002, Department of Defense news briefing
Decision Uncertainty Comes from a Variety of Sources

Political, economic, organizational & social uncertainty (outside scope of discussion) [know we know] Model uncertainty (also outside discussion scope, although approaches to be discussed may provide mechanisms for addressing this) [know we dont know] Data uncertainty. Data uncertainty refers to the uncertainty introduced into decision-making by uncertainty associated with data sets used to support decisions [Dont know we dont know] Data Uncertainty: primary focus of the trainingsampling programs play important role.
Decision Quality Only as Good as the Weakest Link in the Data Quality Chain
Sampling
Sampling Design SubSampling
Analysis
Extract Cleanup Method Result Reporting
Interpretive
Sample Support
Sample Preservation
Sample Prep Method
Determinative Method
Relationship between Measurement Parameter & Decision Parameter
Each link represents a variable contributing toward the quality of the analytical result. All links in the data quality chain must be intact for data to be of decision-making quality!
Taking a Sample for Analysis

Population Soil Core Sample Lab Subsamples (Duplicates) Analytical Sample Unit
Field 10June2008 Subsample
Analytical Sample Prep
GC
23.4567 ppm
11
Historically the Focus Has Been Analytical Quality

Emphasis on fixed laboratory analyses following well-defined protocols Analytical costs driven to a large degree by QC/QC requirements Result:
analytical error typically on order of +/-30% for replicate analyses traditional laboratory data treated as definitivebut definitive (definite) about what?
The Biggest Cause of Misleading Data
10June2008
13
Within-Sample Variability: Interaction between Contaminant & Matrix Materials

Firing Range Soil Grain Size Pb Concentration in (Std Sieve Mesh Size) fraction by AA (mg/kg)
Greater than 3/8 (0.375)
Adapted from ITRC (2003)
10 50 108 165 836 1,970 927 (wt-averaged)

14
Between 3/8 and 4-mesh Between 4- and 10-mesh Between 10- and 50-mesh Between 50- and 200-mesh Less than 200-mesh Bulk Total
10June2008
The decision determines representativeness Triad Investigations: New Approaches and Innovative Strategies
Concentrated Particles within Less Concentrated Matrix = Nugget Effect

Regulatory & field practices have long assumed that sample size/volume has no effect on analytical results The Nugget Effect
Soil Subsample
Sample Prep
2g
5g
Now we know the assumption is inaccurate because of microscale (within-sample) heterogeneity.

Sample volume affects the analytical result! 10June2008 Triad Investigations: New Approaches and Innovative Strategies
15
Micro-scale Heterogeneity Causes Highly Variable Data Results

True sample mean known to be 192 ppm
Replicate Range of results How many subsamples subsample weight to average so result is for 20 replicate taken from a large, close (+25%) to true subsamples ground & sieved soil sample mean? (ppm) sample [144 - 240 ppm]
1g 10 g 100 g
101 - 800 136 - 343 170 - 230
39 5 1
Adapted from DOE (1978) americium-241 study

Short-Scale Variability Can Also be Significant

286 ppm 7 416 ppm 2 41,400 ppm 1,220 ppm 6 1 2 ft
Figure adapted from Jenkins (CRREL), 1996
3 136 ppm
5 27,700 ppm
10June2008
4 42,800 ppm
17
Heterogeneity Overwhelms Variability from Different Analytical Techniques

331 On-site 286 Lab
500 On-site 416 Lab
1,280 On-site 1,220 Lab
39,800 On-site 41,400 Lab
95% of data variability due to sample location over a 4 ft diameter
2 ft
164 On-site 136 Lab
24,400 On-site 27,700 Lab

10June2008
27,800 On-site 42,800 Lab

18
Uncertainty Math Magnifies Weakest Links Effects in Data Quality Chain Uncertainties add according to (a2 + b2 = c2)
Analytical Uncertainty Total Uncertainty
Sampling Uncertainty Example: AU = 10 ppm, SU = 80 ppm: TU = 81 ppm

AU = 5 ppm, SU = 80 ppm: TU = 80 ppm AU = 10 ppm, SU = 40 ppm: TU = 41 ppm AU = 20 ppm, SU = 40 ppm: TU = 45 ppm
How Do We Reduce Data Uncertainty?

For analytical errors:
Modify current or switch to another analytical technique Improve QC on existing techniques
For sample prep and handling errors:

Improve sample preparation
For sampling errors:

Collect samples from more locations
We cant control the effects of uncertainty on our decisions if we dont know where it is coming from. Historically sampling programs have focused resources on the wrong sources of data uncertainty.
10June2008
21
Module 3
You cant find the answer if you dont know the question!
To be [averaged] or not to be [averaged], that is the question

From William Shakespeare's Hamlet, Prince of Denmark, Act III, scene I, as translated by course instructors
10June2008
23
Sometimes the Simplest Questions are the Most Complex

Does this site pose an unacceptable risk? Do groundwater concentrations exceed drinking water standards? Do soil concentrations exceed cleanup requirements?
10June2008
24
Sample Support, Representativeness and Decision Unit Support are Intertwined

What sample support is most representative of the decision? #1 #2 #3
Surface layer of interest
Layer impacted by deposition
The decision driving sample collection: Can it be shown that atmospheric 10June2008 Triad Investigations: New Approaches and Innovative Strategies deposition caused contamination?
25
Advances in Sampling & Measurement Technologies Highlight Representativeness Issues

MIP = membraneinterface probe (w/ ECD detector)
GW data results HIGHLY dependent on sample support
10June2008
Triad Investigations: New adapted Approachesfrom and Innovative Strategies Graphic Columbia Technologies
26
The Same Holds True for Soils

0.080 0.080 0.070 0.070 0.060 0.060 Frequency Frequency 0.050 0.050 0.040 0.040 0.030 0.030 0.020 0.020 0.010 0.010 0.000 0.000 0 0 100 100 200 200 300 300
Homogenized Discrete Samples` XRF Readings Action Level Multi-Increment Samples
400
500 500
600 600
Concentration (ppm) Concentration (ppm)
10June2008
27
The Decision Unit is Often Not WellDefined

Lead should not exceed 400 ppm in soils or TCE should not exceed 5 ppb in groundwater
Decisions are often ambiguous because cleanup criteria do not provide enough information to define the decision units.
10June2008
28
Complete Cleanup Criteria Definitions

Cannot achieve data representativeness w/o a complete definition of cleanup criteria Incomplete criteria leads to confusionexample:
an in situ XRF Pb reading from a yard is 560 ppm, while a homogenized sample from same is 200 ppm, while the average for the yard is 50 ppm.
Different sample supports different concentration estimates that are all correct but lead to different conclusions
Must DEFINE population of interest to interpret data!!

For Soils, Three Cleanup Requirement Definitions are Most Common:

Never-to-Exceed Criteria: Lead concentrations cannot be > 400 ppm Hot-Spot Criteria: Lead concentrations cannot be > 400 ppm averaged over 100 m2 Averaged Criteria: The average concentration of lead over an exposure unit cannot be > 400 ppm
10June2008
30
Technically Defensible Sampling Programs Require Complete Criteria

Technically defensible: making decisions with known level of confidence Impossible to design a defensible program for never-to-exceed criteria Hot spot criteria typically require the most intensive sampling to be technically defensible Exposure unit averages are the easiest
10June2008
31
All Solid Samples are Composites!!

A data result = average for mass of soil digested/ extracted during analytical prep (the analytical sample support) Questions:
Is analytical sample support representative of original sample support as recd by lab? Is sample recd by lab representative of decision support as defined by project team? Is teams SAP representative of regulatory criterion?
Sample support critical, yet currently determined by convenience or whim of samplers & analysts. If so, data quality being left to chance!!
Defensible Statistical Sampling Program Design and Data Analysis Requires:

Clearly defined decision units and decisionmaking (e.g., action level) criteria Sample supports that are representative of the decision unit of interest Analytical method implementation consistent with required sample support
10June2008
33
Module 4
Beware of Sampling Programs Built on Erroneous Statistical Assumptions
Statistics: The only science that enables

different experts using the same figures to draw different conclusions. Evan Esar: Esar's Comic Dictionary American Humorist (1899 - 1995)
10June2008
35
Wed prefer to ignore statistics when they tell something we dont want to hear
10June2008
36
Statistical Packages Can Give an Aura of Defensibility
But, if underlying assumptions are wrong, the conclusions are wrong!

Representativeness Assumed
SAP says: Representative samples will be collected. But provides no explanation of how or what the samples are supposed to represent. Non-representative data decision errors
Sample support mismatched to cleanup criteria (single grab vs. area average) Samples with different supports mixed together in databases & statistical analysis Use spatially clustered locations or biased samples when calculating average concentrations Mix different populations together
0.080 0.070 0.060 0.050 0.040 0.030
Action Level (AL)
Remember these Examples?

0.080 0.070
Frequency
0.020 0.010
0.060
0.000 Concentration (ppm)
Frequency
100
200
300
400
0.050 500 0.040 0.030 0.020 0.010 0.000 0
600
Sample support vs action level
0.080
100
200
300
400
500
0.070 600 0.060
Concentration (ppm)
Frequency
0.050 0.040 0.030 0.020 0.010 0.000
Lognormal seen with small sample supports & when data from different supports are mixed together 10June2008 Triad Investigations: New Approaches and Innovative Strategies
0 100 200 300
400
500
600
Concentration (ppm)
39
Bias Your Answer by Biasing Your Sampling Approach

Exposure Unit #1: Biased sampling & spatial clustering when goal is to calc the area average
EU#1
Dump
Exposure Unit #2: Mix 2 populations (cleaner area & dump) into the same sampling design & data set
10June2008
EU#2
40
Pretend All Data Sets Are Normal

Normal distributions make statistics easy Can ignore complexities of spatial & non-random relationships Many common statistical tests (typical UCL calculation, Student t test, etc.) assume normality
10June2008
41
But Contaminated Sites Are Rarely Normal

Distributions are usually heavily skewed to the right Often can be bimodal (i.e., two-humped) Usually reflect overlaying or mixed populations, e.g,: -- Contaminated over background, or -- Air deposition over discrete surface releases
Assuming Normality Can Under-estimate the 95% UCL on the Mean

400 ppm Pb requirement for exposure unit 4 lab lead results: 20, 24, 86, and 189 ppm Average of the 4 results: 80 ppm Too few samples to know whether normal or not.
If assume data distribution is the 95% UCL is Normal distribution Gamma distribution Lognormal distribution Non-parametric distribution
10June2008
Options for how ProUCL can calculate 95%UCL

172 ppm 434 ppm 246 33,835 ppm 144 472 ppm
43
Assuming Normality Can Under-Estimate Sample Number Requirements

VSP example: How many samples recommended to demonstrate that mean concentration is less than action level?
Assuming normal distribution: 10 Assuming non-parametric distribution: 23
10June2008
44
Consequences of Under-estimating Sample Numbers

Only know statistics were wrong when datas back from lab!
95%UCL is basis for many EPA decisions about risk & compliance. When calculate 95%UCL from the data, find that decisions cannot be made at desired (95%) confidence
95% UCL > action level, even if mean < action level
325
95%LCL
425
mean
500 525
AL 95%UCL
Need more samples if want to make confident decision about risk or compliance: redo project
Assume We Know Altho We Dont

What info is required to use statistics properly? VSP is a software pkg that calculates number of samples based on user inputs. For soil sampling projects, user must supply:
The conc variability that is present (recall that data variability is a function of sample volume) Underlying contaminant distribution (ditto) How statistically confident do you want to be?
10June2008
46
0.080 0.070 0.060 0.050 0.040 0.030
Action Level (AL)
Variability & population distribution depends on sample support

0.080
Frequency
0.020
0.070
0.010
0.060
0.000 Concentration (ppm)
Frequency
100
200
300
400
0.050 500 0.040 0.030 0.020 0.010 0.000 0
600
0.080
100
200
300
400
500
0.070 600 0.060
Concentration (ppm)
Frequency
0.050 0.040 0.030 0.020 0.010 0.000 0 100 200 300 400 500 600
10June2008
Concentration (ppm)
47
More that We Need to Know

What more is required to use statistics properly? User must also supply:
Width of gray region (requires prediction of the true mean for the area under investigation)
500
95%LCL
650
mean
700
AL
725
95%UCL
Recognize that these input values will be different for different contaminants on the same site
Different field concs, different ALs VSP may predict 10 samples for Zn, 500 for PAHs, 1000 for Hg
Characterize or verify cleanup? Statistical confidence desired How close to each other are the true mean & AL? How much variability is present in the soil concentrations
How choose the input values?

Choose wisely: It Makes a Big Difference in Sample Numbers!

Actual mean closer to AL --->
GR width: 350 ppm StDev 50 ppm 200 ppm 300 ppm
200 ppm
100 ppm
Increasing variability ----->
2 5 8
3 10 21
4 36 79
50
10June2008
Fact is, if we knew everything we needed to know in order to design a statistical sampling program correctly, we wouldnt need to do the sampling!!
10June2008
51
Dilemma Resolutions
How can good approximation inputs be chosen?
Data & experience from similar sites Historical data from your site Pilot study (efficient if part of dynamic field work)
How does a non-statistician ensure the quality of VSP applications?

When VSP used to justify sample numbers, require that submissions include explanations for how each input value was chosen Verify that the explanations are reasonable
There are three kinds of lies: lies, damned lies, and statistics
-- Attributed to Benjamin Disraeli (as popularized by Mark Twain)
Verifying Assumptions is a Cure for Statistical Misrepresentation

Module 5
The Cure for the Sampling Blues
uncertainty mgt
All truths are easy to understand once

they are discovered; the point is to discover them. Galileo
10June2008
55
The Triad Framework
Systematic Project Planning
Uncertainty mgt
Dynamic Work Strategies
Real-time Measurement Technologies
10June2008
56
Triad Data Collection Designs & Data Analysis Built On:

Planning systematically (CSM is central) Improving representativeness (& stats) Addressing the unknown (with dynamic work strategies)
10June2008
57
Systematic Planning Addresses:

Planning Systematically
Defining sample representativeness Accurately delineating contaminant populations
Identifying populations requires using a CSM Helps design a sampling plan to cultivate data that conforms to a well-behaved statistical distribution
Discovering what we dont know

Systematic Project Planning
Uncertainty mgt
58
10June2008
Systematic Planning & Data Collection Design

Planning must define decisions, decision units & sample support requirements Planning must identify sources of decision uncertainty & strategies for uncertainty management Planning must clearly define cleanup standards Conceptual Site Models (CSMs) play a foundational role All the above help define sampling populations
Triad Investigations: New Approaches and Innovative Strategies 59
10June2008
2 Fundamental Concepts for Sampling Design & Statistics: (1) Decision Unit
Decision Unit: Area, volume, or set of objects (e.g. -acre area, bin of soil, set of drums) All items treated as a single unit for decision-making Statistical goal: discover true mean for that single unit
Amount of variability w/in the unit creates uncertainty in estimating the true mean Therefore, statistics used to express amount of uncertainty around the estimate of the mean
Examples: exposure unit, survey unit, remediation unit

10June2008
Valley of the Drums: These need to be characterized, transported, and disposed properly. What is the decision unit? How do you sample it?
Batch #1
Batch #2
Batch #3
Batch #4
40 drums were cleaned in batches of 20. You need to ensure the cleaning process worked. What is the decision unit and how would you sample it?
2nd Fundamental Concept: Population

Population: Set of objects or material volumes sharing a common characteristic; can be synonymous w/ decision unit.
2 populations (clean As one & dirty Pb one) overlap w/in a single decision unit (such as residential yard) A population is so large that more than 1 decision unit is needed to cover it. Example: Suspected clean population of 50 acres; but decision unit (exposure unit) is 1 acre.
Examples where they are not synonymous:
10June2008
Dont Get Confused by Similar Terms!

Statistical Population Distribution: Number of times (frequency) that particular values occur in a data set drawn from the population (also called frequency distribution)
This data set is called a sample by statisticians & it contains >1 physical sample Example: How many times does the value 10 ppm occur? Spatial Population Distribution: A spatial pattern created by the locations of values (such as low, medium & high concentrations) across an area or within a volume Conversion of a spatial distribution to a statistical distribution results in loss of spatial & physical relationship information
10June2008
Relationship between spatial population distribution & statistical population distribution
Different spatial distributions

Histogram
10 F re q u e n c y 5 0 0 5 10 15 20 25 More
65
but same statistical distribution

10June2008
Bin
CSMs Should Articulate Decision Uncertainty Planning Systematically

CSM captures current understanding about site conditions
Identifies additional information needed for confident decision-making Data collection needs & design flow from the CSM
A well-articulated CSM serves as the point of stakeholder consensus. CSMs are livingas new data become available, incorporate into CSM.
CSM is mature when desired decision confidence is achieved
10June2008
Statistics & the CSM

Spatial considerations are irrelevant to classical statistics
Inputs for calculating how many samples does not include area to be sampled If VSP inputs are the same, a 100-sq mi area will get same # samples as 1-sq ft area YOU must use the CSM to create decision units & select proper statistics for each decision unit
Statistics cannot be used properly w/o a CSM that defines the statistical populations!!
10June2008
Improving Representativeness
Improving Representativeness & Statistical Performance

Sample support
Volume/dimensions of sample, for XRF in situ analysis, the sample support is the field of view Match it to decision needs & populations
Control within-sample heterogeneity

Appropriate sample preparation important (see EPA EPA/600/R-03/027 for additional detail) Uncertainty effects quantified by appropriate sub-sample replicate analyses
Control short-scale between-sample heterogeneity

Multi-increment sampling (physical averaging) Multiple readings (mathematical averaging)
10June2008
Guidances on Multi-Increment Sampling/Compositing May Differ Somewhat

Verification of PCB Spill Cleanup by Sampling and Analysis (EPA-560/5-85-026, August, 1985)
up to 10 adjacent samples allowed
Cleanup Standards for Groundwater and Soil, Interim Final Guidance (State of Maryland, 2001)
no more than 3 adjacent samples allowed
SW-846 Method 8330b (EPA Rev 2, October, 2006)

30 adjacent samples recommended
Draft Guidance on Multi-Increment Soil Sampling (State of Alaska, 2007)

30 50 samples recommended for compositing
10June2008
All Samples Are Composites at Some Scale

A soil subsample contains many particles that are extracted or digested & analyzed together A bulk heap of soil taken from a jar is a composite of many individuals particles Jar contents is a composite of many individual particles The source of the jar contents is a composite of many, many particles Compositing (from different locations at the between-sample level) is the same principle as stirring a jar to bring spatially separated particles into the same analytical sample at the within-sample scale
10June2008
Multi-Increment Sampling vs. Compositing

Multi-increment sampling: a strategy to costeffectively control the effects of heterogeneity multi-increment averaging Compositing: a strategy to reduce overall analytical costs when conditions are favorable while looking for contamination composite searching Same soil aggregation process, but done for very different reasons
10June2008
Multi-Increment Sampling vs. Compositing

Assumption: the cleanup criteria is averaged over decision unit
Multi-Increment Sampling
Decision Unit 1
Decision Unit 3 Decision Unit 4
Compositing
Decision Unit 1 Decision Unit 2
MI Sample
Decision Unit 5
sample
Decision Unit 6
Form one multi-increment sample for analysis
Form one composite sample for analysis

72
10June2008
Multi-Increment Sampling
Effective when cost of analysis is significantly greater than cost of sample acquisition/ handling How many increments?
Practical upper limit imposed by homogenization capacity, background concentration & magnitude of non-background concentration Enough to bring sampling error under control relative to other sources of error
10June2008
73
How Many Increments per MI Sample?

Addressing the unknown
For statistically valid conclusions, the number of increments is matched to level of heterogeneity present within decision unit
At least 10 preliminary measurements across decision unit give good estimate of variability (SD) Use SD & mean of preliminary data in VSP to determine # increments needed for desired statistical confidence in the decision units mean estimate Dynamic strategies make this process highly efficient
Dynamic Work Strategies
74
Uncertainty mgt
10June2008
How Many MI Samples Per DU?

Addressing the unknown
Only 1 MI sample per decision unit (DU) causes loss of spatial variability information
no means to QA (evaluate sufficiency) of increment #s difficult to implement statistical tests if DU average exceeds action level, cant identify where in the DU the problem is
Multiple MI samples per decision unit (e.g., 5)

provides the benefits of MI sampling (cost reduction, improved performance) while providing variability information to allow statistical tests (e.g., Student t test, Sign test, 95%UCL calculation, etc.) can help identify where contamination is in decision unit Dynamic work strategies highly efficient
10June2008
Statistical & Decision Benefits of MI Sampling

0.080 0.070 0.060 0.050 0.040 0.030 0.020 0.010 0.000 0 100 200 300 400 Concentration (ppm)
MI sample
Frequency
0.080 0.070 0.060 Frequency 0.050 0.040 0.030 0.020 0.010 0.000 0 100 200 300 400 500 600 Concentration (ppm)
600
500
Large grab sample
Physical equivalent of averaging many individual sample results mathematicallyMI sampling creates larger sample supports & tends to normalize statistical data distributions
Benefits & Limitations of MI Sampling

Significantly reduce analytical costs Significantly improve decision-making performance:
Reduce decision-making errors Much more likely that hot spots will be found and accounted for
Not as useful for subsurface & other sampling where sampling costs higher than analytical Requires special design & handling for volatile contaminants (Hg, VOCs, etc.) In situ & other cost-effective high density analyses (like XRF) potentially substitute or augment MIS
10June2008
Addressing the Unknown
Addressing the Unknown through Dynamic Work Strategies

Optimize data collection design
Real-time testing of CSM & obtain statistical design parameters (SD, preliminary mean, sample support)
Adaptive analytics
Strategies to produce collaborative data sets with sufficient analytical & sampling QC checks
Adaptive sampling
Strategies for confident estimates of DUs mean Strategies for delineating contaminant populations
Adaptive compositing
Efficient strategies for searching for contamination
10June2008
Adaptive Composite Searching

Goal: looking for contamination or demonstrating that large areas are compliant Assumptions:
Most of the area is clean/compliant Contamination is believed to be spotty Action level is significantly greater than background levels Sample acquisition/handling costs are less than analytical costs Appropriate methods exist for sample acquisition & aggregation
10June2008
Composite Searching Designs

Must determine
appropriate number of samples to aggregated into composite develop decision criteria to indicate when analyses of contributing samples are necessary
Performance (cost/benefit) best when

contamination is spotty big difference between background & action level big difference between average concentration & AL Best case: no composite requires re-analysis Worst case: every composite requires re-analysis
10June2008
Recipe for Adaptive Compositing

Determine appropriate number of samples to composite & decision criterion w/ this equation: Decision criteria = (action level - background)/(# of samples in composite) + background. Sample and split samples. Use one set of splits to composite and save other set. If:
composite result < decision criteria, done. composite result > decision criteria, analyze splits contributing to the pooled composite.
10June2008
How Many Samples to Composite?

Normalized Expected Cost vs Composite Size
1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3
s C t c p x E d e z i l a m r o N
Hit Prob = 0.2 Hit Prob = 0.1 Hit Prob = 0.05
How probable is it that contamination is present? The less likely it is that contamination is present, the larger the number of samples that can be composited
0.2 0.1 0.0 0 5 10
Hit Prob = 0.01

Hit Prob = 0.001
15 20
Number Contributing to Composite
Graph at left illustrates optimal sample numbers for different probabilities

82
10June2008
Example Decision Criteria

Background: 10 ppm; Action Level: 100 ppm Determine decision criteria for a 2-sample, 3sample, 4-sample, 5-sample & 6-sample composite:
Decreasing Analytical Costs Increasing Chance of Failing
2-sample composite: 55 ppm 3-sample composite: 40 ppm 4-sample composite: 33 ppm 5-sample composite: 28 ppm 6-sample composite: 25 ppm
10June2008
83
Performance (Cost/Benefit) Calculation

Compositing has a positive cost/benefit ratio as long as: Ff < 1 1/Nc where:
Nc = number contributing to composite Ff = fraction of composite samples failing (results above decision criteria)
10June2008
Other Assorted Statistical Strategies

Useful classical statistics strategies
Stratified sampling designs Bernards sequential t-test sampling design Binomial Sequential Probability Ratio Test (SPRT) (sequential non-parametric sampling design) Adaptive cluster sampling Ranked set sampling
Geostatistics (free software, Google: SADA, geostatistics)

Probability maps [cleanup if Pr(non-compliance) > X%]
GeoBayesian (free BAASS software, see Bob Johnson)

USACE: Managing Uncertainty with an XRF & Geostatistics
Color coding for probabilities that 1-ft deep volumes > 250 ppm Pb (actual Pb conc not shown)
Decision plan: Any soil w/ Pr(Pb > 250 ppm) > 40% will be landfilled. 10June2008 Triad Investigations: New Approaches andbe Innovative Strategies 86 Soil with Pr(Pb > 250 ppm) < 40% will reused in new firing berm.
The Biggest Bang Comes from Combining

CSM knowledge, with Multi-increment sampling, with Collaborative data sets, with Adaptive analytics, with Adaptive QC & data uncertainty reduction, with Adaptive compositing, with Adaptive sample location selection.
Module 6
Project Case Study:

Adaptive X-ray Fluorescence (XRF) Sampling & Analysis Design to Achieve Decision Confidence for Residential Soil Lead Concentrations
10June2008
Triad Investigations Conference
88
This slimmed down case study illustrates how to determine & control data error in real-time to generate definitive data This project used a handheld X-ray fluorescence (XRF) instrument to measure Pb in minutes at the site of sample collection
Plastic bag of soil
10June2008
89
This Projects Decisions

Is the Pb conc for each residential property below the 500 ppm risk-based AL? What is the greater source of data variability when reporting Pb conc results & how can it be reduced as needed?
10June2008
90
Data Collection Design

An entire yard is an exposure unit Grassy yard initially stratified into 3 sections
Section F: Front of house Section S: Side of house Section B: Behind house
Divide each section into 5 equal area subsections The subsections will be sampled by taking 1 grab soil sample (~300 g) per subsection & placing it into a plastic bag for XRF analysis
Illustrative Sampling Design & Results

Property: 702 Main Street
Front Yard: 5 Samples Side Yard: 5 Samples
Area fraction = 0.25
Action Level = 500 ppm
1 bagged grab sample
Back Yard: 5 Samples
House Footprint
Area fx = 0.15
Area fx = 0.6
Total yard average determined statistically (& area-weighted) as 10June2008 Triad Investigations: New Approaches and Innovative Strategies 92 410 +/- 25 (385 435 ppm Pb)
Front yard average (at 95% statistical confidence) = 700 +/- 150 (550 850 ppm Pb) Side yard average (at 95% statistical confidence) = 500 +/- 100 (400 600 ppm Pb) Back yard average (at 95% statistical confidence) = 300 +/- 50 (250 350 ppm Pb)
XRF Bag Analysis

Take 4 30-sec XRF readings on each bag (2 on front & 2 on back of bag) Real-time Pb results (2 min to handle each bag) entered into pre-programmed spreadsheet Spreadsheet immediately calculates average & statistical uncertainty for Pb conc. for
each bag each yard section, and the entire yard
10June2008
Follow Decision Tree #1 Triad Investigations: New Approaches and Innovative Strategies
93
Decision Tree #1
Evaluate statistical results for the yard & compare to the 500 ppm AL
Is there statistical confidence that mean is below AL? Is there statistical confidence that mean is above AL?
yes
If neither condition is true
yes
Decide Pb conc for the yard is below AL

200 +/- 50 (150 250)
Confident that no action needed
10June2008
Decision too uncertain: more information needed

300 +/- 100 (150 520)
Decide Pb conc for the yard is above AL

700 +/- 150 (550 850) Confident that action is required
94
Go to Decision Tree #2
Information from spreadsheet to feed into Decision Tree #2

to identify the most important source of data variability (aka, statistical error)
Average within-bag error (std dev, SD) for each of the 5 bags from a yard section Between-bag error SD for all bags from a yard section. Compare the average within-bag SD to the between-bag SD See example data set 10June2008 Triad Investigations: New Approaches and Innovative Strategies
95
Error Measurement in Data Results for a Backyard Section

Within-Bag Data Variability
Sect. Bag #1 Shot #1 700 #2 670 #3 740 #4 650 Bag Mean 690 W/in-Bag SD
Bag #2 #1 550 #2 534 654 #3 #4 590
39
582
Bag #3 #1 534 #2 440 #3 430 #4 420
54
456
Bag #4 #1 769 #2 710 #3 960 #4 800
53
810
Bag #5 #1 450 #2 400 #3 500 #4 550
475
107
65
Mean of within-bag SDs = (39+54+53+107+65)/5 = 63 Between-Bag Data Variability
690
10June2008
582
456
810
475
96
Between-bag error (SD) for and the 5 bag Strategies means = 150 Triad Investigations: New Approaches Innovative
Decision Tree #2
Determine the greater source of data variability (decision uncertainty) Is within-bag variability GREATER than between-bag variability? yes Go to Decision Tree # 3 no, they are ~equal Go to Decision Tree #5
10June2008
no Is within-bag variability LESS than between-bag variability? yes Go to Decision Tree #4

97
Decision Tree #3
Major source of data error: heterogeneity within sample bag (subsampling error)
To control this source of variability: Re-shoot each bag another 4 times (total of 8 shots/bag. Add results to spreadsheet & recalculate stats for whole yard. Examine results.
Is within-bag variability sufficiently reduced? no Take addl corrective action
10June2008
yes Make decision at 500 ppm AL w/ desired statistical confidence

98
Decision Tree #2
Determine the greater source of data variability (decision uncertainty) Is within-bag variability significantly GREATER than between-bag variability? yes Go to Decision Tree # 3 no, they are ~equal Go to Decision Tree #5 no Is within-bag variability significantly LESS than between-bag variability? yes Go to Decision Tree #4
99
10June2008
Major source of data error is from concentration variations across the yard section area.
To control this source of variability: Collect another 5 bag samples from section area. Analyze 4 times/bag. Add results to spreadsheet & recalculate statistics for whole yard.
Is between-bag variability sufficiently reduced? no Take addl corrective action yes Make decision at 500 ppm AL w/ desired statistical confidence
100
Decision Tree #4
10June2008
Decision Tree #2
Determine the greater source of data variability (decision uncertainty) Is within-bag variability significantly GREATER than between-bag variability? yes Go to Decision Tree # 3 no, they are ~equal Go to Decision Tree #5 no Is within-bag variability significantly LESS than between-bag variability? yes Go to Decision Tree #4
101
10June2008
Decision Tree #5
Concentration variability across yard section & within samples about the same.
Analyze original bags an addl 4 times each. Also collect another 5 bag samples from the section & analyze 8 times each. Add all results to spreadsheet & recalculate statistics for whole yard.
Is statistical decision uncertainty now sufficiently resolved? no Take addl corrective action
10June2008
To control both sources simultaneously:
yes
Make decision at 500 ppm AL w/ desired statistical confidence Triad Investigations: New Approaches and Innovative Strategies 102
Benefits of the Dynamic XRF Strategy

Data gathered in real-time Data evaluated against decision goals in realtime Data uncertainty identified & measured in realtime (definition of definitive data) Decision tree guides actions to resolve data uncertainty in real-time Final decisions can be made in real-time Property owners informed of decisions in realtime

Best Practices for Soil Sampling Design

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Best Practices for Soil Sampling Design

Uploaded by

Copyright:

Available Formats

Module 1

Best Practices for Efficient Soil Sampling Designs

The Point of this Workshop

is not to say statistics are wrong, never appropriate, or cannot be used.

Software Resources and Disclaimer

Triad Investigations: New Approaches and Innovative Strategies

Take Away Points

Triad Investigations: New Approaches and Innovative Strategies

Where Does Uncertainty Come from When Making Restoration Decisions?

Decision Uncertainty Comes from a Variety of Sources

Sample Prep Method

Relationship between Measurement Parameter & Decision Parameter

Taking a Sample for Analysis

Field 10June2008 Subsample

Analytical Sample Prep

Triad Investigations: New Approaches and Innovative Strategies

Historically the Focus Has Been Analytical Quality

The Biggest Cause of Misleading Data

Triad Investigations: New Approaches and Innovative Strategies

Within-Sample Variability: Interaction between Contaminant & Matrix Materials

10 50 108 165 836 1,970 927 (wt-averaged)

Concentrated Particles within Less Concentrated Matrix = Nugget Effect

Now we know the assumption is inaccurate because of microscale (within-sample) heterogeneity.

Micro-scale Heterogeneity Causes Highly Variable Data Results

101 - 800 136 - 343 170 - 230

Adapted from DOE (1978) americium-241 study

Short-Scale Variability Can Also be Significant

Triad Investigations: New Approaches and Innovative Strategies

Heterogeneity Overwhelms Variability from Different Analytical Techniques

500 On-site 416 Lab

1,280 On-site 1,220 Lab

39,800 On-site 41,400 Lab

95% of data variability due to sample location over a 4 ft diameter

164 On-site 136 Lab

24,400 On-site 27,700 Lab

27,800 On-site 42,800 Lab

Triad Investigations: New Approaches and Innovative Strategies

Sampling Uncertainty Example: AU = 10 ppm, SU = 80 ppm: TU = 81 ppm

How Do We Reduce Data Uncertainty?

For sample prep and handling errors:

For sampling errors:

Triad Investigations: New Approaches and Innovative Strategies

To be [averaged] or not to be [averaged], that is the question

Triad Investigations: New Approaches and Innovative Strategies

Sometimes the Simplest Questions are the Most Complex

Triad Investigations: New Approaches and Innovative Strategies

Sample Support, Representativeness and Decision Unit Support are Intertwined

Surface layer of interest

Layer impacted by deposition

Advances in Sampling & Measurement Technologies Highlight Representativeness Issues

The Same Holds True for Soils

Concentration (ppm) Concentration (ppm)

Triad Investigations: New Approaches and Innovative Strategies

The Decision Unit is Often Not WellDefined

Triad Investigations: New Approaches and Innovative Strategies

Complete Cleanup Criteria Definitions

Must DEFINE population of interest to interpret data!!

For Soils, Three Cleanup Requirement Definitions are Most Common:

Triad Investigations: New Approaches and Innovative Strategies

Technically Defensible Sampling Programs Require Complete Criteria

Triad Investigations: New Approaches and Innovative Strategies

All Solid Samples are Composites!!