Cannabis Testing Laboratory Round Robin

An inter-lab comparison study between certified laboratories in Washington State

2017-08-03
Purpose
With the overarching goal of Lab Standardization at the core of their intent, eight Certified Cannabis Testing
Laboratories in Washington State collaborated on a four-part Round Robin experiment to effectively gauge the
reproducibility between them on homogenized Cannabis samples typical to the cannabis industry. The purpose of this
experiment was to improve our understanding of the differences between these labs in how they measure and report
cannabinoid concentrations, and to establish a baseline for the variance between them.

Participating Labs
Eight labs volunteered to participate in the study, and all eight successfully upheld their commitment. Additionally,
three of the eight labs are dues-paying members of The Cannabis Alliance and have worked extensively and
collaboratively on regulatory revisions that will help to bring greater Lab Standardization to the industry.

Cannabis Alliance Participating Laboratory Members:

Confidence Analytics Testing Technologies Steep Hill Washington

Certification #: 0003 Certification #: 0007 Certification #: 0002
14797 NE 95th St 19834 Viking Ave. NW 720 Industry Drive
Redmond, WA 98052 Poulsbo, WA 98370 Tukwila, WA 98188
(206)743-8843 (360) 930-0083 (253) 277-8936

All Participating Laboratories:

Analytical 360
Confidence Analytics
Green Grower Labs
Medicine Creek Analytics
Molecular Testing Labs
Steep Hill, Washington
Testing Technologies
Trace Analytics

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 2 of 12
Summary
Samples of Cannabis flower and concentrate were donated from lots/batches of final product with known pedigree.
All participating labs utilized high-performance liquid chromatography (HPLC) for their analysis and results were
blinded by simple randomization.

The spread of reported results on homogenized flower was consistent between two samples with two different
primary analytes. CBDA and THCA in Sour Tsunami and Dutch Treat flower samples both yielded a 3 point spread
between labs and the relative standard deviation was consistent among all samples. Each lab has good precision in
their measurements, and through improved accuracy by each lab individually, a tighter spread between the labs could
be readily achieved.

Three prominent areas are identified where the labs can make meaningful improvements to elevate cannabis
laboratory standardization:

Consistency Across Labs: Labs who tend to report higher than the mean for THCA do so for all samples, as do labs that
report lower, or close to the mean. There are a wide variety of procedural steps that can affect the precision of a HPLC
method, and it is through standardization that greater consistency can be achieved.

Compound Identification: Labs disagreed about the identity of minor cannabinoids (specifically CBDA and CBGA), with
some participating labs identifying the same compound as different analytes. This is difficult, but can be resolved
through comparisons of chromatograms and spiking samples with known standards.

Detection Sensitivity and Quantification Limits: Some discrepancies in limits of detection between the labs were
revealed. In analytical chemistry, Limit of Detection (LOD) and Limit of Quantification (LOQ) are the terms used to
describe the sensitivity of a method, and have many definitions across many industries. In cannabis analysis, no
standard definitions yet exist, and so labs may be using alternate definitions adopted from other industries.

CALLS TO ACTION: The Cannabis Alliance requests that labs engage in an open dialog to:

Identify sources of discrepancy and develop standards for HPLC analysis of cannabis
Better understand challenges around measurement of minor cannabinoids
Establish a consensus for instrument quantification and detection sensitivity definitions and limits

Moving Forward
In their continued commitment to lab standardization, The Cannabis Alliance and the participating labs will continue
to facilitate further conversation and investigation toward that goal. In the coming weeks, a follow-up round robin by
the labs is investigating within-grow-room variability. While the current experiment provides a look at the laboratorys
contribution to variance, the next experiment will investigate the grow rooms contribution to variance. Again,
multiple cannabis labs will be participating. Please stay tuned.

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 3 of 12
Methods
Samples of Cannabis flower and Cannabis concentrate were donated by licensed marijuana producers and processors,
representing four distinct Cannabis materials taken randomly from lots/batches of final product with known pedigree,
ready for sale, and tracked in the seed-to-sale traceability system. These four distinct samples were chosen for their
diversity of cannabinoid content, both in absolute quantity of total cannabinoid and in diversity of cannabinoid
chemical constituents. The producer/processors donating the material are credited below.

The Four Twenty Collection: Sour Tsunami (Flower) and Dutch Treat (Flower)

Trail Blazin Productions: Dutch Treat (Kief) Gravity Thieves: Mixed (BHO)

McCallum Company: Mylar Bags

Before distributing to labs, the sample material was homogenized by freeze-milling. Seven grams of each sample were
placed in sanitary 50 ml polycarbonate test tubes with screw caps along with two 11 mm stainless steel ball bearings.
The tubes and contents were chilled to -40 C in an IRINOX MF 30.2 ETL blast freezer for 30 minutes. Upon removal
from the blast freezer, the tubes and contents were placed in a SPEX SamplePrep 2010 Geno/Grinder automated
tissue homogenizer at 1500 RPM for one minute.

Homogenization strategy is an important component in an experiment like this one, and should not be understated.
Previous experiments by other industry groups have attempted to homogenize Cannabis flower by simply grinding the
plant material in a kitchen blender. While the powdered material produced by the blender may appear homogenous
at a glance, it consists of widely varying particle size that sifts immediately upon disturbance and has non-uniform
static properties. In this way, the kitchen blender may actually make the Cannabis flower less homogenous than its
native state. In comparison, the freeze-milling technique described above results in a homogenate of highly uniform

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 4 of 12
particle size with consistency like hydrophobic sand (Magic Sand) with noticeably less static tendency than the
powder produced via the kitchen blender.

Once each of the samples was successfully homogenized, they were each subdivided into 0.5 gram subsamples placed
in 4 ml polycarbonate sample containers with screw caps. Those containers were further packaged in 3 mil Mylar
baggies and affixed with tamper evident labels assigning unique identifiers to each. One subsample from each of the
four samples was manifested and delivered to each participating lab with instructions to test for cannabinoid content.

All participating labs utilized high-performance liquid chromatography (HPLC) for their analysis. Labs returned their
results directly to The Cannabis Alliance. The results were then blinded by simple randomization, and the results were
returned to the labs such that each lab could view the results from each of the other labs, but could not easily identify
which lab had returned which results.

Outcomes
Each of the eight labs who volunteered to participate in this study successfully completed their commitment to the
project. Their willingness to collaborate with each other and with industry stakeholders toward an advancement in
standardization is both commendable and highly valuable to the future of the cannabis industry. At its core, science is
a collaborative discipline, fraught with uncertainty, and susceptible to doubt and denial. To have industry leaders, and
business competitors, working together toward meaningful improvements to standards of practice is especially
needed in a nascent industry where the unknowns are multivariate and the guidelines are still developing.

Primary outcomes from the experiment are illustrated below. The primary analyte for each of the four samples is
graphed in percent by weight as reported by each lab. The boxes represent the first and third quartiles of the
distributions, and the horizontal line represents the median.

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 5 of 12
The spread of reported results on homogenized flower was consistent between two samples with two different
primary analytes. CBDA in Sour Tsunami and THCA in Dutch Treat both yielded a 3 point spread between participating
labs, ranging 11-14% and 15-18%, respectively. Spread was wider on higher concentrations, but the relative standard
deviation was consistent among all samples, with standard deviations of 0.08, 0.07, 0.06, and 0.07 as a proportion
relative to the mean (/) for CBDA in Sour Tsunami Flower, THCA in Dutch Treat Flower, THCA in Dutch Treat Kief,
and THCA in Mixed BHO, respectively.

Labs that tested above the median when measuring THCA in one sample tended to test above the median when
measuring THCA in all samples, and the same is true of labs that tested below the median. This outcome in
conjunction with the consistency in relative standard deviation and observed linearity of each lab individually when
compared to the mean implies that each lab has good precision in their measurements, and through improved
accuracy by each lab individually a tighter spread between the labs could be readily achieved. The differences in
measurements between these labs do not appear random, they appear systematic, and thus can be attributed to
differences in calibration or protocol rather than attributed to error. Ultimately, this is good news, as systematic bias is
often easier to correct than random error.

Room for Improvement

While the outcomes of this experiment lend credibility to those labs willing to collaborate, and show that the
variability between them (described as one standard deviation) is less than 1/10th the measurement, there is always
room for improvement. Below, we briefly discuss three prominent areas where the labs can make meaningful
improvements moving forward as a direct result of their continued participation in what is an ongoing industry effort
to elevate cannabis laboratory standardization.

While this experiment may reveal better reproducibility than has been published before, the labs participating in this
study represent only a subset of the labs certified in Washington. Labs participating in this study are at the forefront of
laboratory standardization in the Washington cannabis industry and have shown the greatest willingness to raise the
bar on lab standardization. Among these labs, there is still room for improvement. Through discussions about
laboratory practices, calibration standards sources, methods of calibration, and quality control checking, we are
confident these labs can improve upon their current performance as they strive for tighter consistency.

Consistency across labs

Probably the most meaningful lab standardization outcome to stakeholders in the cannabis industry is consistency
between labs. Producers and processors want assurance that they will get representative and comparable results
regardless of what lab they may choose to hire. Within-lab precision is certainly better than between-lab precision,
and while the latter will never be as tight as the former, through collaboration the latter can approach the former.

The graph below depicts the linearity and slope of each lab individually as compared to the mean value of all the labs
when reporting THCA on each of the four samples. Here, we are assuming that the mean value between all labs is the
true value, and we are graphing each labs reported results as individual series. Logistic regression comparing each

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 6 of 12
labs reported THCA values to every other labs THCA values yields Pearsons Correlation Coefficients (R^2) of 0.9970
or higher for all 28 pair-wise comparisons. Excluding lab 814 from the analysis, whose series is visibly kinked between
sample #3 and sample #4, all remaining pair-wise comparisons give R^2 values greater than 0.9992. Generally,
chromatographers consider a R^2 correlation of 0.995 or higher as good, and 0.999 or higher as excellent. R^2 = 1 is a
perfect correlation.

Labs who tend to report higher than the mean for THCA do so for all samples with a relative distance from the mean
roughly consistent across samples (Lab Codes 171, 490, and 682). Labs who tend to report lower than the mean
behave similarly (Lab Codes 548 and 814). So, too, do labs who tend to report close to the mean (Lab Codes 378, 743,
and 926). Given this observation, it becomes clear that the differences between labs as they report on the primary
analyte, THCA are the result of systematic inaccuracies in a broader context of high precision within each lab
independently. Thus, these systematic differences must be attributable to process accuracy more so than process
precision. While there are a wide variety of procedural steps that can affect the precision of a high-performance liquid
chromatography method including sample preparation parameters, instrument acquisition parameters, and
instrument maintenance and calibration it is fair to assume that through standardization of these procedural steps
greater consistency between labs can be achieved.

Call to action: The Cannabis Alliance hereby requests that the labs begin an open dialogue between them to identify
their sources of inaccuracy, to develop a set of standardized criteria for preparing samples for HPLC analysis, to
identify sources of calibration error, and to ensure that each is doing their due diligence in their own instrument
maintenance. By no means do we expect labs to divulge their proprietary methods or intellectual property, we only
ask that they work collaboratively to identify their largest sources of inaccuracy and to take meaningful steps toward
improvement as a group.

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 7 of 12
To facilitate discussion among the labs, the following is a brief list of the factors that can contribute to variance in a
cannabis testing circumstance. This list is not exhaustive, and not all the factors below are directly within the
laboratorys control. Ultimately, the standardization of test results is an industry-wide endeavor.

Between- and within-harvest variability Instrumentation Calibration method and frequency

Between- and within-grow-room variability
Between- and within-strain variability
Between- and within-lot variability
Between- and within-sample variability
Sampling protocol
Sampler (person conducting sampling protocol)
Sample handling and transport
Homogenization strategy
Subsampling strategy
Balance calibration
Extraction solvent selection
Dilution factor and dilution method
Diluent solvent selection
Sample hold time
Mobile phase Quantification wavelength
o Organic solvent selection Detector calibration (refraction grating)
o Mobil phase pH Compound separation
o Elution gradient Retention time stability
Sample compartment temperature Integration parameters (esp. tailing)
Column compartment temperature Quality control parameters
Capillary internal diameter and injection Internal reference standard
volume Use of blanks and controls
Stationary phase Certificate of Analysis nomenclature
o Packing material Labeling consistency and nomenclature
o Pore size Regulatory consistency and nomenclature
o Column diameter Advertising/ media use of nomenclature
Back pressure Budtender consistency and nomenclature
Certified Reference Material supplier

Compound identification
In more than one case during this experiment it appears that labs disagree about the identity of minor cannabinoids,
with some participating labs identifying the same compound as different analytes. This is made difficult by the sheer
diversity of compounds in the cannabis matrix, but can be resolved through comparisons of chromatograms and
spiking samples with known standards. Again, through investigation and discussion this is an area where improved
standardization between labs can be readily achieved.

Within this experiment, one particular circumstance stands out, and we will use it here to highlight the issue of
compound identification. Trail Blazin Productions Dutch Treat kief sample contained measurable amounts of CBDA
and CBGA, in roughly equal quantities (2-3%), but most participating labs did not measure and report both analytes at
roughly equal quantities. To provide clarity on the matter, lab #378 has shared their chromatogram of the Trail Blazin
Productions Dutch Treat kief showing peak areas of roughly equal amounts for CBDA and CBGA. At our request, lab
#378 went a step further and spiked the Trail Blazin Productions Dutch Treat kief sample with Certified Reference
Materials (CRMs) obtained from Cayman Chemical to highlight the correct identity of the two peaks in question.

The following page shows three chromatograms, all from the same Trail Blazin Productions Dutch Treat kief sample,
as tested by lab #378. The chromatograms are different only in that the second and third chromatograms come from
the sample after spiking with CBDA and CBGA CRMs, respectively.

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 8 of 12
Trail Blazin Productions Dutch Treat kief sample without standard addition

Trail Blazin Productions Dutch Treat kief sample with CBDA standard addition

Trail Blazin Productions Dutch Treat kief sample with CBGA standard addition

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 9 of 12
In the chromatograms on the previous page, we can see the instrument acquisition for the Trail Blazin Productions
Dutch Treat kief sample. From left to right, the first peak to elute, highlighted in red, is the internal standard, a spike of
a non-cannabinoid chemical that lab #378 puts in every sample to monitor instrument stability and sample integrity.
The second peak, highlighted in yellow, is the peak eluting in the expected CBDA retention time window. The third
peak, highlighted in blue, is the peak eluting in the expected CBGA retention time window. In this discussion, we are
focusing on the second and third peaks, CBDA and CBGA, respectively.

Chromatographers use retention time (the x-axis) to identify each compound in the sample mixture. Due to varying
polarity, solubility, size, shape, and other factors, different chemical constituents of a chemical mixture will travel
through an HPLC stationary phase at different speeds, and the retention time is a measure of how much time each
chemical constituent took to exit the stationary phase (the column) after they all entered the stationary phase at the
same time. In this way, chromatography separates the components of a mixture, so that each chemical constituent
can be individually measured.

The y-axis of the chromatograms shows the relative absorbance of light for each of the separated compounds. The
amount of light absorption is directly proportional to the concentration of the component absorbing the light, as
described by what scientists refer to as Beers law or the Beer-Lambert law. Thus, the area under each peak on the
chromatogram is directly proportional to the concentration of that chemical in the chemical mixture. The blue line on
the chromatograms above is a recording of the absorbance of light at the 220nm wavelength (Dad1A), which is the
frequency of light that most cannabis labs use to measure cannabinoids.

Calibration curves for CBDA and CBGA demonstrate that the response factors for these two analytes at the 220nm
wavelength are nearly identical. This means that the relative heights of those two peaks in a sample is nearly
equivalent to the relative concentrations of those two analytes. We can see in the first, unspiked, chromatogram the
relative concentrations of CBDA and CBGA are roughly 3:2 (the CBDA peak is slightly higher than CBGA peak). The
second chromatogram is identical to the first, but the sample was spiked with a CRM of CBDA at a nominal equivalent
of 3% in a sample, and we can see that the peak height of CBDA is now much higher than CBGA, with a ratio between
them of roughly 6:2. In the third chromatogram, the sample was instead spiked with a CRM of CBGA at the same spike
concentration, and we can see that the ratio between the two is now roughly 3:5.

The above observations demonstrate that lab #378 correctly identified both the CBDA and CBGA peaks, was able to
separate the two components to baseline, and the Trail Blazin Productions Dutch Treat kief sample contains both
analytes in relative concentrations of roughly 3:2, respectively. While five labs in this round robin identified at least
one analyte in this sample at a concentration of ~2-3%, few found both analytes at that relative concentration, with
most labs underreporting CBDA. While there are many possible factors that could account for such discrepancy, the
most likely causes are: retention time stability, chromatographic separation, or QC criteria. Standards additions, as
described above, are the appropriate means by which retention times can be confidently assigned to appropriate peak
areas.

Call to action: We call on all labs in this round robin to investigate their chromatograms for the Dutch Treat kief
sample, as lab #378 has done. They need not share them publicly, but should investigate on their own both with and
without spikes and should collaborate with each other to reach a better understanding of how to measure these two
important analytes.

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 10 of 12
Detection Sensitivity and Quantification Limits
This experiment was principally focused on the primary cannabinoid in each sample, with the primary cannabinoid
defined as the cannabinoid in greatest abundance. The labs were asked to additionally report all measured
cannabinoids. This revealed some discrepancies in limits of detection between the labs, with some labs reporting a
lower detection threshold than others. In some cases, labs quantified minor cannabinoids at concentrations higher
than the stated limit of detection by other labs that did not detect the same analyte. This indicates that the stated
limits of detection may not be as described. Through discussion about methods of calculating minimum detection
limits, and by exchanging method parameters that can improve dynamic range, the labs will be able to more
appropriately describe and adhere to the limits of detection they specify on their Certificates of Analysis.

In analytical chemistry, two terms are often used to describe the sensitivity of a method. They are: Limit of Detection
(LOD) and Limit of Quantification (LOQ). The LOD describes the lowest quantity of a substance that can be
distinguished from the absence of that substance within a stated confidence limit. In other words, the LOD is the
lowest amount of chemical analyte that the method can confidently detect. The LOQ is always a number higher than
the LOD and describes the lowest concentration of the analyte that the method can not only detect but also quantify
with reasonable accuracy.

There are many different definitions for LOD and LOQ, and different industries have standardized different definitions
for themselves. In cannabis analysis, no standard definitions have been described, and so each cannabis laboratory is
likely using a different definition adopted from another industry. Toward the goal of lab standardization, we describe
below some proposed definitions for the cannabis industry that are reasonable given the underlying constraints of
cannabis testing.

RJ Lee Group, the cannabis laboratory accreditation body contracted by the Washington State Liquor and Cannabis
Board (WSLCB) to audit and certify the labs recommends a definition for LOD described by Title 40 Part 136 found in
Appendix B of the Code of Federal Regulations (CFR) as related to testing drinking water and wastewater for trace-
level contaminants. While the CFR describes what it calls a Method Detection Limit, it is synonymous with method
LOD. Briefly, the CFR directs the determination of LOD using 7 or more replicate spikes on a known-blank sample
matrix at a level at or near the predicted LOD, and gives some guidance for predicting LOD, which can be the methods
LOQ. It goes on to direct calculating the standard deviation (S) of the spikes as follows:

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 11 of 12
The CFR then directs using the standard deviation to calculate the LOD as follows:

In this way, the LOD calculation uses precision at a low-spike level as a proxy for estimating the level at which the
analyte would be detected and discerned from a blank with 99% confidence.

LOQ is often easier to define than LOD, because LOQ is a quantifiable measure. Reasonably, the lab could elect to
define LOQ as the nominal in-sample concentration of the lowest calibration point for that analyte. Furthermore,
the signal to noise ratio at the LOQ should be greater than 10. In this way, it is guaranteed that the lowest quantity
ever reported is within the calibration range, and is reasonably discernable from background noise.

Whatever method is used for LOD and LOQ determination/ definition, the precision of the LOQ should be routinely
challenged by the laboratory. The LOD determination method directed by the CFR is adequate both for determining
the LOD and for challenging the precision at the LOQ.

Call to action: The Washington cannabis labs should come together to find consensus on the definitions of LOD and
LOQ. Each lab should independently investigate the LODs and LOQs for their instruments and methodologies for every
chemical analyte they report, and should put forth serious effort to describe those limits accurately on their
Certificates of Analysis.

Follow on
In their continued commitment to lab standardization, The Cannabis Alliance and the participating labs will continue
to facilitate further conversation and investigation toward that goal. In the coming weeks, a follow-up round robin
orchestrated by Molecular Testing Labs in Vancouver is investigating within-grow-room variability. A producer has
agreed to submit several dozen samples from the same harvest, same room, selected by the lab at multiple places in
the room and multiple places on the plants, to describe the variance within and between strains, plants, locations on
plants, and locations in a room/row. While the current experiment provides a look at the laboratorys contribution to
variance, the next experiment will investigate the grow rooms contribution to variance. Again, multiple cannabis labs
will be participating. Please stay tuned.

3233 S. Hanford St.

Seattle, WA 9814 Copyright 2017 The Cannabis Alliance
Email: info@thecannabisalliance.us Page 12 of 12