USDA/GIPSA Proficiency Program

Testing for the Presence of Transgenic Events in Corn and Soybeans

October 2004 Final Report

 

Purpose of USDA/GIPSA Proficiency Program

Through the USDA/GIPSA Proficiency Program, USDA seeks to improve the overall performance of testing for biotechnology-derived grains and oil seeds.  The USDA/GIPSA Proficiency Program helps organizations identify areas of concern and take corrective actions to improve testing accuracy, capability and reliability.

 

Program Description

In February 2003, USDA/GIPSA’s Technical Services Division expanded the program to offer samples for qualitative or quantitative analysis.  Participants could request samples for qualitative analysis or quantitative analysis.  In this round of the USDA/GIPSA Proficiency Program one set of samples was used for both qualitative and quantitative analyses.  The samples were fortified with various combinations and concentrations of transgenic events, and participants had the choice of providing qualitative or quantitative results.  Scoring of the participant’s results was done by computing the “percentage of correctly reported transgenic events” in the samples.  Two new biotechnology corn events commercialized in the U.S. in 2003 were included in this round of samples:  MON863 (Monsanto event) and TC1507 (Dow AgroScience/Pioneer Dupont event)

 

Sample Composition  

The corn samples contained various combinations and concentrations of the following transgenic events: T25, CBH351, MON810, GA21, E176, Bt11, NK603, TC1507, and MON863; or, no events (i.e., negative corn sample).  To produce the various concentration levels the appropriate amount of ground transgenic corn was mixed with non-transgenic corn.  Fortified corn samples were prepared with concentrations ranging from 0.1% to 5% of the transgenic event.  The soybean samples were either non-transgenic soybeans, or fortified to a range of 0.1 % to 3 % of the glyphosate-tolerant event (CP4-EPSPS).   
 
Each participant received six corn samples and three soybean samples.  Each sample contained 20 grams of ground material.

 

Program Participants

GIPSA mailed samples to sixty-nine participants and received results from sixty participants.    Participants included organizations from Africa, Asia, Europe, North America, and South America.  Each participant received a study description and a data report form by electronic mail, and with the samples.  Participants submitted results by electronic mail, FAX, or regular mail.  No methodologies were specified, and organizations used both DNA- and protein-based testing technologies. 

 

Sixty organizations returned results by the deadline date: 

 

·                     Twenty-six participants submitted qualitative only results

 

·                     Three participants submitted quantitative only results

 

·                     Thirty-one participants submitted a combination of qualitative and quantitative results. 

 

 

In this report, participating organizations were identified by either a confidential “Participant Identification Number”, or by name.  Appendix I identifies those organizations who gave GIPSA permission to list them as participants in the USDA/GIPSA Proficiency Program.

 

Data Summary Results

Data submitted by the participants are summarized in this report primarily in tables and figures.  Participants reported their results on a qualitative basis, quantitative basis, or a combination of both types.  The quantitative results were qualitatively scored (i.e., on the basis of “incorrect” or “correct’ for the presence or absence of a transgenic event in an unknown sample) instead of being scored on the basis of how accurate their reported quantification value was to the gravimetric fortification level. Quantitative results were reported as the concentration of a particular event in the sample, but were scored qualitatively—either correct or incorrect for a specified event.  Participants that reported results “less than” the limit of detection of their analytical method (e.g., < 0.05%) for non-fortified (negative) corn and soybean samples were assessed a “correct” result on those relevant samples.

 

 

Qualitative Data Summaries.  This section summarizes qualitative sample analysis data:

 

·              Table One.  Performance scores for participants sorted by event (DNA-based assays).

 

·              Figure One.  Summary of qualitative results for each event combined with the total number of results (inset values) submitted for the event (DNA-based testing using conventional PCR).

 

·              Table Two.  Percentage correct scores for all participants by event based on lateral flow strip testing (Protein-based assays).

 

·              Table Three.  Percentage correct scores for all participants by event based on ELISA (Enzyme-Linked Immunosorbent Assay testing; Protein-based assays).

 

·              Table Four.  Error analysis: Percentage of false negative and false positive results in quantitative analysis results using conventional PCR, lateral flow strips, and ELISA.

 

 

Quantitative Data Summaries.  This section summarizes quantitative sample analysis data:

 

·         Table Five.  Performance scores for participants sorted by event (DNA-based assays).

 

·         Figure Two.  Summary of quantitative results for each event combined with the total number of results (inset values) submitted for the event (DNA-based testing using conventional PCR).

 

·         Table Six.  Error analysis: percentage of false negative and false positive results in quantitative analysis results using real-time PCR.

 

·         Table Seven.  Summary of participant’s results relative to gravimetric fortification levels.

 

 

 

Qualitative Analysis Results

 

Table One:  Performance scores for participants sorted by event (DNA-based assays).

 

 

 


Figure One: Summary of qualitative results for each event combined with the total number of results (inset values) submitted for that particular event (DNA-based testing using conventional PCR).

 

 

 

Table Two:  Percentage correct scores for participants based on lateral flow strip testing (Protein-based assays).

 

 

 

 

 

 

 

 

 

Table Three:  Percentage correct scores based on Enzyme-Linked Immunosorbent Assay testing (Protein-based assays).

 

 

Table Four:  Error analysis: percentage of false negative and false positive results in qualitative analysis results using conventional PCR, lateral flow strips, and ELISA.

 

% False Positive = Total number of incorrect positive reported samples

                               Total number of negative samples provided

 

% False Negative = Total number of incorrect negative reported samples

                                 Total number of positive samples provided

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Quantitative Analysis Results

 

Table Five: Performance scores for participants sorted by event (DNA-based assays).

 

 

 

Figure Two. Summary of quantitative results for each event combined with the total number of results (inset values) submitted for the event (DNA-based testing using real-time PCR).

Table Six:  Error analysis: Summary of percentage of false negative and false positive results in quantitative analysis results using real-time PCR. 

 

% False Positive = Total number of incorrect positive reported samples

                               Total number of negative samples provided

 

% False Negative = Total number of incorrect negative reported samples

                                 Total number of positive samples provided

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table Seven.  Summary of participant’s results relative to event fortification levels.

*Indicates that some values were excluded from analysis because they were outliers (i.e., participant’s reported quantifications were > 20x above the fortification level).

 

 

 

 

Summary

Qualitative Analysis Results

For DNA-based testing using conventional PCR, the participants scored 100% correct on three of twelve events and nearly perfect (≥ 95% correct) on five of twelve events; however, the event that evinced the lowest performance was T-25 (Figure One).  The participants were unable to detect 0.1% to 0.4% T-25 in their fortified corn samples about 10% of the time.  An analysis of the source of their collective error showed that the lower performance scores for T-25 were due mainly to the reporting of false negative results (Table Four).  This observation of apparent “reduced detection efficiency” for the T-25 insert has been noted in previous results of the proficiency testing program (e.g., April 2004 Final Report).

 

For protein-based testing, using lateral flow strips, participants achieved a 100% correct score whenever samples were analyzed for CBH351 (Table Two); however, when samples were analyzed for T-25 and Cry1Ab a mixture of false positives and/or false negatives were observed (Table Four).  In a previous sampling round (April 2004), similar lower performance scores were observed for those two events when analyzed by lateral flow strips.  When participants used ELISA, they achieved perfect results for the events they chose to analyze--with an exception for TC1507 (Table Three) which identified two false positive results (Table Four). 

 

Quantitative Analysis Results

The quantitative results were qualitatively scored (i.e., on the basis of “incorrect” or “correct’ for the presence or absence of a transgenic event in an unknown sample) instead of being scored on the basis of how accurate the reported value was to the fortification level.  With GIPSA’s lenient scoring guidelines for DNA-based testing, using real-time PCR, the participants scored a perfect 100% correct on six of twelve events and nearly perfect (≥ 95% correct) on all other events with the exception of MON863 (Figure Two).  The reported errors in sample analysis for MON863 were due entirely to false positives (Table Four). When comparing October 2004 quantitative results with that of previous rounds of proficiency test results (i.e., April 2004), the participants achieved a perfect score on nine of the twelve events— including MON863.  When participants report results based on the use of real-time PCR (RT-PCR) methods, there are a greater number of perfect performance scores achieved per event compared with laboratories reporting qualitative results using conventional PCR methods. We suggest that better performance scores were achieved using RT- PCR as compared to conventional PCR because: 1) the RT-PCR method has a lower LOD (greater sensitivity) than conventional PCR and therefore is able to avoid reporting false negative results on samples that are fortified at very low levels (e.g., 0.1% w/w biotechnology-derived plant material),  2) laboratories that use RT-PCR may be more proficient in their methods 3) quantitative results, reported as the concentration of a particular event in the sample (%w/w), were scored qualitatively as either “correct” or “incorrect”, regardless of the fortification level in the sample (e.g., a sample that was fortified at 5.0% but was reported as 0.1% received a “correct” result for that analysis).  Tables Four and Six show that, for example with T-25, there was a four-fold decrease (a significant decrease when tested at the 95% confidence level) in the rate of false negative reported results when RT-PCR was used instead of [or in conjunction with] conventional PCR.

 

We prepared the proficiency samples to contain a range of transgenic events and fortification levels, thus participants who analyzed them were tasked to report an array of transgenic events and concentration values.  The collective results of their analysis are summarized in Table Seven according to the number of reported results for that event (N), the group mean, and standard deviation.  Additionally, a measure of “precision” or reproducibility was computed as the “% CV” (s.d. divided by the mean) x 100, and a measure of the “accuracy” was computed as “% relative error” (reported quant value minus fortification value, then divide the difference by the fortification value) x 100.  The notable trends in this dataset were as follows:

  • The participants reported values in this round were generally more accurate than values reported in the previous round (April 2004).  There was not as strong of an inverse relationship between “% relative error and fortification level” and between “%CV and fortification level” as had been observed in the previous sampling round.
  • The inter-laboratory precision was not measurably improved in an overall manner as compared to the April 2004 results, as there were still a fair number of imprecise measurements (CV ≥ 25%).  This high level of inter-laboratory variation (i.e., the variance contributed by all participants for a given transgenic type and concentration level) could be explained by at least two main factors:  1) heterogeneity in the sample matrix being tested, and 2) analytical test method variability between various laboratories.  We suggest that the second factor may contribute the greater effect on the high levels of variation in reported quantifications. These results suggest that there would be technical difficulty in distinguishing between lots of grain having two different transgenic concentrations.  For example, if a randomly selected lab were tasked to distinguish a lot of grain containing MON810 fortified at the “0.4% level” from a lot of grain with “0.8% MON810” (Table Seven), then due to the high % CV’s inherent to each mean value there would be some overlap of MON810 determinations when analyses were performed on these two different lots of grain.  Some of the determinations made on the 0.4% lot would intersect at the low end of the variance associated with determinations made on the 0.8% lot; conversely, there would be some 0.8% quantifications that overlapped on the high end of the variance associated with determinations made on the 0.4% lot of grain.
  • The events with the consistently lowest relative errors (i.e., % relative errors ≤ 20%) at all fortification levels were: CBH 351, BT-11, TC1507 and NK603.  Yet, this was only observed on four of the nine transgenic events, it is apparent that there exists considerable difficulty with achieving quantifications that are accurate.
  • Another possible trend in this data set was the high number of “under-reported” quantifications or reported quantifications that were less than the fortification level as indicated by a (-) sign in front of the listed relative error values in Table Seven (e.g., MON810, GA-21, and NK603).

·                To compare the deviations between the reported quantifications from the known fortification levels we computed the “% relative error”.  It is a simple measure of “accuracy” (Table Seven).  If a given lab desired to know by how many “standard deviation units” their reported quantification differed from the collective group mean and standard deviation, they can compute their Z-score from the information given in Table Seven as follows:  Lab Reported Value – Listed Group Mean, then divide that result by the listed group’s “standard deviation” for the particular mean.  We advise the use of due caution in interpreting such Z-scores because we cannot definitively state that reported values were normally distributed—in some cases the group means were either positively or negatively skewed from gravimetric values.

 

We wish to know what types of between groups or within groups comparisons of performance and data analyses are of interest to our participants; please feel free to contact us as we prepare future reports.

 

Future Studies

In one study we are using the participating laboratory’s “continent of origin” as a blocking variable to see if there are differences in performance due to geographical location.  For this analysis we are compiling performance data that have been collected since the inception of the proficiency program (February 2002) so that a comprehensive and historical view can be developed.  These results will be submitted for publication in the near future.  In another study, we want to know if “tenure in the proficiency program” has any effect on the performance of testing labs. We are comparing the performance of veteran (experienced) labs with novice labs (first-time participants) for a given biotechnology-derived event and concentration group.

 

 

Note:  It is important to understand that there are no internationally recognized standard reference materials for all transgenic events.   The transgenic seed or grain used to prepare these samples was made available to GIPSA by the Life Science Organizations.  Care was taken to ensure the transgenic material was either essentially 100 % positive for the event, or adjusted accordingly.  The fortified samples were prepared using a process that has been verified to produce homogenous mixes, and representative samples were analyzed to ensure proper fortification and homogeneity.

 

To obtain additional information on the USDA/GIPSA Proficiency Program, contact Dr. Ron Jenkins, USDA/GIPSA Proficiency Program Manager, at US 816-891-04442, or by e-mail at biotech-lab@usda.gov.

 

 

 

 

List of Participants: October 2004

These laboratories declared that they wished to be cited as Participants in the Proficiency Program.  Other labs were in the program, but expressly wished their identity to remain confidential.

 

Dr. F. Bois

A. Bio. C – Molecular Biology Division

Route de Samadet

64410 ARZACQ

France

 

David Tomas

AINIA

Benjamin Franklin 5-11

Parque Tecnologico

46980 Paterna

Valencia

Spain

 

Imma Folch

Applus Agroalimentario

Crta. Cabrils, s/n

08348-CABRILS

Barcelona

Spain

 

 

Juan J. Giorda 

 

Bolsa de Comercio de Rosario

Córdoba 1402- 2oPiso

 

Rosario S2000AWV – Santa Fe

Argentina

 

 

 

Peter Brodmann

Biolytix AG

Benkenstrasse 254

CH-4108 Witterswil

Switzerland

 

Raquel Cuellar

BIOTOOLS B&M Labs

Valle de Tobalina

52 nave 43

28021 Madrid

Spain

 

Bureau of Food and Drug Analysis (BFDA), DOH, Taiwan

National Laboratory of Foods and Drugs, Department of Health, Taiwan

161-2, kuen Yang Street

Nankang

 

Taipei, Taiwan

 

Bureau of Quality and Safety of Food Department of Medical Sciences

88/7 Tiwanon Rd.

Amphur Muang

 

Nonthaburi 11000

 

Thailand

 

 

 

 

Parm Randhawa

California Seed and Plant Lab.

7877 Pleasant Grove Road

Elverta, CA  95626

 

 

Cheryl Dollard

Canadian Food Inspection Agency

Ottawa Lab Fallowfield - MATU

3851 Fallowfield Road

Ottawa, Ontario

K2H 8P9

Canada

 

 

Blanca Jauregui, Ph.D.

CNTA-Laboratorio del Ebro

Ctra N-134 km 50

31570 San Adrian

Navarra

Spain

 

 

Dr. Lutz Grohmann

CONGEN Biotechnology GmbH

Robert Roessle Str. 10

13125 Berlin, Germany

 

 

Apiwan Yoojinda

DNA Technology Laboratory

Kamphaengsaean, Nakorn Pathon 73140

Thailand

 

Marco Mazzara

European Commission - JRC

Via Fermi 1

I-21020 Ispra (VA)

Italy

 

Dr. Satoshi Futo

FASMAC CO., LTD

5-1-3 Midorigaoka, Atsugi-shi

Kanagawa 243-0041 

JAPAN

 

Dr. Antje Dietz-Pfeilstetter

Federal Biological Research Centre for Agriculture and Forestry

Institute for Plant Viology, Microbiology and Biosafety

Messeweg 11-12

D-38104 Braunschweig

Germany

 

Dr. Gabriele Muecher

GEN-IAL GmbH

Muelheimerstr. 26, Tor 3, Geb. 159

D-53840 Troisdorf

Germany

 

Dr. Castor Menendez

GeneScan Analytics GmbH, Freiburg

Engesserstr. 4

79108 Freiburg. Br.

Germany

 

Flavia Machado

GeneScan do Brasil Ltda

Avenida Antonia Gazzola, 1001

3 andar

13.301-245  ITU - SP - Brazil

 

Dr. Frank Spiegelhalter

GeneScan USA. Inc.

2315 N. Causeway Blvd.

Metairie, LA  70001

 

 

Jane Pappin/Bernd Schoel

Genetic ID NA

501 Dimick Drive

Fairfield, Iowa  52557

 

 

Mariangela Marudelli

Istituto di Botanica e Genetica vegetale

Facoltà di Agraria

Università Cattolica Sacro Cuore

Via Emilia parmense, 84

29100 Piacenza

 

 

Celine Beinchet

IdentiGen

Unit 9, Trinity Enterprise Center

Pearse Street

Dublin 2

Ireland

 

Dr. Reinhard Baier

JenaGen Diagnostik-Gentechnik-Biotechnologie

Loebstedter Str. 78

D-07749 Jena

Germany

 

Gerda Hempel          

Landesuntersuchungsanstalt fur das Gesundheits-Veterinarwesen Sachsen

Sitz Dresden

Amtliche Lebensmitteluberwachung

Fachgebiet 6.6

Jagerstrabe 10

D-01099 Dresden

Germany

 

Tamara Roustan, Ph.D.

LEM Laboratoires

Department Biologie moleculaire

38 rue de l’industrie

BP 70192

67405 Illkirch Cedex

France

 

Dr. Diana Hormisch

LUFA Speyer

Obere Langgasse 40

D-67346 Speyer

Germany

 

 

Dr. Brigitte Roth

LUFA Augustenberg

D 76227 Karlsruhe

Nesslerstr. 23

Germany

 

 

 

Filippo Odasso

Laboratorio CHMICO CCIAA TORINO

Via Vettimiglia 165

10127 Turin, Italy

 

Dr. Martino Barbanera/ Dr. Sonia Scaramagli

Laboratorio COOP ITALIA

Via del Lavoro 6/8

40033 Casalecchio di Reno

Bologna, Italy

 

John C. Jackson, Ph.D.

Monsanto

Q4A

800 N. Lindbergh Blvd.

St. Louis, MO 63167

 

 

Kalyn Brix-Davis

Mid-West Seed Services

236 32nd Avenue

Brookings, SD  57006

 

Dr Gatti Marcello

NEOTRON

NEOTRON Spa Stradello Aggazzotti

104 Santa Maria di Mugnano Modena Italy 41010

 

 

Dr. Jana Zel

National Institute of Biology

Vecna pot 111

1000 Ljubljana

Slovenia

 

 

Ming Zhang

National Testing Center of GMO (Jinlin)

No.6 xixinghua street,

gongzhuling city, Jilin province, PR,

China. Post code:136100

 

Merike Kelve

National Institute of Chemical Physics and Biophysics

Laboratory of Molecular Genetics

Akadeemia tee 23,

Tallin 12618, Estonia

 

Ms Kumi Goto

Nippon Yuryo Kentei Kyokai Yokohama Laboratory

Bankokubashi Bldg 5-26-1

Kaigan-dori Naka-ku

231-0002,Yokohama

Japan

 

Dr. Farin Hajar

OMIC USA Inc

3344 NW Industrial Street

Portland, OR 97210

 

Dr. Beni Kaufman

Pioneer Hi-Bred

10700 Justin Drive

Urbandale, IA  50322

 

Dave Goins

Q Laboratories, Inc.

1400 Harrison Avenue

Cincinnati, Ohio  45214

 

 

 

 

 

 

 

Andrew P Tingey, PhD.

Reading Scientific Services Ltd.

The Lord Zuckerman Research Centre

Whiteknights

Reading RG66LA

United Kingdom

 

Dr. Carolina Fernandes Ribas

Superinspect Ltda.

Rua do Comercio, 83

11010-141 Centro

Santos - Sa~o Paulo

Brazil

 

 

Angela Pérez Pérez

Sistemas Genomicos S.L.

Valencia Technology Parck,

C/Benjamin Franklin Avenue, 12

E-46980 Paterna Valencia

Spain

 

PeO Gummeson/Anders Dahlqvist

ScanGene AB

P.O.Box 166

SE-230 53 Alnarp

Sweden

 

Maria Saldanha

SGS do Brasil Ltda.

Av. Vereador Alfredo das Neves, 480

Alemoa

11095-510

Santos-SP  Brazil

 

 

 

 

 

Dr. Daniel Wetsch

Silliker, Inc.

405 8th Ave SE

Cedar Rapids, IA  52401

 

Cristhiane Abegg Bothona

Syngenta Seeds Ltda

BR 452 Km 142

Uberlandia-MG

Brazil

38405-232

 

Annelis-Reanate Winterstein

Thuringer Landesamt fur Lebensmittelsicherheit und Verbraucherschutz

Sitz Jena

Amtliche Lebensmitteluberwachung

Nauburger Str. 96 b

D-07743 Jena

Germany

 

Dr. Dahlia Garwe

Tobacco Research Board

Kutsaga Station

Airport Ring Road

Box 1909

Harare

Zimbabw

 

Boyce Butler

Thionville Surveying Company

5440 Pepsi Street

Harahan, LA  70123

 

 

Dr. Daniela Contri

TECAM

Rua Fabia, 59

Sao Paulo – SP – CEP:  05051-030

Brazil

 

 

Marc Rindal

Environmental Protection Agency

701 Mapes Rd.

Ft. Mead, MD  20755

 

Dr. Juergen Schwendinger

Zentrales Institut des Sanitatsdienstes der Bundeswehr Munchen

Ingolstadter Landstrasse 102

85748 Garching-Hochbruck

Postfach 45 06 43

80906 Munchen

Germany