Foundation Medicine published a wonderful paper in Nature Biotechnology last month. Development and validation of a clinical cancer genomic profiling test based on massively parallel DNA sequencing describes their approach to personalised cancer genomics.
Foundation Medicines "Foundation 1" test allows NGS analysis of 287 genes from FFPE samples for SNPs, InDels, CNA's and fusions. They describe the design and testing of this panel along with results from over 2000 clinical samples, finding "clinically actionable alterations in 76% of tumours" 3x more than they say are found with current methods.
Is Sanger-seq is dead:
"To validate, or not to validate, that is the question": and Foundation Medicine did not add any Sanger validation to their paper. This is pretty much unheard of, nearly every Cancer NGS paper reports a couple of traces in a figure or at least the supplemental data. We’ve been so used to those four-colour traces showing tumour:normal, but it looks like the time is coming when we’ll say goodbye. There really is no prospect of being able to Sanger validate 4557 exon sequences (the number screened by Foundation1). It is pretty much impossible to validate every finding with Sanger, not enough time, money, DNA or inclination and as we get more and more used to the quality and sensitivity of NGS Sanger is going to disappear. Still 30 years is not bad!
"To validate, or not to validate, that is the question": and Foundation Medicine did not add any Sanger validation to their paper. This is pretty much unheard of, nearly every Cancer NGS paper reports a couple of traces in a figure or at least the supplemental data. We’ve been so used to those four-colour traces showing tumour:normal, but it looks like the time is coming when we’ll say goodbye. There really is no prospect of being able to Sanger validate 4557 exon sequences (the number screened by Foundation1). It is pretty much impossible to validate every finding with Sanger, not enough time, money, DNA or inclination and as we get more and more used to the quality and sensitivity of NGS Sanger is going to disappear. Still 30 years is not bad!
Testing the test: The paper used a set of reference samples created by mixing cell lines in multiple pools to produce SNP, CNA’s and InDel’s at known allele frequencies. These spanned a wide range (5-100%) but this is perhaps not wide enough for some studies. Tumour heterogeneity may well require the ability to characterise mutant allele frequencies to the sub-1% level.
Figure 2b shows the detection sensitivity as a function of sample median exon coverage. The text states they generated between 150-500x coverage but the graph shows data at only 50x with almost 90% detection sensitivity for MAF >10%. As sequencing will be a significant part of the test price might dropping coverage be appropriate in some instances?
The authors very clearly demonstrate the efforts they went to to evaluate test performance for base-substitutions, InDels and CNA's, with each case being separately discussed. This is important as the different somatic variations require quite different analysis approaches.
They also showed excellent correlation between Foundation 1 and results from 250 samples on Sequenom mass-spec genotyping; ERBB2 & AR amplification and PTEN loss by FISH analysis; and CCND1 amplification by IHC. Test reproducibility was measured using 5 replicates from each of 6 FFPE samples across multiple tests and longitudinal reproducibility with 2 FFPE colon cancer samples, concordance was over 97% - demonstrating how NGS can be as accurate as standard assays being used in clinical labs. There is perhaps an unanswered question about whether FFPE artifacts prevent measuring below a 5% MAF, it will be interesting to see how much effort is placed on increasing detection limits over the next few years; is MAF >1% needed?
How much sequencing do we need for clinical NGS: The paper reported that 500x coverage of non-PCR duplicate read-pairs was required, yet the 2000 samples were sequenced to an average of 1134x. This is 60% more data than required; does that matter?
t might if we can do something positive with the extra data, e.g. sequence more patients, or more samples from the same patient. The authors make an important point when discussing clinically actionable alterations: that matched normal samples are not routinely collected. This limited their analysis to a focus on regions known to be involved in tumourigenesis. The impact that a matched normal can have on reducing the number of false positive somatic mutations surely makes collection and analysis worthwhile? And the additional costs associated with this need not be so incredibly high, in fact it may make sense to add duplicate library-prep and sequencing (at lower coverage) to reduce false positives to as low as possible. At the moment this somewhat depends on the depth of sequencing, but as these costs drop we'll need to focus more on the costs of library generation.
What size should a cancer panel test be: The authors report on the usefulness of their broad-spectrum approach to cancer testing citing ERBB2 as an example. They demonstrate that ERBB2 is altered in 12 cancer types where ERBB2 drugs are not currently prescribed. 5% of patients tested had a mutation in ERBB2 that may be treatable, and 40% of these patients had mutations that would be missed by current ERBB2 screening. It remains to be seen if these patient groups would benefit form treatment but without a broad-spectrum test they will be missed. This is surely more evidence that a larger test is better in most cases. Again the simple costs of collecting the sample, processing, analysing and reporting are reasonably fixed. It costs about the same to analyse 50 genes as 500!
What about ctDNA: Those of you who've read this blog over the past couple of years will have seen posts on circulating DNA analysis. It is the work I've been most excited about and involved in recently. The authors did not test any ctDNA samples so it remains unanswered as to whether Foundation 1 will work on ctDNA or not. However ctDNA and FFPE share many similarities; limited availability of amplifiable genome copies and short fragments so it should be possible.
If the authors are reading this then who do I talk to about trying it!
How much did it cost and will they make any money: Foundation Medicine's revenues were reported by GenomeWeb at around $4 million in the third quarter of 2012, which at a "negotiated price per test" of about $3,500 could be as many as 1150 paid for tests. This is only half those mentioned on the GenomeWeb coverage of their announcements A simple calculation is never likely to give us much of an insight into demand for Foundation 1, but I can see demand for tests like it being very very high indeed.
How much sequencing do we need for clinical NGS: The paper reported that 500x coverage of non-PCR duplicate read-pairs was required, yet the 2000 samples were sequenced to an average of 1134x. This is 60% more data than required; does that matter?
t might if we can do something positive with the extra data, e.g. sequence more patients, or more samples from the same patient. The authors make an important point when discussing clinically actionable alterations: that matched normal samples are not routinely collected. This limited their analysis to a focus on regions known to be involved in tumourigenesis. The impact that a matched normal can have on reducing the number of false positive somatic mutations surely makes collection and analysis worthwhile? And the additional costs associated with this need not be so incredibly high, in fact it may make sense to add duplicate library-prep and sequencing (at lower coverage) to reduce false positives to as low as possible. At the moment this somewhat depends on the depth of sequencing, but as these costs drop we'll need to focus more on the costs of library generation.
What size should a cancer panel test be: The authors report on the usefulness of their broad-spectrum approach to cancer testing citing ERBB2 as an example. They demonstrate that ERBB2 is altered in 12 cancer types where ERBB2 drugs are not currently prescribed. 5% of patients tested had a mutation in ERBB2 that may be treatable, and 40% of these patients had mutations that would be missed by current ERBB2 screening. It remains to be seen if these patient groups would benefit form treatment but without a broad-spectrum test they will be missed. This is surely more evidence that a larger test is better in most cases. Again the simple costs of collecting the sample, processing, analysing and reporting are reasonably fixed. It costs about the same to analyse 50 genes as 500!
What about ctDNA: Those of you who've read this blog over the past couple of years will have seen posts on circulating DNA analysis. It is the work I've been most excited about and involved in recently. The authors did not test any ctDNA samples so it remains unanswered as to whether Foundation 1 will work on ctDNA or not. However ctDNA and FFPE share many similarities; limited availability of amplifiable genome copies and short fragments so it should be possible.
If the authors are reading this then who do I talk to about trying it!
How much did it cost and will they make any money: Foundation Medicine's revenues were reported by GenomeWeb at around $4 million in the third quarter of 2012, which at a "negotiated price per test" of about $3,500 could be as many as 1150 paid for tests. This is only half those mentioned on the GenomeWeb coverage of their announcements A simple calculation is never likely to give us much of an insight into demand for Foundation 1, but I can see demand for tests like it being very very high indeed.
Imagine 6 samples a year taken as part of a liquid biopsy ctDNA monitoring program for all Breast cancer patients. With ?,000 patients in the US that could translate to a $billion order! Even if the price drops from $3500 to just $350 per patient.
Cancer genome testing appears to be a market of almost limitless potential, as long as it makes a difference to patients as well healthcare payers (insurance companies or the NHS). We might need to rob Peter to pay Paul but if doing so keeps people out of hospital and alive longer then who's going to object?
Reference:
Frampton GM, Fichtenholtz A, Otto GA, Wang K, Downing SR, He J, Schnall-Levin M, White J, Sanford EM, An P, Sun J, Juhn F, Brennan K, Iwanik K, Maillet A, Buell J, White E, Zhao M, Balasubramanian S, Terzic S, Richards T, Banning V, Garcia L, Mahoney K, Zwirko Z, Donahue A, Beltran H, Mosquera JM, Rubin MA, Dogan S, Hedvat CV, Berger MF, Pusztai L, Lechner M, Boshoff C, Jarosz M, Vietz C, Parker A, Miller VA, Ross JS, Curran J, Cronin MT, Stephens PJ, Lipson D, & Yelensky R (2013). Development and validation of a clinical cancer genomic profiling test based on massively parallel DNA sequencing. Nature biotechnology, 31 (11), 1023-31 PMID: 24142049
Cancer genome testing appears to be a market of almost limitless potential, as long as it makes a difference to patients as well healthcare payers (insurance companies or the NHS). We might need to rob Peter to pay Paul but if doing so keeps people out of hospital and alive longer then who's going to object?
Reference:
Frampton GM, Fichtenholtz A, Otto GA, Wang K, Downing SR, He J, Schnall-Levin M, White J, Sanford EM, An P, Sun J, Juhn F, Brennan K, Iwanik K, Maillet A, Buell J, White E, Zhao M, Balasubramanian S, Terzic S, Richards T, Banning V, Garcia L, Mahoney K, Zwirko Z, Donahue A, Beltran H, Mosquera JM, Rubin MA, Dogan S, Hedvat CV, Berger MF, Pusztai L, Lechner M, Boshoff C, Jarosz M, Vietz C, Parker A, Miller VA, Ross JS, Curran J, Cronin MT, Stephens PJ, Lipson D, & Yelensky R (2013). Development and validation of a clinical cancer genomic profiling test based on massively parallel DNA sequencing. Nature biotechnology, 31 (11), 1023-31 PMID: 24142049
Did they talk about how to overcome the PCR GC bias in CNV calling without a matching normal?
ReplyDeleteThey use a process matched normal (diploid) control sample.
ReplyDeleteAlso, targets selected with DNA baits, not PCR amplicons.
ReplyDeleteThanks for Anonymous' reply. But according to their technical guide, they only take the tumor sample? So what do you mean by "matched normal control sample"?
ReplyDeletehttp://www.foundationone.com/about-foundationone/ONE-1-002-20130904_FoundationOne_Technical.pdf
A normal diploid DNA sample is processed in every batch to use as a control for CNA calling
ReplyDeleteTo address the question of high coverage: a 500x average coverage means that you will get regions with 500x coverage, regions with 2000x or higher coverage, but also regions with very low (0-50x) coverage. To get sufficient coverage in those "bad" regions, you need an overall high coverage. Also, higher coverage helps in finding CNVs and subclonal mutations.
ReplyDeleteAlso keep in mind that this is a diagnostic test! 90% sensitivity is not good enough, you'll want more! High coverage is definitely needed. We are using >500x coverage for our diagnostic cancer panel (and we do sequence tumor+normal tissue for even higher sensitivity)
Thanks Mr Menzel for his input. Are you in anyway related to Foundation Medicine?
ReplyDeleteDo you mean now FM is doing 1000x as oppose to the 250x mentioned in the outdated technical guide?