Skip to Main Content
Centers for Medicare & Medicaid Services

View Public Comments for Draft Guidance for the Public, Industry, and CMS Staff Coverage with Evidence Development in the context of coverage decisions

Hillner, Bruce
National Oncologic PET Registry

January 28, 2013

Louis B. Jacques, MD
Director, Coverage and Analysis Group
Centers for Medicare & Medicaid Services
7500 Security Blvd
Baltimore, MD 21244

Via Electronic Delivery

RE: Draft Guidance on Coverage with Evidence Development—Joint Comments of the World Molecular Imaging Society (WMIS) and the National Oncologic PET Registry (NOPR) Working Group

Dear Dr. Jacques:

The World Molecular Imaging Society (WMIS) and National Oncologic PET Registry (NOPR) Working Group are pleased to submit these joint comments on the Centers for Medicare & Medicaid Services (CMS) draft guidance on Coverage with Evidence Development (CED).[1]

WMIS is an international scientific and educational organization dedicated to the understanding of biology and medicine through multimodal in vivo imaging and the utilization of quantitative molecular imaging in patient care. The foremost and authoritative society in molecular imaging, WMIS was formed in 2011, as a result of the merger of two of the premier molecular imaging societies—the Academy of Molecular Imaging (AMI) and the Society for Molecular-Genetic Imaging.

The NOPR is sponsored by WMIS and managed by the American College of Radiology (ACR). The NOPR is one of the first CED projects undertaken by CMS and has been in operation since 2006. The NOPR Working Group’s original belief that data collection is an important tool for both CMS and Medicare beneficiaries has been reinforced by our seven years of experience in implementing, improving, and operating a large-scale CED study.

Statutory Authority for CED Informs Many Aspects of CED

CMS defines the statutory authority for CED by citing two different prongs of Social Security Act, Section 1862(a)(1), which enumerate non-covered services. Subsection (a)(1)(A) denies payment for services not reasonable and necessary in the diagnosis and treatment of disease, and subsection (a)(1)(E) denies payment for services not reasonable and necessary to further the purposes of the Agency for Healthcare Research and Quality (AHRQ). CMS states that it will only invoke CED when a service is not reasonable and necessary under subsection (a)(1)(A), such as when evidence is insufficient for “confident” conclusions. In order to make this determination, CMS must review the existing body of evidence for the service and find it faulty. CMS presumably will define how it has found the evidence to be insufficient, which will usually fall into one of four categories articulated in Section V.A of the Draft Guidance:

  1. The evidence is not relevant enough to health outcomes;
  2. The evidence is not generalizable enough to the Medicare population;
  3. The evidence base has evolved and no longer gives confidence in the benefits of the service; or
  4. The evidence is not generalizable enough to typical settings of Medicare beneficiaries.

We believe an effective CED policy should be structured around an evidence review that identifies a promising medical technology for Medicare beneficiaries, but concludes that there is an evidence gap. The CMS review should provide a clear description of the specific data CMS would like to see generated (subject to public comment). This approach will lead in a rational way to a CED-based study that provides sufficient data to address the evidence gap.

The nature of the individual technology may point to a registry as the ideal study, or may reveal an evidence gap that can only be resolved by a randomized controlled trial. Identifying the extent and precision of the evidence needed will also assist the stakeholder sponsors and CMS in developing the study design, projecting the necessary study size and the likely study duration, and establishing the appropriate period of follow-up required. Alternatively, depending upon the type of evidence that CMS seeks to generate, CMS could simply delineate an explicit set of study design elements: design, size, duration, follow-up, and primary end-point. In past instances of CED, CMS has often negotiated the details of the CED with the developer sponsor long after the release of the final decision memorandum.

Ending Data Collection Requirements

The NOPR is one of the first CED programs that successfully moved from data collection to a positive National Coverage Decision. Additionally, CMS has a pending National Coverage Analysis (NCA) to end the data collection requirements for the remaining FDG-PET oncologic indications.[2] To this end, NOPR is concerned that the process for ending data collection for a CED study is still not clearly articulated in the Draft Guidance. In the case of the CED for FDG-PET, it will have taken two NCAs and over three years to end the data collection requirements. Unlike NOPR, which was designed to be self-funded, many CED programs may lack sufficient resources to continue generating data for such an extended length of time. Moreover, there is a policy implementation question of whether a CED study should continue operating after it completes the collection of the data it was designed to collect.

The Draft Guidance’s “overarching principle” is that CED should end in one of three circumstances: 1) When no CED studies are approved within the requisite timeframe; 2) When approved CED studies are not completed within the requisite timeframe; or 3) When CED studies “are completed.” As we noted in our prior comments, our extensive experience in operating the NOPR registries under CED has convinced us of the importance of having in place a clear and defined process for terminating CED prior to the initiation of CED data collection. While the specifics will likely vary from question to question, articulating such a process in advance will expedite the transition without the need for a lengthy (and often nearly year-long) NCA process.

We thus encourage CMS to clearly delineate in the Final Guidance between procedures for ending CED because CED has failed (which would be the case under the first two conditions) and procedures for ending CED because CED has been successfully completed (which would be the case under the third condition). We believe that CMS has already begun to take valuable steps with regard to minimizing the possibility of “failed CED” by establishing timespan rules and requirements in recent NCDs incorporating CED. For example, qualifying CED trials may be required to launch within a stated period, and be completed within a stated period. We believe sponsors will be incented to finish trials within this period, because after this period CMS financial support via payment of the service covered under CED will end.

However, the third condition—in which CED data was successfully collected, analyzed and publically reported (thus completing the CED requirements) is where we believe that CMS needs to provide additional guidance and a clearer time frame and process pathway. We strongly encourage CMS to include in the Final Guidance a transition pathway from a successful CED program to expanded coverage without the typical 9-to-12 month NCA process. As the Draft Guidance acknowledges, at present the coverage status of a service under CED cannot be revised without a second NCA. This approach is lengthy, time consuming, and requires the dedication of significant resources for both the CED sponsors and CMS staff. Indeed, in practice, the formal NCA requirements can allow an unsuccessful CED project to continue operating longer than is warranted. Neither Medicare beneficiaries nor the taxpayer are well served by the current approach.

We thus encourage CMS to provide additional detail in the Final Guidance about how CMS will determine when a successful CED study is deemed “completed,” and to articulate the procedures by which administrators of approved CED studies can seek such a determination from CMS if and when necessary.

Preventing Coverage Gaps Following CED

CMS acknowledges a major beneficiary access issue is that “[u]nder CED, there is a potential period of noncoverage between the end of the study and the agency’s review of the scientific results.” We agree that the potential disruption of coverage that may follow the period of coverage via CED could have adverse consequences for Medicare beneficiaries. We urge CMS to include provisions in the Final Guidance that would mitigate this possibility.

This issue could arise in one of several ways. For instance, in some cases, such as registries, sponsors may enter real-time data as to accrual and metrics into the project database, enabling more rapid analysis of the resulting data. In other cases, interim abstract data or data from other unrelated trials (perhaps international ones) may appear and provide strong evidence that the service is medically valuable. In such instances, hard-wiring the end of coverage into an NCD involving CED and requiring nine months or more to allow coverage through a NCA would potentially harm Medicare beneficiaries.

    CMS states in the Draft Guidance that:

    "We may address the issue of ongoing coverage by working with investigators to develop integrated research strategies during the planning of CED studies. For example CED studies may be designed to accommodate the complementary roles of randomized controlled trials (RCTs) and practical observational studies to close outstanding evidence gaps and allow coverage after an RCT ends where appropriate. An interim analysis, based on pre-agreed public criteria, would serve to open or close enrollment in the follow up study."

We support the concept advanced in the Draft Guidance that, where this gap in coverage follows an RCT-based CED study, a gap in coverage could be bridged by combining such RCTs with “practical observational studies.” Where the CED study is of the registry type, CMS could work with sponsors to define a rational cut-point for data analysis and publication, but the registry would not need to end at that deadline. Both approaches would allow CMS “to close outstanding evidence gaps and [also] allow coverage after an RCT ends where appropriate.”

Evidentiary Criteria for CED

The Draft Guidance states that CED will be triggered by the discovery of evidence gaps that would otherwise bar full coverage for a service. High-quality CED study design will go a long way toward establishing when a particular service under review will be considered reasonable and necessary, both by defining at the outset how (and by how much) the existing evidence falls short, and by establishing CED trial designs, scope, and endpoints that will bridge the current evidence gap and provide the required evidence for coverage.

From our own experience in implementing CED with the NOPR, we believe it is essential that the Final Guidance provide additional detail on how CMS will—for each proposed CED study—individualize and suggest the primary endpoint of interest to CMS. For diagnostic and prognostic technologies in particular, we believe there is a need to identify acceptable intermediate endpoints of improved patient outcomes. For example, in November 2012 the Medical Imaging & Technology Alliance (MITA) convened a Positron Emission Tomography (PET) Endpoints Workshop that brought together stakeholders to discuss a framework for the coverage of emerging PET radiopharmaceuticals and procedures. The Workshop reached consensus that diagnostic endpoints should most commonly be assessed on the basis of evidence for impact on patient management via changes in treatment planning.

Broadening Coverage with Research

While Draft Guidance suggests using a combination of RCTs and practical observational studies to bridge potential gaps in ongoing coverage, we note that all CED programs to date have limited coverage to beneficiaries participating in a study. We believe that this limitation is not necessary in all instances, and encourage CMS to add language in the Final Guidance enabling broader access by Medicare beneficiaries to diagnostic technologies even during the period of the CED study. The concept has been referred to in the academic literature as “only with research,” in contrast to “only in research.”

At present, only those Medicare beneficiaries who are actually participating in the CED-sponsored registry or trial are entitled to coverage for their use of the CED technology. Yet in certain instances, such as a novel therapeutic where the endpoint of interest is survival and the condition of study has no current effective therapy, it may be appropriate for conditional and time-limited coverage to be made available to all Medicare beneficiaries while the technology is being evaluated under CED, regardless of whether those specific individuals are themselves participating in the registry or trial. This approach also addresses, in part, legitimate questions that have been raised about the ethics of CED.[3]

For example, an RCT (or a very complex registry) could be ongoing in a relatively small number of subjects, while all other subjects would be entered into a low-level registry that collects certain key patient attributes, accompanied by a plan to utilize claims data to examine outcomes at some later time. We believe such an inclusive policy would actually result in higher-quality CED data being collected in a subset of patients and/or facilities. Indeed, while there are always concerns for referral or spectrum biases, this approach could allow more in-depth data collection and sub-group analyses.

Most importantly, we believe the importance of ensuring high-quality data need not (and should not) have the consequence of denying the potential benefit of the CED technology during the CED process to Medicare beneficiaries who—through no fault of their own—do not have access to the CED registry or trial.


WMIS and the NOPR Working Group appreciate the opportunity to provide these joint comments on the Draft Guidance for CED. We appreciate the assistance and support that CMS has provided to the NOPR over the past seven years, and look forward to working with CMS to provide any additional information or insight that would be valuable in the decision making process about the future of CED.


Zaver Bhujwalla, Ph.D.
President, WMIS

Bruce E. Hillner, M.D.
Chair, NOPR Working Group

Barry A. Siegel, M.D.
Co-Chair, NOPR Working Group

[1] Draft Guidance for the Public, Industry, and CMS Staff — Coverage with Evidence Development in the Context of Coverage Decisions (Nov. 29, 2012), available at
[3] Relyea-Chew A. Ethical considerations in CMS’s coverage with evidence development. J Am Coll Radiol 2011; 8:838-41.