To: Administrative File: CAG- 00036R
Intestinal/Multivisceral Transplants
From: Steve Phurrough, MD, MPA
Director, Coverage and Analysis Group
Marcel Salive, MD, MPH
Director, Division of Medical and Surgical Services
Susan Harrison, MPP
Lead Analyst, Division of Medical and Surgical Services
Joseph Chin, MD, MS
Lead Medical Officer, Division of Medical and Surgical Services
Beverly Lofton, MHA
Analyst, Division of Medical and Surgical Services
Subject: Proposed Decision Memorandum for Intestinal/Multivisceral Transplants
Date: February 10, 2006
I. Proposed Decision
CMS has been asked to reconsider our current requirements of an annual volume of 10 intestinal transplants per year with a 1-year actuarial survival of 65 percent as a condition of approval as an intestinal transplant facility. CMS proposes that the evidence is adequate to conclude that our current requirements are necessary to ensure that intestinal transplants are furnished in a manner that is reasonable and necessary for the treatment of disease. Thus, we propose no change to current policy on intestinal/multivisceral transplant facility requirements.
Additionally, we propose to delete the following reference to a defunct website from our manual: More specific criteria can be found at http://cms.hhs.gov/providers/transplant/default.asp.
We are requesting public comment on this proposed decision.
II. Background
Since the early 1990s, intestinal (small bowel) transplantation has been considered a therapeutic option for patients with intestinal failure who have also failed total parenteral nutrition (TPN). In recent years, there have been a number of advancements in immunosuppressive medication, patient selection and surgical technique. With these advances, the number of patients receiving intestinal transplantations has also increased, albeit slowly, since intestinal transplantation remains challenging due in part to the immunogenicity of the intestine and to bacterial colonization. The immunogenicity of the small bowel, that is, the ability of this particular organ to elicit an immune response in the recipient, has a direct effect on the need for immunosuppressive medications and organ rejection.
In 2000, CMS provided coverage for intestinal and multivisceral (small intestine with other visceral organs such as stomach, pancreas, liver, spleen, and others) transplantation with conditions as follows: Medicare will cover intestinal transplantation for the purpose of restoring intestinal function in patients with irreversible intestinal failure only when performed for patients who have failed TPN and only when performed in centers that meet approval criteria. The criteria for approval of centers will be based on an annual volume of 10 intestinal transplants per year with a 1-year actuarial survival of 65 percent.
The 2000 decision was based on an analysis of the relevant clinical literature including 11 distinct studies submitted by the requestor, the Thomas E. Starzl Transplantation Institute. CMS (then HCFA) also analyzed a July 1999 technology assessment performed by Blue Cross Blue Shield, and an April 2000 technology assessment performed by Agency for Healthcare Research and Quality (AHRQ). Of the 11 distinct studies, one by Grant, et al. (1999) reported that programs that have performed at least 10 transplants had significantly higher graft survival rates. The 2000 Decision Memorandum notes two studies that support linking annual volume levels of other types of high risk surgical procedures to successful outcomes (Hosenpud et al., 1994, and Edwards et al., 1999). This was the rationale for Medicare's implementation of an annual volume criterion.
In 2005, CMS was asked by faculty of the University of Wisconsin Medical School to reconsider the Agency's position on volume of transplants and allow 10 cumulative transplants over any time frame as a condition of approval. Thus, this determination will focus on the approval criteria and not on the indications for intestinal transplantation.
III. History of Medicare Coverage
CMS has determined that intestinal/multivisceral transplantation falls within the benefit categories of inpatient hospital services and physicians' services.
On April 1, 2001, Medicare covered intestinal transplantation for the purpose of restoring intestinal function in patients with irreversible intestinal failure.
NCD Manual 100-3 Section 260.5
Approved Transplant Facilities
Intestinal transplantation is covered by Medicare if performed in an approved facility. The criteria for approval of centers will be based on a volume of 10 intestinal transplants per year with a 1-year actuarial survival of 65 percent using the Kaplan-Meier technique. More specific criteria can be found at: http://www.cms.hhs.gov/providers/transplant/default.asp.
IV. Timeline of Recent Activities
On August 11, 2005, CMS opened an NCD to reconsider the approval criteria for intestinal and multivisceral transplant centers.
On September 11, 2005, initial public comments were posted to the tracking sheet available at: http://www.cms.hhs.gov/mcd/viewpubliccomments.asp?nca_id=168#0811200509112005
V. FDA Status
Not applicable.
VI. General Methodological Principles
When making national coverage determinations, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding that an item or service falling within a benefit category is reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member. The critical appraisal of the evidence enables us to determine to what degree we are confident that: 1) the specific assessment questions can be answered conclusively; and 2) the intervention will improve net health outcomes for patients. An improved net health outcome is one of several considerations in determining whether an item or service is reasonable and necessary.
A detailed account of the methodological principles of study design that the agency utilizes to assess the relevant literature on a therapeutic or diagnostic item or service for specific conditions can be found in Appendix A. In general, features of clinical studies that improve quality and decrease bias include the selection of a clinically relevant cohort, the consistent use of a single good reference standard, and the blinding of readers of the index test, and reference test results.
Public comment sometimes cites the published clinical evidence and gives CMS useful information. Public comments that give information on unpublished evidence such as the results of individual practitioners or patients are less rigorous and therefore less useful for making a coverage determination. CMS uses the initial public comments to inform its proposed decision. CMS responds in detail to the public comments on a proposed decision when issuing the final decision memorandum.
VII. Evidence
A. Introduction
In this reconsideration, which is focused on the annual volume criterion, we considered studies and evidence that were published after the prior decision in 2000. Health outcomes of interest to CMS include mortality, graft survival, rejection and infections. This National Coverage Analysis (NCA) focuses on the following question: "What is the evidence on net health benefits for intestinal transplantation provided by facilities that performed 10 procedures annually compared to centers that performed 10 cumulative transplants?"
B. Discussion of evidence reviewed
1. Literature Search
CMS searched PubMed (2000 to present) for publications of randomized clinical trials (RCTs), observational studies and reviews on intestinal transplantation. General keywords included intestine and transplant. Studies must have presented original data, examined primary health outcomes and been published in peer-reviewed English language journals. Abstracts were excluded.
2. External technology assessments and clinical reviews
In 2005, CIGNA issued a coverage position that did not include a volume requirement.1 It was noted that: “The size of the center, the number of procedures performed and the patient status prior to transplant were all significantly associated with patient survival.”2 This was based upon data from the International Intestinal Transplant Registry.
In 2005, Middleton and Jamieson published a report on the current status of small bowel transplantation. The authors stated: “Graft survival is also better in those who have received antibody induction therapy with lymphocyte depleting or anti-IL-2 receptor antibody and when the center has had experience of more than nine procedures.”3 This was based upon data from the International Intestinal Transplant Registry. Center experience referred to total cumulative experience.
In 2004, Moon and Tzakis published a review of intestinal and multivisceral transplantation. They reported: “Most recent International Intestinal Transplant Registry (ITR) was presented at VIII International small bowel transplant symposium in September 2003. There were 61 centers (23 U.S. facilities) from 19 countries reporting. One year graft and patient survival for the recipients of transplants after 2001 were ranged 60-70% and 60-80% depending on the types of grafts. Data from individual centers, which performed more than 100 intestinal and/or multivisceral transplantations, showed better survival than ITR data. One year graft and patient survival reached up to 80% and 70% respectively.”4
In 2004, Organ Procurement and Transplantation Network (OPTN) and Scientific Registry of Transplant Recipients (SRTR) published their annual report on organ donation and transplantation in the United States. For intestinal transplantation, the report stated: “Intestine transplantation continues to be performed with modest but increasing frequency, with 112 transplants (including multi-organ procedures) performed in 2003. While total registrations have increased annually nearly every year since 1994, the increase in deceased donor organs has been minimal. In contrast to prior trends with liver and kidney transplantation, live donor intestine transplantation has not become a viable option for the vast majority of cases, with only four living donors registered for calendar year 2003. This is likely related to the majority of registrants being very small children, for whom living donation has not been offered for technical reasons.” For patient survival, the report stated: “Adjusted patient survival rates after intestine transplantation were 89% at three months, 77% at one year, 61% at three years, and 49% at five years. The younger pediatric patients <1 year and 1-5 years old, as well as the older adult patients 35-49 and 50-64 years old, had poorer survival rates than other groups. The best survival rate was seen for the 6-10 year old group. No differences were seen by gender. As with graft survival, patient survival was highest among those with a diagnosis other than short gut syndrome or functional bowel disorders and African Americans demonstrated slightly lower survival rates at all time points after transplantation than white recipients. Unadjusted analyses demonstrated similar differences in survival by age at transplant, but no differences according to gender, blood type, ethnicity, or residency were seen where adequate data was available for analysis. African American patients were again shown to have inferior survival rates at all time points. Patient survival at three months was not affected as was graft survival by severity of recipient illness at the time of transplant. Patients with a primary diagnosis of ‘other’ had improved short- and long-term survival compared to patients with short gut or functional bowel problems. Centers performing between two and seven transplants per year had better outcomes than those performing eight or more. This suggests that intermediate-size centers can achieve comparable patient and graft survival rates, and that volume criteria currently required for Medicare approval might need to be reevaluated. Adult donors between 18 and 34 years old were associated with the best survival rates while the age of pediatric donors had no influence on outcome.”5
In 2004, Langnas published a review of small intestinal transplantation. He noted that: “According to UNOS (United Network of Organ Sharing) 2002 data, patient survival rates at 3 months, 1 year, 3 years, and 5 years after intestine transplantation were 85%, 74%, 59%, and 50%.”6 The author did not report information on center volume.
In 2003, the American Gastroenterological Association published a technical review on intestinal transplantation. The Association noted: “Patient (P= 0.001) and graft (P= 0.002) survival at 1 and 5 years were superior at centers that had performed >10 transplants in the ITR data, although this was not supported by the UNOS registry data.”7 This statement was based upon the analysis by Grant in 1999.8
3. Internal technology assessment
We retrieved seven articles that met our search criteria. We also reviewed the original article by Grant in 1999 since it was referenced in several reports.
In 2005, Grant and colleagues published the results of an analysis of the Intestine Transplant Registry, “to determine the scope and success of intestine transplantation in the current era.”9 The authors sent report forms to “all known intestinal transplant programs asking for information on intestine transplants performed between April 1985 and May 31, 2003.”10 The authors noted that 61 programs provided data on 923 patients in 19 countries. Of these, 28 centers had performed a procedure within the past 2 years. Kaplan-Meier survival analysis was used. For transplants from 1998, the 1-year graft and patient survival rates were 65% and 77%, respectively, for intestinal transplantations; and 61% and 66% for multivisceral grafts. The authors reported: “Factors associated with improved patient survival in the log-logistic model included transplantation of patients waiting at home compared with patients waiting in hospital (75% versus 60% at 1 year; AF, 0.39; CI, 0.23-0.67; P<0.001); centers that had performed > 10 transplants compared with those that had performed = 11 The actual number of centers that performed more than 10 procedures was not presented. It would be important to create subgroups of relatively equal size. There was no mention of data verification or confirmation of information.
In 2005, Hanto and colleagues reported the results of an analysis of data from the national Organ Procurement and Transplant Network and the Scientific Registry of Transplant Recipients (SRTR) to Scientific Registry of Transplant Recipients (SRTR). This report was an excerpt from the OPTN/SRTR 2004 Annual Report (Chapter VII) noted above.
In 2005, Tzakis and colleagues reported the results of 98 multivisceral transplantations, including liver and intestine transplantations, “to summarize the evolution of multivisceral transplantation over a decade of experience and evaluate its current status.”12 The authors reported: “Kaplan Meier estimated patient and graft survivals for all cases were 65% and 63% at 1 year.”13 These outcomes were comparable to reported outcomes from the ITR.
In 2003, Middleton and colleagues reported the results of a case series of 14 small intestinal transplantations between 1991 and 1999. Thirty-six patients were evaluated for transplantations but many (13) of these patients were found to be maintainable on home parenteral nutrition and were alive at the end of the study period. The authors reported: “Twelve patients survived the transplantation procedure. Of these, seven patients were alive at 1 year, five at 3 years and three at 5 years. Three patients remain alive. Patient and graft survival improved with experience; the 1-year survival rate improved in the last 4 years of this experience from 43 to 57 per cent, and the 3-year survival rate from 29 to 43 per cent.”14
In 2002, Nishida and colleagues reported the results of 95 intestinal transplants performed at the University of Miami. Of the 95 cases, 54 were children and 41 were adults. There were 49 males and 46 females. The authors reported: “Since 1998, the 1-year patient and graft survival rates for isolated intestinal transplants have been 84% and 72%, respectively.”15 In this case series, Kaplan-Meier survival analysis was used. The results reported were better than the results reported from the International Intestinal Transplant Registry.
In 2001, Abu-Elmagd and colleagues reported the results of 155 intestinal and multivisceral transplants over an 11-year period, “to assess the long-term efficacy of intestinal transplantation under tacrolimus-based immunosuppression and the therapeutic benefit of newly developed adjunct immunosuppressants and management strategies.”16 The authors reported: “The actuarial survival rate for the total population was 75% at 1 year, 54% at 5 years, and 42% at 10 years. Recipients of liver plus intestine had the best long-term prognosis and the lowest risk of graft loss from rejection (P = .001). Since 1994, survival rates have improved.”17
In 1999, Grant reported the results of analyses of the Intestinal Transplant Registry. Small bowel with or without colon, intestine and liver, and multivisceral transplants from 1985 to February 1997 were included. There were 273 transplants in 260 patients. The author reported: “The 1-year graft/patient survival for transplants performed after February 1995 was 55%/69% for intestinal grafts; 63%/66% for small bowel and liver grafts; and 63%/63% for multivisceral grafts. Transplants since 1991 and programs that had performed at least 10 transplants had significantly higher graft survival rates.”18 For these analyses, the Kaplan-Meier survival method was used. There was no mention of multivariate analyses to control for confounding variables. The method to determine the 10 transplant categorization was not noted. The number of centers or programs that performed <=10 or > 10 transplants were not reported. It would be important to understand the scientific rationale of this grouping and to see that the subgroups had fairly equal numbers for statistical purposes.
4. MCAC
Not applicable.
5. Guidelines
Not applicable
6. Professional Society Position Statements
Not applicable.
7. Expert Opinion
Not applicable.
8. Public Comments
During the initial 30-day public comment period, CMS received two comments. One commenter advocates for the revision of CMS’ current policy on intestinal and multivisceral transplantation. The other commenter is against revision of the policy.
The first commenter, the University of Wisconsin Hospitals and Clinics, suggests that the criteria for approval should permit centers that have performed 10 cumulative intestinal transplants with 1-year patient survival of 65 percent (using the Kaplan-Meier technique) to be approved for Medicare coverage for intestinal and multivisceral transplantation. This commenter proposes that the revised criteria for approval would be consistent with CMS’ commitment in rulemaking to “focus on an organ transplant center’s ability to perform successful transplants and deliver quality patient care as evidenced by good outcomes and sound procedures” (70 Fed.Reg.15264). The commenter states that since only seven centers are approved by Medicare, many patients still must travel significant distances to receive an intestinal transplant.19
The second commenter notes that the most recent data from the Intestinal Transplant Registry continues to show the negative impact of center size (1-10 cases) on patient and graft survival (exhibit A-E). The commenter states that such a significant effect persists despite limiting the data analysis to the most recent cases that have been performed during 2003-2005 (exhibit E). The commenter suggests that a cumulative number of 10 is unequivocally associated with a significantly lower patient and graft survival. Additionally, the commenter suggests that to change the current CMS approval criteria, specifically the number of transplants, further studies would be required to determine the appropriate number of transplants for center approval. The commenter states that one of his main concerns with revising the current policy is the potential failure to deliver the optimal surgical and medical care needed for a relatively small group of patients with high disease gravity. He also adds, “The procedure in its three different prototypes is very challenging and demands a cumulative experience with high surgical skills.”
VIII. CMS Analysis
National coverage determinations (NCDs) are determinations by the Secretary with respect to whether or not a particular item or service is covered nationally under title XVIII of the Social Security Act § 1869(f)(1)(B). In order to be covered by Medicare, an item or service must fall within one or more benefit categories contained within Part A or Part B, and must not be otherwise excluded from coverage. Moreover, with limited exceptions, the expenses incurred for items or services must be “reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member.” § 1862(a)(1)(A).
The current request for reconsideration is limited. The requestor did not seek to eliminate the current coverage for intestinal and multivisceral organs. Rather, the reconsideration was limited to the standards for appropriate facilities in terms of the volume requirement and the actuarial survival percentage. In analyzing the evidence, CMS asked: “What is the evidence on net health benefits for intestinal transplantation provided by facilities that performed 10 procedures annually compared to centers that performed 10 cumulative transplants?”
As in many other types of surgery, center and surgeon experience are widely accepted factors to consider in patient outcomes.20 Although determining the appropriate volume criterion is difficult, CMS does believe that it is important to establish some volume criterion in light of the mortality and morbidity associated with intestinal transplantation. We believe that a rate or time constrained number of intestinal transplants would more accurately reflect the current state of practice at any given facility than a cumulative number where the transplants may have occurred in the past and are no longer actively performed. In addition, the use of a time-limited approach is widely accepted in physician certification and licensing.21
In our review, we did not find any studies on mortality or morbidity that directly compared centers based upon the number of procedures per year or in total. There were several case series, analyses of registry data and clinical reviews that provided indirect supporting evidence for the current volume requirement of 10 transplants per year and a survival of 65% or greater. In 2005, Grant and colleagues reported an analysis of the Intestine Transplant Registry that included intestinal transplants between 1985 and 2003. The authors noted that center experience (centers that had performed 10 transplants compared with those that had performed <=10 cases in total) was associated with improved patient survival.22 This finding was similar to the earlier analysis by Grant in 1999, which was considered in developing the current coverage policy.23 Several reviews (CIGNA, Middletown, AGA) have referenced these analyses of the ITR data.
The 2004 OPTN/SRTR (OPTN/SRTR, Hanto) report mentioned that “centers performing between two and seven transplants per year had better outcomes than those performing eight or more.”24 The specific analysis to support this statement was not presented. Without specific details on how this result was obtained, it is difficult to give considerable weight to this evidence since it is contrary to the vast body of literature on the effect of surgical volume. On the other hand, three specific studies (Tzakis, Nishida, Abu-Elmagd) showed that centers with high volumes of close to or more than 100 total transplantations over several years had better survival rates than reported from the International Intestinal Transplant Registry.25,26,27 One study (Middleton) found 1-year survival rates lower than ITR reported rates in a low-volume center (14 total transplants from 1991-1999), conflicting directly with the results published by Grant and reiterating concerns about low-volume centers.
From the literature reviewed in this reconsideration, there is more evidence on survival at centers that have performed 100 or more cumulative intestinal and multivisceral transplantations than any other total volume value, as noted by Moon and Tzakis. These numbers were collected over several years and, on an annual basis, represent a volume of 10 or more cases per year, and supports retaining the volume requirement. With the development of newer medications and advances in surgical techniques since 1999, better outcomes have been achieved, as shown by the analyses of the ITR and OPTN/SRTR data. The improvements demonstrated in both patient and graft survival since our initial decision in 2000 also support maintaining the current volume requirements.
In the February 4, 2005, Federal Register, CMS proposed new conditions of participation for the approval and re-approval of Medicare transplant centers. When the regulation becomes final, CMS expects to rescind the sections of current transplant NCDs that address approval and re-approval criteria. Thus, there will no longer be NCD transplant center criteria for demonstrating that a center is capable of performing transplants in a reasonable and necessary manner. Transplant centers will then become approved for transplants through the certification process that will be outlined in the final rule. CMS hopes to publish the final rule in early 2007.
In summary, we did not find the evidence reviewed since 2000 to be sufficiently compelling to change the annual volume or survival requirements. Thus, CMS will maintain the current coverage requirements.
Because the manual references a defunct website, for editorial reasons we propose to delete the following sentence from the NCD manual, “More specific criteria can be found at http://cms.hhs.gov/providers/transplant/default.asp.”
IX. Proposed Decision
CMS has been asked to reconsider our current requirements of an annual volume of 10 intestinal transplants per year with a 1-year actuarial survival of 65 percent as a condition of approval as an intestinal transplant facility. CMS proposes that the evidence is adequate to conclude that our current requirements are necessary to ensure that intestinal transplants are furnished in a manner that will be reasonable and necessary for the treatment of disease. Thus, we propose no change to current policy on intestinal/multivisceral transplant facility requirements.
Additionally, we propose to delete the following reference to a defunct website from our manual: More specific criteria can be found at http://cms.hhs.gov/providers/transplant/default.asp.
Appendix A: General Methodological Principles of Study Design
When making national coverage determinations, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding that an item or service falling within a benefit category is reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member. The critical appraisal of the evidence enables us to determine whether: 1) the specific assessment questions can be answered conclusively; and 2) the intervention will improve net health outcomes for patients. An improved net health outcome is one of several considerations in determining whether an item or service is reasonable and necessary.
CMS divides the assessment of clinical evidence into three stages: 1) the quality of the individual studies; 2) the relevance of findings from individual studies to the Medicare population; and 3) overarching conclusions that can be drawn from the body of the evidence on the direction and magnitude of the intervention’s risks and benefits.
The issues presented here represent a broad discussion of the issues we consider when reviewing clinical evidence. However, it should be noted that each coverage determination has unique methodological aspects.
1. Assessing Individual Studies
Methodologists have developed criteria to determine weaknesses and strengths of clinical research. Strength of evidence generally refers to: 1) the scientific validity underlying study findings regarding causal relationships between health care interventions and health outcomes; and 2) the reduction of bias. In general, some of the methodological attributes associated with stronger evidence include those listed below:
- Use of randomization (allocation of patients to either intervention or control group) in order to minimize bias.
- Use of contemporaneous control groups (rather than historical controls) in order to ensure comparability between the intervention and control groups.
- Prospective (rather than retrospective) studies to ensure a more thorough and systematical assessment of factors related to outcomes.
- Larger sample sizes in studies to help ensure adequate numbers of patients are enrolled to demonstrate both statistically significant as well as clinically significant outcomes that can be extrapolated to the Medicare population. Sample size should be large enough to make chance an unlikely explanation for what was found.
- Masking (blinding) to ensure patients and investigators do not know to which group patients were assigned (intervention or control). This is important especially in subjective outcomes, such as pain or quality of life, where enthusiasm and psychological factors may lead to an improved perceived outcome by either the patient or assessor.
Regardless of whether the design of a study is a randomized controlled trial, a non-randomized controlled trial, a cohort study or a case-control study, the primary criterion for methodological strength or quality is the extent to which differences between intervention and control groups can be attributed to the intervention studied. This is known as internal validity. Various types of bias can undermine internal validity. These include:
- Different characteristics between patients participating and those theoretically eligible for study but not participating (selection bias)
- Co-interventions or provision of care apart from the intervention under evaluation (confounding)
- Differential assessment of outcome (detection bias)
- Occurrence and reporting of patients who do not complete the study (attrition bias)
In principle, rankings of research design have been based on the ability of each study design category to minimize these biases. A randomized controlled trial minimizes systematic bias (in theory) by selecting a sample of participants from a particular population and allocating them randomly to the intervention and control groups. Thus, randomized controlled studies have been typically assigned the greatest strength, followed by non-randomized clinical trials and controlled observational studies. The following is a representative list of study designs (some of which have alternative names) ranked from most to least methodologically rigorous in their potential ability to minimize systematic bias:
- Randomized controlled trials
- Non-randomized controlled trials
- Prospective cohort studies
- Retrospective case control studies
- Cross-sectional studies
- Surveillance studies (e.g., using registries or surveys)
- Consecutive case series
- Single case reports
When there are merely associations but not causal relationships between a study’s variables and outcomes, it is important not to draw causal inferences. Confounding refers to independent variables that systematically vary with the causal variable. This distorts measurement of the outcome of interest because its effect size is mixed with the effects of other extraneous factors. For observational, and in some cases randomized controlled trials, the method in which confounding factors are handled (either through stratification or appropriate statistical modeling) are of particular concern. For example, in order to interpret and generalize conclusions to our population of Medicare patients, it may be necessary for studies to match or stratify their intervention and control groups by patient age or co-morbidities.
Methodological strength is, therefore, a multidimensional concept that relates to the design, implementation and analysis of a clinical study. In addition, thorough documentation of the conduct of the research, particularly study’s selection criteria, rate of attrition and process for data collection, is essential for CMS to adequately assess the evidence.
2. Generalizability of Clinical Evidence to the Medicare Population
The applicability of the results of a study to other populations, settings, treatment regimens, and outcomes assessed is known as external validity. Even well-designed and well-conducted trials may not supply the evidence needed if the results of a study are not applicable to the Medicare population. Evidence that provides accurate information about a population or setting not well represented in the Medicare program would be considered but would suffer from limited generalizability.
The extent to which the results of a trial are applicable to other circumstances is often a matter of judgment that depends on specific study characteristics, primarily the patient population studied (age, sex, severity of disease, and presence of co-morbidities) and the care setting (primary to tertiary level of care, as well as the experience and specialization of the care provider). Additional relevant variables are treatment regimens (dosage, timing, and route of administration), co-interventions or concomitant therapies, and type of outcome and length of follow-up.
The level of care and the experience of the providers in the study are other crucial elements in assessing a study’s external validity. Trial participants in an academic medical center may receive more or different attention than is typically available in non-tertiary settings. For example, an investigator’s lengthy and detailed explanations of the potential benefits of the intervention and/or the use of new equipment provided to the academic center by the study sponsor may raise doubts about the applicability of study findings to community practice.
Given the evidence available in the research literature, some degree of generalization about an intervention’s potential benefits and harms is invariably required in making coverage decisions for the Medicare population. Conditions that assist us in making reasonable generalizations are biologic plausibility, similarities between the populations studied and Medicare patients (age, sex, ethnicity and clinical presentation), and similarities of the intervention studied to those that would be routinely available in community practice.
A study’s selected outcomes are an important consideration in generalizing available clinical evidence to Medicare coverage determinations because one of the goals of our determination process is to assess net health outcomes. We are interested in the results of changed patient management not just altered management. These outcomes include resultant risks and benefits such as increased or decreased morbidity and mortality. In order to make this determination, it is often necessary to evaluate whether the strength of the evidence is adequate to draw conclusions about the direction and magnitude of each individual outcome relevant to the intervention under study. In addition, it is important that an intervention’s benefits are clinically significant and durable, rather than marginal or short-lived.
If key health outcomes have not been studied or the direction of clinical effect is inconclusive, we may also evaluate the strength and adequacy of indirect evidence linking intermediate or surrogate outcomes to our outcomes of interest.
3. Assessing the Relative Magnitude of Risks and Benefits
Generally, an intervention is not reasonable and necessary if its risks outweigh its benefits. Net health outcomes are one of several considerations in determining whether an item or service is reasonable and necessary. For most determinations, CMS evaluates whether reported benefits translate into improved net health outcomes. CMS places greater emphasis on health outcomes actually experienced by patients, such as quality of life, functional status, duration of disability, morbidity and mortality, and less emphasis on outcomes that patients do not directly experience, such as intermediate outcomes, surrogate outcomes, and laboratory or radiographic responses. The direction, magnitude, and consistency of the risks and benefits across studies are also important considerations. Based on the analysis of the strength of the evidence, CMS assesses the relative magnitude of an intervention or technology’s benefits and risk of harm to Medicare beneficiaries.
1 CIGNA 2005.
2 Ibid.
3 Middleton and Jamieson, 2005.
4 Moon and Tzakis, 2004.
5 OPTN/SRTR, 2004.
6 Langnas, 2004.
7 AGA, 2003.
8 Grant et al., 1999.
9 Grant et al., 2005.
10 Ibid.
11 Ibid.
12 Tzakis et al., 2005.
13 Ibid.
14 Middleton et al., 2003
15 Nishida et al., 2002.
16 Abu-Elmagd et al., 2002.
17 Ibid.
18 Grant, 1999.
19 Our website identifies eight centers.
20 Edwards et al., 1999.
21 Steinbrook, 2005.
22 Grant et al., 2005.
23 Grant, 1999.
24 OPTN/SRTR, 2004.
25 Tzakis et al., 2005.
26 Nishida et al., 2002.
27 Abu-Elmadg et al., 2001.
