National Coverage Analysis (NCA) Decision Memo

Ultrasound Stimulation for Nonunion Fracture Healing

CAG-00022R

Expand All | Collapse All

Decision Summary

CMS has determined that the evidence is adequate to conclude that non-invasive ultrasound stimulation for the treatment of nonunion bone fractures prior to surgical intervention is reasonable and necessary.

CMS will amend the Medicare National Coverage Determinations Manual Section 150.2 to delete the statement, “Indications that the patient failed at least one surgical intervention for the treatment of the fracture.”

As with all items and services, CMS will perform post-coverage analysis of claims data to continue to examine the net health benefit of ultrasound stimulation for nonunion fractures without prior surgery.

Decision Memo

To:		Administrative File: (CAG-00022R)  
		Reconsideration of Ultrasound Stimulation for Nonunion Fracture Healing  
  
From:	Steve Phurrough, MD, MPA  
		Director  
		Coverage and Analysis Group  
  
		Marcel Salive, MD, MPH  
		Director  
		Division of Medical and Surgical Services  
  
		Shamiram Feinglass, MD, MPH  
		Medical Officer  
		Division of Items and Devices  
  
		Niccole Corbin, DPM  
		Health Policy Analyst  
		Division of Medical and Surgical Services  
  
Subject:		Coverage Decision Memorandum for Ultrasound Stimulation for Nonunion Fracture Healing  
  
Date:		April 27, 2005

I. Decision

CMS has determined that the evidence is adequate to conclude that non-invasive ultrasound stimulation for the treatment of nonunion bone fractures prior to surgical intervention is reasonable and necessary.

CMS will amend the Medicare National Coverage Determinations Manual Section 150.2 to delete the statement, “Indications that the patient failed at least one surgical intervention for the treatment of the fracture.”

As with all items and services, CMS will perform post-coverage analysis of claims data to continue to examine the net health benefit of ultrasound stimulation for nonunion fractures without prior surgery.

II. Background

On July 27, 2004, CMS began a national coverage determination (NCD) for reconsideration of the requirement that patients must have failed at least one surgical intervention before treatment of a nonunion bone fracture within ultrasound stimulation.

Normally, the body is able to effectively heal fractures with conservative medical treatment within 8 to 12 weeks. Like most biological processes, normal fracture healing is complex and involves the integrated action of numerous cells, genes, and extracellular matrix. Healing of a fracture is the re-uniting of disrupted bone caused by a break in continuity. The process can be divided into three main stages – inflammatory, reparative, and remodeling. These factors are documented in detail with the original decision memo on ultrasound stimulation for nonunion fractures. 1

Many factors are involved in the healing of a fracture. The severity of the fracture, its location, the degree of bone loss, the extent of immobilization, nature of the blood supply, the amount of soft tissue damage, the involvement of a tumor, the use of radiation therapy, or the presence of infection can all retard the successful union of a fracture. Health considerations such as diabetes, vascular insufficiency, osteoporosis, nutritional deficiency, anemia, hormonal deficiency, smoking, advanced age, or certain medications can also affect the quality and rate of fracture healing. Co-morbidities associated with aging and the aforementioned health considerations may predispose the Medicare population to delayed or impaired healing. Additionally, certain patients with chronic diseases may be ineligible for surgery due to the risk of complications.

Fractures that do not heal within usual time frames are known as delayed unions and nonunions. Nonunions occur when there is no indication of healing for at least 3 months.2 A nonunion is defined clinically as the point when bone healing is stopped and will not proceed without some type of intervention. A nonunion is clinically established when there is no visible progression of the healing process. A delayed union is defined when healing has not advanced at the “average” rate for the location and type of fracture.3 Treatment options for nonunion fractures range from non-surgical treatments to various surgical techniques. Non-surgical treatments include osteogenic stimulation (involving pulsed electromagnetic fields, capacitive coupling, direct electrical stimulation or ultrasound), immobilization, and/or casting. Surgical techniques may involve external fixation and internal fixation that includes pins, nails, screws, wires, intramedullary rods, compression plates, and/or bone grafts. Successful treatment of a nonunion often depends on appropriate reduction of the fracture, bone grafting if necessary, and stabilization (internal or external fixation).

Ultrasound stimulation treatments have been advocated for nonunion fractures. It is a non-invasive external treatment and may be used alone, in conjunction with surgical stabilization of the fracture, or after surgical failure.

Scientific literature provides evidence that both mechanical and electrical stimuli can send regulatory signals to the bone causing physical remodeling of tissue. Ultrasound stimulation is a form of mechanical energy that is transmitted into tissue as high frequency acoustical pressure waves. Specific physiological effects that have been attributed to ultrasound stimulation include increased signaling pathways in osteoblasts, increased release of growth factors, increased enzymatic activity, increased calcium absorption, increased blood flow to the fracture site and increased callus formation.

III. History of Medicare Coverage

In August 1996, the Technology Advisory Committee (TAC) reviewed available data relating to ultrasound treatment and found insufficient evidence for effectiveness in the Medicare population. Therefore, the Health Care Financing Administration (HCFA, now CMS) revised the Coverage Issues Manual section 35-48 to issue a national noncoverage policy of ultrasound for all indications.

In November 1998, HCFA reviewed ultrasound for the treatment of fresh fractures again and national noncoverage continued.

In July 2000, Medicare completed a review on ultrasound stimulation for nonunion fractures but not for fresh fracture or delayed unions. Following this review, CMS modified its policy to allow ultrasound stimulation for nonunion when the following criteria are met:

A minimum of two sets of radiographs obtained prior to starting treatment with the osteogenic stimulator, separated by a minimum of 90 days. Each radiograph must include multiple views of the fracture site accompanied with a written interpretation by a physician stating that there has been no clinically significant evidence of fracture healing between the two sets of radiographs. Also, indications that the patient failed at least one surgical intervention for the treatment of the fracture.4

On July 1, 2004, CMS received a formal request for reconsideration of the previous national coverage determination (NCD) on ultrasound treatment of nonunion fractures from Smith & Nephew, Inc., the manufacturer of an ultrasound bone healing system. The requestor asked specifically for reconsideration of the national coverage decision requiring a surgical intervention for the Medicare patient prior to utilization of the ultrasound bone stimulator.

Benefit Category Determination

For an item or service to be covered by the Medicare program, it must meet one of the statutorily defined benefit categories outlined in the Social Security Act. CMS’s Center for Medicare Management (CMM) has determined that the application of non-invasive ultrasound stimulation for the treatment of nonunion fractures is included in the following benefit category:

§1861 (n) Durable Medical Equipment (DME), 42 U.S.C § 1395x(n)

IV. Timeline of Recent Activities


Date Action
May 24, 2004Representatives from Smith & Nephew, Inc. met with CMS staff to discuss the process for coverage reconsideration.
July 27, 2004CMS accepted the formal request for national coverage reconsideration of ultrasound bone stimulation for the treatment of long bone fractures. CMS also began its standard, initial 30-day comment period on this NCD to obtain public and scientific input relevant to the issue under consideration.
Sept. 20, 2004CMS met with Dr. Mark Dollard, a practicing podiatrist, to discuss the biological effects of ultrasound stimulation on the bone healing process.
Sept. 20, 2004Initial public comments posted to CMS website at http://www.cms.hhs.gov/mcd/viewtrackingsheet.asp?id=135
January 27, 2005CMS posted the proposed Decision Memo for a 30-day public comment period to obtain public and professional remarks regarding the proposal to cover beneficiaries enrolled in comparative prospective clinical trials.
February 27, 2005The 30-day public comment period closed and all received comments were posted to the CMS website at http://www.cms.hhs.gov/mcd/viewpubliccomments.asp?nca_id=135.

V. Food and Drug Administration (FDA) Status

On February 22, 2000, FDA approved the premarket approval application (PMA) for the low intensity ultrasound fracture treatment system, Exogen 2000®,indicated for the non-invasive treatment of established nonunions, excluding skull and vertebra. In addition, the device is indicated foraccelerating the time to a healed fracture for fresh, closed, posteriorlydisplaced distal radius fractures and fresh, closed or grade 1 open tibial diaphysis in skeletally mature individuals when these fractures are orthopedically managed by closed reduction and cast immobilization. A nonunion is considered to be established when the fracture site shows no visibly progressive signs of healing.5

VI.     General Methodological Principles of Study Design

When making NCDs, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding that an item or service is reasonable and necessary. The critical appraisal of the evidence enables us to determine to what degree we are confident that: 1) the specific assessment questions can be answered conclusively; and 2) the intervention will improve net health outcomes for patients. Evidence may consist of external technology assessments, internal review of published studies, recommendations from the Medicare Coverage Advisory Committee (MCAC), evidence-based guidelines, professional society position statements, expert opinion, and public comments (as appropriate).

A detailed account of the methodological principles of study design the agency staff utilizes to assess the relevant literature on a therapeutic or diagnostic item or service for specific conditions follows the conclusion and references for this proposed decision memorandum (see Appendix B).

VII. Evidence

A. Introduction

We are providing a summary of the evidence we considered including evidence submitted through the public comment period. This summary represents the body of evidence describing use of ultrasound stimulation for nonunion fractures without prior surgical intervention. The health outcomes of interest to CMS include healing rate and proportion with complete fracture healing.

In analyzing the evidence, CMS focused upon the following question, “Is there adequate evidence to conclude that non-invasive ultrasound stimulation improves net health outcomes in the treatment of nonunion bone fractures without prior surgical intervention in the Medicare population?”

B. Discussion of Evidence Reviewed

The evidence reviewed includes CMS’s 1998 and 2000 ultrasound stimulation decision memoranda, CMS’s internal technology assessment of new or reconsidered evidence, as well as any professional society position statements and expert opinion.

1. Internal Technology Assessment

Literature Search

CMS extensively searched PubMed (1990 to present) for new randomized controlled trials (RCTs) and systemic reviews evaluating the use of non-invasive ultrasound stimulation for the treatment of nonunion fractures. The literature search was limited to the English language and specific to the human population. The search was limited to those articles published in the last 10 years, but included studies conducted in all countries, including the United States (see evidence tables in Appendix A). In addition, CMS reviewed all articles submitted by the requestor and incorporated studies previously reviewed in earlier decision memoranda if they included patients treated with ultrasound stimulation without prior surgical intervention.

New Evidence

Since the 2000 decision, only two new sources of evidence are available, a case series with 18 subjects and a meta-analysis (see Appendix A –Evidence tables). Both have subgroups of patients with nonunion fractures without prior surgery and both find healing improved with the use of self-administered ultrasound bone stimulation for 20 minutes per session.

Lerner, et al (2004), reported a small (n=18) case series of severe compound high-energy long bone fractures of from 1997-2001. Subjects were aged 19 through 63. Sixteen of 18 patients had surgical stabilization prior to ultrasound stimulation. Authors found that 16/18 nonunion fractures treated with ultrasound united within one year after starting treatment (median time 26 weeks, 89% heal rate). There were no side effects of the procedure reported. Authors recommend ultrasound as an “adjunct modality in the treatment of severe high-energy injuries.”

Busse, et al (2002) conducted a meta-analysis of ultrasound treatment and found three articles meeting their criteria with only one addressing nonunion fractures (Mayr 2000, see below). This one article was considered in the precious NCA. Time to fracture healing was shorter with ultrasound (64 days shorter on average). The authors conclude that ultrasound may be beneficial to fracture healing, and that further clinical trials are needed.

During the 30-day comment period, Smith and Nephew submitted FDA PMA study data. These data represent a pooling of all the individual studies submitted for the PMA. These data have a larger sample size (n=55) of nonunion fracture patients without prior surgery (see Appendix A). The heal rates for the nonunion fractures without prior surgery are similar to those with prior surgery.

Finally, CMS pooled the data of all individual studies considered and found that, of the 173 nonsurgical nonunion fractures that received ultrasound stimulation, 148 healed, for a heal rate of 85%. This heal rate is similar to that of the nonunion fractures with prior surgery that received ultrasound stimulation.

Review of Evidence Previously Considered

Three studies that include results of ultrasound treatment in patients with nonunion fractures without prior surgery were identified from the literature reviewed for the previous decision memoranda.

Mayr, et al (2000), in a retrospective case series from a registry, found 153 nonunion patients without surgery prior to ultrasound stimulation. These patients had a success rate of 86% (132/153) and an average heal time of 140 days after beginning ultrasound therapy. These results are similar to those nonunion cases with surgery prior to ultrasound stimulation (success rate 85%, average heal time of 169 days).

Nolte, et al (2001) examined 41 cases of nonunion fractures in Dutch patients 18-90 years old from 1995-1997. Of the 29/39 cases that completed the study, the average fracture age was 1.2 years. Only eight cases had no prior surgery. Of these eight, 7 cases healed following ultrasound and the average heal time was 157 days which is similar to those who had surgery (152 days). In this self-paired analysis of pooled data, heal rates for cases with prior surgery (86%) and no surgery (87.5%) were similar.

Gebauer et al (2000), reported a case series of 67 nonunion fracture patients, including only 10 non-surgical nonunion cases treated with ultrasound. Of these cases, 7 healed following ultrasound.

2. Medicare Coverage Advisory Committee (MCAC)

This issue was not referred to the MCAC.

3. Evidence-Based Guidelines

No evidence-based guidelines were identified.

4. Professional Society Position Statements

The American College of Foot & Ankle Surgeons (ACFAS), a professional society of nearly 6,000 board-certified foot and ankle surgeons, strongly opposed the CMS proposed decision to retain the failed prior surgery requirement currently in place for coverage of ultrasound stimulation on nonunion fractures. The College asserts that this would limit the physicians’ ability to prescribe FDA-approved bone stimulator products and thus severely limit patient care. Implementation of this proposed decision would deny Medicare beneficiaries access to a non-invasive FDA-approved therapy that their physicians may otherwise wish to prescribe.

The ACFAS expressed several areas of concern regarding the CMS proposed decision. The College stated that randomized controlled clinical trials in which patients are randomized to surgery or to ultrasound will be non-reflective of standard protocol and would not provide useful clinical information for decision making which would be highly inappropriate. Such a trial comparing ultrasound to surgery would also be unworkable because patients would drop out if randomized into a treatment group that is not desired.

The society also points out that the current clinical evidence for Exogen’s success in healing nonunions with or without prior surgery is more than adequate. They state that the FDA-reviewed data clearly demonstrates success healing non-unions in Medicare patients without prior surgery and cited a widely-published economic model indicating Exogen is the most cost-effective treatment option. The College believes strongly that the use of external bone stimulators can be effective without prior surgery and respectfully requests CMS remove the failed prior surgery requirement currently in place. CMS reviewed this evidence and has concluded that a clinical trial is not necessary and that ultrasound stimulation for the treatment of nonunion fractures without prior surgery is reasonable and necessary.

CMS received no additional statements from professional societies. However, the American Association of Orthopaedic Surgeons and American Academy of Orthopaedic Surgeons (AAOS) published a position statement entitled, “Crendentialing in the Use of Specialized Instrumentation on Orthopaedics.” The statement indicated that use of specialized instrumentation should be based on educational exposure to the instruments and an understanding of their indications and contraindications. It further suggested that clinical privileges should be based on a surgeon’s knowledge, skills and experience, and should be the responsibility of local hospitals. AAOS’s position statement is available electronically at http://www.aaos.org/wordhtml/papers/position/1105.htm

5. Expert Opinion

CMS received no expert opinions.

6. Public Comments

During the initial comment period, July 27, 2004 to August 27, 2004, CMS received 14 public comments supporting the expansion of coverage for ultrasound osteogenic stimulation in the treatment of nonunion fractures.

The 30-day public comment period following the proposed decision memo posting on January 27, 2005 resulted in a total of 127 comments from individual physicians and patients. While all individual public comments supported the expansion of coverage for ultrasound osteogenic stimulation to patients without a previous failed surgical intervention, no further published evidence was presented.

The public comments stressed various possibilities that ultrasound stimulation for nonunion fractures used prior to surgery may prevent future surgical interventions. Forty-three comments stated ultrasound may provide a non-surgical option for beneficiaries with risk factors (co-morbidities). Twenty-three comments suggest ultrasound treatment for nonunion fractures may provide shorter healing times and 31 public commentaries believe early ultrasound treatment may totally prevent the need for surgical intervention. There was no new peer-reviewed published evidence provided to support changing the national coverage policy.

CMS received 29 public comments focused on the anticipated CMS cost savings of treating with ultrasound stimulation prior to surgery. CMS’s decision to expand coverage to beneficiaries without prior surgery was done without the consideration of cost. When making national coverage decisions, CMS evaluates relevant published and peer-reviewed clinical evidence to determine whether or not the evidence is of adequate quality to support a finding of reasonable and necessary.

One public commenter stated it was unethical to withhold FDA approved treatments for fracture care. Even though we are not requiring a trial here, CMS believes it would be possible to design an ethical trial.

All public comments were taken into consideration during the NCD reconsideration process that concluded with an expansion of coverage.

VIII. CMS Analysis

National coverage determinations (NCDs) are determinations by the Secretary with respect to whether or not a particular item or service is covered nationally under title XVIII of the Social Security Act § 1869(f)(1)(B). In order to be covered by Medicare, an item or service must fall within one or more benefit categories contained within Part A or Part B, and must not be otherwise excluded from coverage. Moreover, with limited exceptions, the expenses incurred for items or services must be “reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member.” § 1862(a)(1)(A).

All of the studies consistently show that ultrasound stimulation treatment results in similar healing rates in two subgroups of nonunion patients, with and without previous surgery. Though the evidence was limited by the small sample sizes and the nature of the studies (case series), the addition of a new study with larger sample size submitted in the public comment period improved the quality of the evidence. Hence, the quality of the evidence was limited but adequate to evaluate the net health benefit of ultrasound stimulation treatment for nonunion fractures prior to surgical intervention.

One of the main reasons for the previous proposed non-coverage decision was the small sample size of the studies. However, upon further analysis and pooling of the data, we find evidence of a net health benefit from ultrasound stimulation for nonunion fractures in patients without previous surgical intervention. The most recently submitted pooled FDA PMA data and the data CMS pooled provided the evidence lacking in the proposed decision.

As found previously, this body of literature is limited by the use of retrospective case series, registry data, the use of overlapping cases, and small studies (as discussed in Appendix B). However, results from all studies were consistent in showing healing in non-union fractures without prior surgery.

One noticeable problem with the use of registry data is the overlap between studies, which weakens the strength of the overall body of evidence. Sequential reports from a given registry often analyze some of the same cases. For instance, two studies contain cases entered in the registry between October 17, 1994 and October 17, 1996. 6,7

One strength of the body of evidence is the applicability of the study procedures beyond the research setting. The majority of the patients in these studies were treated at home under the instruction of their own physician. Therefore, the studies conducted on the ultrasound treatment of nonunion are relevant to the everyday clinical setting. This strength was enhanced by the recently submitted pooled data.

The articles reviewed for this reconsideration are fairly consistent both in terms of design and outcome. All the studies pertaining to nonunion fractures use an outcome of healed versus failure to heal and they all utilize ultrasound treatment without concomitant treatment.

When reviewing the available studies, the conservatively treated patients (those without surgery) had a heal rate similar to those with prior surgery. Though the sample size of this subset was smaller, these results are consistent with those in the surgical cohort and pathophysiologically, there is no sound reason why the requirement of surgery would change the outcome in this subset of patients.

Though the limitations have not changed from the previous analysis in 2000, we find that the results from these articles, the addition of the pooled data analysis provided in the public comment period and one generated internally by CMS, and the subsets of patients who did not have surgery prior to nonunion and ultrasound are consistent. The evidence was adequate to remove the requirement of surgery prior to ultrasound stimulation for a nonunion fracture.

In conclusion, though evidence was adequate to show that ultrasound stimulation is effective in treating nonunion fractures, this is an area ripe for research.

A study comparing surgery to ultrasound stimulation for nonunion fractures would be appropriate. Other methods of analyzing data do exist, such as post-coverage analysis of claims data. As with all other services, CMS plans to conduct analyses of claims data to continue to evaluate the net health benefit of ultrasound stimulation for nonunion fracture without prior surgery.

Conclusion:

CMS has determined that the evidence is adequate to conclude that non-invasive ultrasound stimulation for the treatment of nonunion bone fractures prior to surgical intervention is reasonable and necessary.

CMS will amend the Medicare National Coverage Determinations Manual Section 150.2 to delete the statement, “Indications that the patient failed at least one surgical intervention for the treatment of the fracture.”

As with all items and services, CMS will perform post-coverage analysis of claims data to continue to examine the net health benefit of ultrasound stimulation for nonunion fractures without prior surgery.



APPENDIX A

Evidence tables from the previous decision memorandum published on July 31, 2000 (including evidence review from Gebauer et al, Mayr et al. and Nolte et al.) are available electronically at http://www.cms.hhs.gov/coverage/download/id76.pdf?orgin=globalsearch&page=/mcd/viewdecisionmemo.asp&id=76 [PDF, 129KB].

Evidence Tables for proposed decision:

Author/Year Study Design/Purpose Intervention/Outcomes Demographics Results

Lerner A, et al 2004

Case series

Effect of low energy ultrasound (US) in healing severe compound high-energy limb injuries; supplemental to surgical skeletal stabilization (“combined treatment”)

ultrasound stimulation for 20 minutes applied by patient at home

treatment continued until clinical and radiographic signs of solid bone union.

N=18 injuries in 17 patients.

Data from 1997-2001

14 male,3 female

age range: (19-63)
2 without prior surgical stabilization

severe compound high energy limb injuries, various modes of injury, various sites:femur, tibia, forearm and humerus 16 with surgical stabilization (wires/screws)

16/18 fractures united with solid bone union within 13-52 weeks (89%).

Conclude useful as adjunct modality in severe injuries.

Busse JW, et al, 2002

Meta-analysis

Evaluate effect of low intensity ultrasound (US) on time to fracture healing

2 independent searchers identified relevant RCTs from 5 major databases; 2 independent readers

Baseline characteristics: 20 minute sessions.

138 studies, 6 met inclusion criteria and reviewed, 3 met final analysis criteria.

Time to fracture heal: significantly shorter with ultrasound-64 days shorter on average.

Ultrasound may be beneficial to fracture healing

They call for further clinical trials

Smith & Nephew, 2004

FDA PMA Study data analysis of pooled data

ultrasound stimulation for 20 minutes applied by patient at home

378 patients with nonunion fractures

55 without prior surgery

Heal rates were similar for both groups

80.0% for no prior surgery

80.2% without prior surgery



APPENDIX B

General Methodological Principles

When making national coverage decisions, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding of reasonable and necessary. The evidence may consist of external technology assessments, internal review of published and unpublished studies, recommendations from the Medicare Coverage Advisory Committee, evidence-based guidelines, professional society position statements, expert opinion, and public comments.

The overall objective for the critical appraisal of the evidence is to determine to what degree we are confident that: 1) specific clinical questions relevant to the coverage request can be answered conclusively; and 2) the intervention will improve net health outcomes for patients.

We divide the assessment of clinical evidence into three stages: 1) the quality of the individual studies; 2) the relevance of findings from individual studies to the Medicare population; and 3) overarching conclusions that can be drawn from the body of the evidence on the direction and magnitude of the intervention’s risks and benefits.

The issues presented here represent a broad discussion of the issues we consider when reviewing clinical evidence. However, it should be noted that each coverage determination has unique methodological aspects.

1. Assessing Individual Studies

Methodologists have developed criteria to determine weaknesses and strengths of clinical research. Strength of evidence generally refers to: 1) the scientific validity underlying study findings regarding causal relationships between health care interventions and health outcomes; and 2) the reduction of bias. In general, some of the methodological attributes associated with stronger evidence include those listed below:

  • Use of randomization (allocation of patients to either intervention or control group) in order to minimize bias
  • Use of contemporaneous control groups (rather than historical controls) in order to ensure comparability between the intervention and control groups.
  • Prospective (rather than retrospective) studies to ensure a more thorough and systematical assessment of factors related to outcomes.
  • Larger sample sizes in studies to help ensure adequate numbers of patients are enrolled to demonstrate both statistically significant as well as clinically significant outcomes that can be extrapolated to the Medicare population. Sample size should be large enough to make chance an unlikely explanation for what was found.
  • Masking (blinding) to ensure patients and investigators do not know to which group patients were assigned (intervention or control). This is important especially in subjective outcomes, such as pain or quality of life, where enthusiasm and psychological factors may lead to an improved perceived outcome by either the patient or assessor.

Regardless of whether the design of a study is a randomized controlled trial, a non-randomized controlled trial, a cohort study or a case-control study, the primary criterion for methodological strength or quality is the extent to which differences between intervention and control groups can be attributed to the intervention studied. This is known as internal validity. Various types of bias can undermine internal validity. These include:

  • Different characteristics between patients participating and those theoretically eligible for study but not participating (selection bias)
  • Co-interventions or provision of care apart from the intervention under evaluation (confounding)
  • Differential assessment of outcome (detection bias)
  • Occurrence and reporting of patients who do not complete the study (attrition bias)

In principle, rankings of research design have been based on the ability of each study design category to minimize these biases. A randomized controlled trial minimizes systematic bias (in theory) by selecting a sample of participants from a particular population and allocating them randomly to the intervention and control groups. Thus, randomized controlled studies have been typically assigned the greatest strength, followed by non-randomized clinical trials and controlled observational studies. The following is a representative list of study designs (some of which have alternative names) ranked from most to least methodologically rigorous in their potential ability to minimize systematic bias:

  • Randomized controlled trials
  • Non-randomized controlled trials
  • Prospective cohort studies
  • Retrospective case control studies
  • Cross-sectional studies
  • Surveillance studies (e.g., using registries or surveys)
  • Consecutive case series
  • Single case reports

When there are merely associations but not causal relationships between a study’s variables and outcomes, it is important not to draw causal inferences. Confounding refers to independent variables that systematically vary with the causal variable. This distorts measurement of the outcome of interest because its effect size is mixed with the effects of other extraneous factors. For observational, and in some cases randomized controlled trials, the method in which confounding factors are handled (either through stratification or appropriate statistical modeling) are of particular concern. For example, in order to interpret and generalize conclusions to our population of Medicare patients, it may be necessary for studies to match or stratify their intervention and control groups by patient age or co-morbidities.

Methodological strength is, therefore, a multidimensional concept that relates to the design, implementation and analysis of a clinical study. In addition, thorough documentation of the conduct of the research, particularly study’s selection criteria, rate of attrition and process for data collection, is essential for CMS to adequately assess the evidence.

2. Generalizability of Clinical Evidence to the Medicare Population

The applicability of the results of a study to other populations, settings, treatment regimens and outcomes assessed is known as external validity. Even well-designed and well-conducted trials may not supply the evidence needed if the results of a study are not applicable to the Medicare population. Evidence that provides accurate information about a population or setting not well represented in the Medicare program would be considered but would suffer from limited generalizability.

The extent to which the results of a trial are applicable to other circumstances is often a matter of judgment that depends on specific study characteristics, primarily the patient population studied (age, sex, severity of disease and presence of co-morbidities) and the care setting (primary to tertiary level of care, as well as the experience and specialization of the care provider). Additional relevant variables are treatment regimens (dosage, timing and route of administration), co-interventions or concomitant therapies, and type of outcome and length of follow-up.

The level of care and the experience of the providers in the study are other crucial elements in assessing a study’s external validity. Trial participants in an academic medical center may receive more or different attention than is typically available in non-tertiary settings. For example, an investigator’s lengthy and detailed explanations of the potential benefits of the intervention and/or the use of new equipment provided to the academic center by the study sponsor may raise doubts about the applicability of study findings to community practice.

Given the evidence available in the research literature, some degree of generalization about an intervention’s potential benefits and harms is invariably required in making coverage decisions for the Medicare population. Conditions that assist us in making reasonable generalizations are biologic plausibility, similarities between the populations studied and Medicare patients (age, sex, ethnicity and clinical presentation) and similarities of the intervention studied to those that would be routinely available in community practice.

A study’s selected outcomes are an important consideration in generalizing available clinical evidence to Medicare coverage determinations. One of the goals of our determination process is to assess net health outcomes, and we are interested in the results of changed patient management not just altered management. These outcomes include resultant risks and benefits such as increased or decreased morbidity and mortality. In order to make this determination, it is often necessary to evaluate whether the strength of the evidence is adequate to draw conclusions about the direction and magnitude of each individual outcome relevant to the intervention under study. In addition, it is important that an intervention’s benefits are clinically significant and durable, rather than marginal or short-lived.

If key health outcomes have not been studied or the direction of clinical effect is inconclusive, we may also evaluate the strength and adequacy of indirect evidence linking intermediate or surrogate outcomes to our outcomes of interest.

3. Assessing the Relative Magnitude of Risks and Benefits

Generally, an intervention is not reasonable and necessary if its risks outweigh its benefits. Among other things, CMS evaluates whether reported benefits translate into improved net health outcomes. The direction, magnitude and consistency of the risks and benefits across studies are important considerations. Based on the analysis of the strength of the evidence, CMS assesses whether an intervention or technology's benefits to Medicare beneficiaries outweigh its harms.


1 Decision Memo on Ultrasound Stimulation for Nonunion Fractures (HCFA, 2000) accessed at http://cms.hhs.gov/mcd/viewdecisionmemo.asp?id=76.

2 AAOS Fact Sheet, retrieved on November 2, 2004 from http://orthoinfo.aaos.org/fact/thr_report.cfm?Thread_ID=386&topcategory=General%20Information

3 Decision Memo on Ultrasound Stimulation for Nonunion Fractures (HCFA, 2000) accessed at http://cms.hhs.gov/mcd/viewdecisionmemo.asp?id=76.

4 NCD Coverage Issues Manual section 150.2 can be accessed at http://cms.hhs.gov/Regulations-and-Guidance/Guidance/Manuals/06_cim/ci35.asp#_1_53.

5 FDA status accessed at http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMA/PMA.cfm?ID=5779

6 Heppenstall et al. Non-invasive nonunion treatment by pulsed low-intensity ultrasound. AAOS Annual Meeting.1999

7 Mayr et al. Ultrasound - an alternative healing method for nonunions? Arch Orthop Trama Surg. 2000; 120 (1):1-8.

Bibliography

Busse et al. The effect of low-intensity pulsed ultrasound therapy on time to fracture healing: a meta-analysis. CMAJ. Feb 2002; 166(4): 437-41.Duncan et al. Mechanotransduction and the functional response of bone to mechanical strain. Calcified Tissue Int. Nov. 1995; 57(5): 344-58.

Heppenstall et al. (abstract) Non-invasive nonunion treatment by pulsed low-intensity

ultrasound. AAOS Annual Meeting.1999.

Korstjens et al. Stimulation of bone cell differentiation by low-intensity ultrasound – a histomorphometric in vitro study. J Orthop Research. May 2004; 22(3): 495-500.

Lerner et al. Compound high-energy limb fractures with delayed union: our experience with adjuvant ultrasound stimulation (exogen). Ultrasonics. April 2004; 42(1-9): 915-7.

Mayr et al. Ultrasound - an alternative healing method for nonunions? Arch Orthop Trama Surg. 2000; 120 (1-2): 1-8.

Nolte et al. Low-intensity pulsed ultrasound in the treatment of nonunions. J Trauma. Oct. 2001; 51(4): 693-702.

Pilla AA. Low-intensity electromagnetic and mechanical modulation of bone growth and repair: are they equivalent? J Orthop Sci. 2002; 7(3): 420-8.