CMS conducts its measurement activities in a transparent manner. Information gathered through various methods informs HHS and CMS about future measurement needs. Stakeholders are persons or organizations with a vested interest or concern related to clinical quality measures. They include, but are not limited to, Persons & Family, Clinicians, Facilities, Measure Developers, and Others, such as state governments or electronic health record system implementers.
There are multiple ways to obtain information from stakeholders such as through informal conversations with patients, conducting focus groups, or by including them on the Technical Expert Panel. Measure developers should prepare a plan for how stakeholders’ input will be solicited, gathered, and meaningfully incorporated into measure development and maintenance processes.
Clicking on the links below will take you to further information regarding stakeholder engagement in each phase.
Conceptualization: The measure developer may contact and interview measure experts, subject matter experts (SMEs) (including vendors and electronic health record system implementers), relevant stakeholders, and other measure developers to identify any measures in use or in development that are relevant to the topic of interest or to offer suggestions regarding appropriate topics for measure development. These or other experts may also be used to provide information about feasibility, importance, usability, and face validity early on before actual measure development begins. A Technical Expert Panel (TEP) is formed to provide structured input into the suggested concepts and the summary reports of those meetings is posted for public comments.
Specification: Though it is advisable to obtain public comments at several points during measure development, a key time to get additional public comments include the measure specification drafting phase. Receiving public comments at this phase helps to identify potential issues early in the process thus preventing rework later. Comments received during the public comment period will be reviewed and taken into consideration by the measure developer, CMS, and the TEP and will often result in revisions to the measure specifications. During this phase, the TEP is very involved in reviewing the initial specifications and providing feedback to the developer in an iterative process.
Testing: Refinement of the measure after testing is complete includes looping back with stakeholders who provided comment on the measure specification phase and discussing the testing results with them. Stakeholders are a critical part of the refinement process, in particular, consultation with statisticians is integral to interpretation of the testing data and taking action on it in the future development of the measure. At this phase, the TEP is reviewing the testing plan and results from the alpha and beta testing. Feedback is provided to the developer in this iterative process.
Implementation: Measures under consideration for implementation are publicly submitted for comment either through the formal federal rulemaking process or through an ad hoc public comment process for measures that are not subject to rulemaking. Measure developers convene stakeholder meetings regarding the implementation of considered measures, and their questions about the measures are resolved iteratively as the measure remains under consideration. The measure implementation process is completely transparent and open to the public for comments and questions. During this phase, the Technical Expert Panel is available to support responses to questions for suggestions from the various stakeholders, the public, and the National Quality Forum (NQF) steering committee.
Maintenance: The annual update, comprehensive reevaluation, and ad hoc reviews all require solicitation of inputs via public comment. Results from, and progress on, each of these review processes are reported publicly. Stakeholders are engaged for comment and, in some cases, a formal Technical Expert Panel review such as during a comprehensive reevaluation.