Maryland General
The GAO Report Recommmendations: The Regulators, Accreditors and Inspectors Respond
The final GAO report contained not only recommendations, but preliminary responses of those affected by the recommendations. See what CMS, CAP, JCAHO, and COLA had to say to the GAO when they saw the first draft of the report.
- Standardize survey findings.
- Ensure consistent advance notice by states
- Ensure identifying deficiencies comes before education
- Impose sanctions on labs with consecutive failures
- Require all survey organizations to require whistle-blower posters
- Require quarterly proficiency testing
- Ensure timely evaluation of survey organizations
- Ensure that new survey rules are reviewed in a timely manner
- CMS must hire more staff
- Perform more validation inspections of state survey organizations
- Require more independent validation of survey organizations
- Collect and review more survey data
- Establish an enforcement database
August 2006
with Sten Westgard, MS
The GAO report on CMS delivered not only 13 recommendations for improvement, but they also included the responses of the regulator and accrediting organizations that were affected by these recommendations. Some of their reponses are very interesting and we think these answers will probably predict what will actually happen to the GAO recommendations.
| GAO Report Recommendations |
Did they respond?
|
|||
| CMS |
CAP
|
JCAHO |
COLA
|
|
| Yes | Yes | Yes | ||
| 2. Ensure consistent advance notice by states | Yes | Yes | ||
| 3. Ensure identifying deficiencies comes before education | Yes | Yes | Yes | Yes |
| 4. Impose sanctions on labs with consecutive failures | Yes | Yes | ||
| 5. Require all survey organizations to require whistle-blower posters | Yes | Already done |
Yes | |
| 6. Require quarterly proficiency testing | Yes | |||
| 7. Ensure timely evaluation of survey organizations | Yes | |||
| 8. Ensure that new survey rules are reviewed in a timely manner | Yes | Yes | ||
| 9. CMS must hire more staff | Yes | |||
| 10. Perform more validation inspections of state survey organizations | Yes | |||
| 11. Require more independent validation of survey organizations | Yes | Yes | Yes | |
| 12. Collect and review more survey data | Yes | |||
| 13. Establish an enforcement database | Yes | Yes | ||
Since the GAO recommendations are leveled directly at CMS, the agency provided a full response. The other agencies were more selective in their responses.
Below we'll break down each recommendation separately, display the significant responses from CMS and the accreditors, and add a bit of commentary on the reponses themselves. If you want to find out more about why a recommendation was made, click the Why was this recommendation made? link
1. Work with exempt-state programs and accrediting organizations to standardize their categorization and reporting of survey findings in a way that tracks to CLIA inspection requirements and allows for meaningful comparisons across organizations, such as the analysis of trends in the citation of condition-level deficiencies.
Why was this recommendation made?
The CMS response
"We endorse this concept but will be cautious as to its scope. In our experience, a straightforward linkage of accrediting organization requirements to CLIA condition-level requirements is limited by our authority under the statute, and still may not make it fully possible to assess labs in a standardized manner." [page 70]
The JCAHO response
"...the GAO recommendation that CMS standardize the categorization and reporting of survey findings, while having the theoretical potential to simplify administrative oversight of the program, has several serious shortcomings. First, compliance with this GAO recommendation would require revamping of our entire accreditation system. GAO fails to recognize that the Joint Commission - like its colleague accrediting bodies - uses a different and more sophisticated approach to assess laboratory performance." [pages 89-90]
"[T]he Joint Commission believes that CMS could and should play a role in developing a common, agreed-upon taxonomy that could be used by all laboratory survey organizations to track serious deficiencies....If all survey organizations were to agree on criteria as to what constitutes a serious deficiency, this would creat the desired comparability without requiring accrediting organizations to change the ways in which they categorize and report findings. [page 90]
The COLA response
"...we caution that the GAO, Congress, and CMS respect that the approved accrediting organizations are all unique in approach and methodology. All survey organizations meet the CLIA required validation thresholds; all thoroughly investigate complaints, and all take immediate action when risk of harm situations is evident. In short, we oversee labs differently, and laboratory quality has improved. We do not agree that standardized surveys would be inherently more accurate or appropriate for the wide rage [sic.] of laboratories and laboratory environments currently overseen by CMS and the accrediting organizations."
Our Take
CMS takes a legalistic approach here, citing a possible lack of authority to make these requirements of accreditors, and avoids the larger question. JCAHO responds with a more realistic objection - standardization could mean the end of their current system, but they agree in principle with the idea of a standardized taxonomy of serious deficiencies. COLA disputes the logic behind the recommendation (things aren't that bad, doing this isn't guaranteed to make things better).
2. Ensure that the advance notice of upcoming surveys provided to physician office labs is consistent with CMS's policy for advance notice provided by state agencies.
Why was this recommendation made?
The CMS response
"We agree. CMS will require any accrediting organization using announced surveys to reduce its lead time to be consistent with CMS policy governing actions of State survey agencies." [page 71]
The COLA response
"We disagree with your assertion that allowing a laboratory to prepare for a survey masks the discovery of laboratory problems. We know of no research that would support such a conclusion.
"While much of a laboratory's evidence of compliance is documentary, there is little of this evidence that can be fabricated in a short period of time....
"Clearly, laboratories that 'fix' or complete records immediately prior to an announced onsite inspection (an example used in your report) have critical management and laboratory operations issues. Our surveyors are trained to spot these problems as well as others that may arise when a laboratory attempts to 'fix' documents or data just before an onsite survey....
"It is important to note that logistically, any reduction in final notice of scheduled survey to laboratories will likely raise the cost to inspect laboratories by the national survey organizations. The increased costs will ultimately be borne by the laboratories themselves....
"We completely agree with your conclusion that unannounced inspections in the smaller lab environment are impractical and have a negative impact on patient care." [page 85]
Our Take
COLA disputes the idea that short notice inspections are the central and only method of evaluating physician office labs. They make a valid point, that when a longer lead time is accompanied by the laboratory's efforts to improve quality, that's a good thing. But we can't assume that every lab makes the best use of a long lead time - too many wait until the last minute. One futher note: the Maryland General scandal may not be research, but certainly most observers (and certainly Congress) concluded that, in light of the problems revealed there, it was time to change how inspections take place.
3. Ensure that regulation of labs is the primary goal of survey organizations and that education to improve lab quality does not preclude the identification and reporting of deficiencies that affect lab testing quality.
Why was this recommendation made?
The CMS response
"We strongly disagree, however, with GAO's statement regarding the 2003 quality control regulations and the cytology proficiency testing for Pap smears. In both instances we strengthened quality requirements and public protections. It was entirely fitting that we emphasize education of providers and full opportunity fr laboratories to understand the new requirements, implement them appropriately, upgrade their systems and practices, and make necessary corrections without unnecessary sanctions.
"The CMS 'educational approach' does not mean that surveyors refrain from identifying deficiencies on the part of laboratories; in fact, objective review and feedback is the bedrock of education. The educational approach does mean that we limit the exercise of sanctions in certain circumstances, particularly in cases of new requirements when the motivational power of sanctions is unnecessary (and may even be counterproductive)....
"CMS believes our educational approach strikes an appropriate balance between (a) the impositions of sanctions for anything less than immediate 100 percent compliance by all laboratories, and (b) any lessening of expectations for proficiency and protection of the public that are not fully reviewed and supported by both evidence and good practice."[page 69]
"For the reasons explained previously, we do not anticipate a change in this policy." [page 72]
The CAP response
"As the GAO correctly notes, CLIA neither requires nor precludes an educational role for surveyors. The College believes that these dual objectives are not mutually exclusive and that education is an inherent and important outcome to the inspection process of identifying and correcting deficiencies." [page 82]
The JCAHO response
"Determining when to employ a punitive versus an educational (or collaborative) approach to promoting compliance can be a difficult judgement. Quality experts maintain that an educational approach is the best way to evaluate weaknesses and achieve and sustain improved performance over time." [page 90]
"The best way to manage at-risk behavior is to remove the negative incentives, create incentives for healthy behaviors, and increase situational awareness. In the patient safety literature overwhelmingly supports the conclusion that punishment encourages organizations to cover up problems." [page 91]
The COLA response
"...we disagree with your suggestion that education and enforcement are mutually exclusive. We feel that enforcement can successfully be coupled with education so that laboratories can learn tools they need for compliance." [page 83]
"COLA feels that such an educational approach is essential to the desired outcome of real improvement in laboratory performance and to prevent the continuation of deficiencies across inspection cycles....There is little benefit to the laboratory and no benefit to public health and safety for the establishment of expectations that laboratories cannot meet. Education is essential to the improvement process so as to empower laboratories to meet or exceed the minimum expectations....
"It has always been the goal of COLA's Accreditation program to bring laboratories, particularly the smaller Physician Office Laboratory (POL) with its less experienced staff, into compliance with the law by a combination of approaches which identifies the deficiencies present and shares with the lab the correct way to assure quality patient testing. COLA then follows up on the identified deficiencies and requires an evidence-based response from the laboratory before their accreditation is approved or continued." [page 84]
Our Take
Every accreditor responded to this recommendation, so clearly they view this as an important point. Remember that this recommendation is confused by the problems with the regulations on "EQC" and cytology PT . Both have been contentious issues, and CMS made them "educational" requirements so they could have time to figure out better requirements to replace them. CMS is flat out rejecting ths recommendation ("we do not anticipate a change"). CAP, JCAHO, and COLA make a valid argument: that education and enforcement are not necessarily mutually exclusive.
4. Impose appropriate sanctions on labs with consecutive condition-level deficiencies in the same requirements.
Why was this recommendation made?
The CMS response
"For those laboratories that continue to incur repeat deficiencies, CMS will use a progressive enforcement approach." [page 69]
"This recommendation is already CMS policy; the issue is our approach to implementation of the policy. CMS' policy of progressive enforcement involves the imposition of sanctions for laboratories failing to correct deficiencies that impact on the quality of laboratory testing, increasing in severity in the event of continuing failures....
"For example, the laboratory could fail in proficiency testing in one year due to neonatal testing, and fail in proficiency testing in a completely different division of the laboratory the next year (e.g. virology). In regard to laboratories with consecutive condition-level deficiencies, the data present by GAO would not permit us to assess whether there is a serious problem because the underlying failures could have been different in the two consecutive surveys for those laboratories that the GAO included in its report." [pages 72-73]
The JCAHO response
"The Joint Commission further disagrees with the GAO recommendation that CMS uniformly impose more sanctions on labs with repeat condition-level deficiencies....Joint Commission believes that GAO's call for CMS to impose more sanctions on laboratories with repeat condition-level deficiencies is likely to be counterproductive." [pages 90-91]
Our Take
Arguing that flexibility is needed, that some repeat deficiencies are not that serious, has some validity. But it is also concerning - if a serious deficiency is actually serious, shouldn't having multiple and/or repeat serious deficiencies require some kind of punishment? Take the above JCAHO example: if a lab is failing their proficiency testing in multiple areas, shouldn't something be done about that? Isn't that a serious sign of things going wrong? Perhaps some deficiencies should be reclassified as not-so-serious, if deficiencies in those areas aren't considered important.
5. Require all survey organizations to develop, and require labs to prominently display, posters instructing lab workers on how to file anonymous complaints.
Why was this recommendation made?
The CMS response
"Information about filing compliants has already been included in the updated Surveyor and Laboratory Interpretive Guideline document and most States already have a Hotline for the receipt of complaints." [page 73]
The JCAHO response
"The Joint Commission believes tha this recommendation is too narrow and prescriptive, and may inadvertantly limit organizations from using other more effective ways to educate lab workers on how to file a complaint." [page 91]
Our Take
JCAHO is right: putting up posters is a strangely narrow and specific recommendation. But posters are cheap. Surely a few posters in the lab can be combined with the other better ways of educating lab workers.
6. Consistent with CLIA, require quarterly proficiency testing, except when technical and scientific considerations suggest that less frequent testing is appropriate for particular examinations or procedures.
Why was this recommendation made?
The CMS response
"CMS already made this determination. While the public explanation emphasized limiting the burden on laboratories, CMS, in conjunction with the Centers for Disease Control and Prevention, concluded on both technical and scientific grounds that proficiency testing three times per year was appropriate." [page 74]
Our Take
We're surprised that this recommendation didn't elicit more responses. After all, if this one gets enforced, labs will have to do a lot more proficiency testing. But perhaps this recommendation is so severe that no one believes it will get enforced. After all, CMS has been lessening regulations and enforcement for years based on practical and budget reasons. It will be interesting to see CMS can find a scientific rationale for reducing from quarterly proficiency testing to just three times a year that actually dates from the time when the decision was made.
7. Ensure that evaluations of exempt-state and accrediting organization inspection requirements take place prior to expiration of the period to expiration of the period for which they are approved in order to ensure the continued equivalency of their requirements with CLIA's.
Why was this recommendation made?
The CMS response
"We recognize the need to complete timely reviews. However, we reserve the right to manage the work within available resources and assessment of priorities." [page 75]
Our Take
Can you imagine what would have happened if Maryland General had responded to their inspection findings by saying "we reserve the right to manage the work within available resources and assessment of priorities"?
8. Ensure that changes to the inspection requirements of exempt states and accrediting organizations be reviewed prior to implementation, as required by regulation, to ensure that individual changes do not affect the overall CLIA equivalency of each organization.
Why was this recommendation made?
The CMS response
"It is correct that the accreditation organization must submit changes to CMS 30 days prior to their implementation [42 CFR 493.557(a)(13)]. However, the regulatory language does not specify a time period for the review of this information by CMS." [page 75]
Our Take
This is a very legalistic response. Clearly the intent of the law is that the CMS makes use of those 30 days to review the changes. Otherwise, why specify 30 days? If no timely review is implied by those 30 days, why specify a time period at all?
9. Allow the CLIA program to utilize revenues generated by the program to hire sufficient staff to fulfill its statutory responsibilities.
Why was this recommendation made?
The CMS response
"CMS faced a decline in CLIA program staff as our workload has increased significantly. We therefore will explore this GAO recommendation." [page 76]
Our Take
CMS is at the mercy of the greater government here. Budget priorities at higher levels have probably dictated the size of their staff, regardless of workload. Looking at the current budget deficits, and those to come, it may be too much to expect CMS will get the authorization to expand its staff.
10. Ensure that federal surveyors validate a sufficient number of inspections conducted by each state survey agency to allow a reasonable estimate of their performance, including a minimum of one independent validation review for each state survey agency surveyor.
Why was this recommendation made?
The CMS response
"we agree that the comparative survey or 'independent validation review' offers a truer assessment of surveyor competency than the observational or participatory Federal Monitoring Survey, and for that reason continue to have the comparative survey as a tool available to Federal surveyors for their oversight responsibilities. We are convinced that Federal surveyors exercise appropriate judgement as to when to select or not select the comparative survey to fulfill their responsibilities for surveyor competency assessment. One must also consider that comparative Federal Monitoring surveys can be disruptive to laboratories as they require two separate surveys conducted during different time frames to separately determine laboratory compliance for CLIA." [page 77]
Our Take
CMS is trying to agree but disagree here. They accept that the independent validations are the best assessment of surveyor competence, but they still insist that surveyors can select a different assessment tool. They are also correct to point out that using more independent validation reviews could be disruptive. The issue of whether or not independent inspections of the same lab can ever agree completely is a topic for another day.
11. Require that almost all validation reviews of each accrediting organizations' surveys be an independent assessment of performance.
Why was this recommendation made?
The CMS response
"CMS will continue to monitor and ensure that the vast preponderance of validation surveys for accrediting organizations takes the form of independent assessments." [page 78]
The JCAHO response
"We further believe that GAO has misinterpreted its validation survey data. It concludes that 'independent' surveys - more commonly referred to as look-behind surveys - are more effective than simultaneous surveys in identifying condition-level deficiencies that were missed by accrediting organizations. However, the data presented in the draft report did not support this assertion. The Joint Commission estimates that 3 percent of the simultaneous validation surveys resulted in findings of condition-level deficiencies missed by accrediting organizations surveyors, compared to the identification of such findings in 5 percent of the 'independent' validation. Thus, thus proportion of condition-level findings is roughly equivalent in both types of surveys." [page 91]
The COLA response
"...we feel that simultaneous validations are often of benefit and helpful to assuring some consistency and predictability between surveying groups. Simultaneous validations should not be assumed to be improper or ineffective....In our experience, most of the discrepant findings in non-simultaneous validation reports are, in fact, incorrect because they are based on false assumptions. For example, CMS' validation process (by virtue of its structure) fails to recognize that COLA performs continual monitoring and enforcement of PT performance is not confined to the onsite survey alone." [page 87]
Our Take
Again, CMS trying to meet GAO halfway. JCAHO disputes the data behind the recommendation. COLA disputes the logic. And again, it is an issue worthy of debate and discussion: can independent inspections of the same lab find the same deficiencies?
12. Collect and routinely review standardized survey findings and other available information for all survey organizations to help ensure that CLIA requirements are being enforced and to monitor the performance of each organization.
Why was this recommendation made?
The CMS response
"We strongly endorse the value of collecting and reviewing survey findings and other available information to monitor, sustain, and improve performance." [page 78]
Our Take
I don't think anyone can disagree with this recommendation. Should we work better, harder, smarter? Of course. Should we look at the data in the survey findings? Of course.
13. Establish an enforcement database to monitor actions taken by state survey agencies and regional offices no labs that lose their accreditation.
Why was this recommendation made?
The CMS response
"Complete the development of the CMS CLIA enforcement database to track and monitor labs that necessitate any potential federal enforcement actions." [page 79]
Our Take
Again, a no brainer.
Conclusion? Thanks but no Thanks
CMS, CAP, JCAHO, and COLA had to endure a lengthy investigation by the GAO. In fact, in many cases, the accreditors had to educate the GAO inspectors on what the accrediting agencies were doing so that the GAO could understand what was going on. Imagine training someone so that they know how to criticize you! Because of this, the GAO recommendations are probably not what most people in the industry expected.
The published early responses note some serious objections by the organizations. Clearly the accreditors are not going to accept everything coming from the GAO. There are some adjustements that they are willing to make (for instance, resisting unannounced inspections was clearly impossible for CAP and JCAHO, so they have already implemented that change), but others they will fight. So expect more responses from CMS, CAP, JCAHO, and COLA, as well as the state agencies in the coming weeks. Clearly there will be a struggle over "educational" and "enforcement" priorities of the inspection processes.
What's been left unsaid thus far is the political appetite for new legislation. Only Congressman Cummings has commented on the report, mostly to state that he thinks the changes already made by CMS, CAP, and JCAHO have been very good, and that there is a proposed law that addresses several of the GAO report recommendations. But no other lawmaker has stepped forward to offer new or broader legislation in the wake of the report. In a year of political warfare (the November 2006 midterms), this is a strangely quiet battlefield.
James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.