EDITORIAL

A Urology Program Director’s View of the ACGME Outcome Project

Approximately seven years ago, the Accreditation Council for Graduate Medical Education (ACGME) launched its Outcome Project. According to its website (www.acgme.org), “…the future for GME accreditation envisioned by the ACGME Outcome Project emphasizes a program’s actual accomplishments through assessment of program outcomes.” As the first major activity of the Outcome Project, the ACGME identified six general competencies for residents. The six general competences are: patient care, medical knowledge, professionalism, systems-based practice, practice-based learning and improvement, and interpersonal and communication skills. While this sounds laudable, I have significant concerns regarding the lack of data supporting implementation of the project.

A pilot project was never carried out, or if it were, the results were never promulgated. Furthermore, after seven years, the ACGME has not published any data measuring the effectiveness of this project to accomplish its overall stated goal. It’s ironic that the ACGME requires data from the training programs to prove their effectiveness, but after 7 years it has provided no data proving the effectiveness of the Outcome Project.

The ACGME has not published any data demonstrating how much time is required to carry out its Outcome Project mandates, let alone how much time it takes to teach the medical and surgical aspects of urology. In an era of appropriate resident work-hour restrictions, it is paradoxical that teaching medical competencies should add more and more topics to teach and measure. No mandates should be imposed without an associated time analysis.

Implementation of the Outcome Project is a costly endeavor. While the ACGME mandates that institutions properly support the training of residents, it has published no data demonstrating any effort on its part to determine what percentage of GME dollars allocated to the sponsoring hospitals are actually spent on GME. Furthermore, the ACGME has not carried out a cost analysis regarding its mandates. Therefore, there are no data revealing how much the project is costing each urology department or the country as a whole. No mandates should be imposed without a cost analysis.

The ACGME requires that programs develop and implement an “organized curriculum.” Certainly, development of a validated curriculum is far beyond the resources of the vast majority of urology residency training programs. There are no ACGME generated data demonstrating that the use of an “organized curriculum” produces better physicians than traditional methods.

The ACGME mandates that we teach and measure the effectiveness of teaching evidence-based medicine (EBM). I suggest that the ACGME practice evidence-based education (EBE). There should be no educational mandates imposed without a data-driven rationale.

Part of the impetus for the initiation of the Outcome Project, as stated on its website, was what some individuals perceived as the threat to the medical profession if we do not regulate our own educational initiatives. In other words, if we did not do it, the government would. However, I have never seen any ACGME generated evidence that any individual or group in government wanted to impose what amounted to an unproven, untested, unfunded educational revolution.

Recently, the Society of Urology Chairpersons and Program Directors (SUCPD) conducted a member survey of the ACGME-approved urology residency training programs in the United States. Dr. Ralph Clayman presented the survey results at the recent combined meeting of SUCPD and the Society of University Urologists (SUU) that was held in conjunction with the 2008 American Urological Association meeting. There were 114 respondents representing approximately 60 urology residency training programs: 90% of the respondents said they noticed no improvement in resident performance; 80% said the outcomes project and competencies provided no new information regarding resident performance or behavior; 75% indicated that they would like to rescind the Outcome Project and competencies if they could; and 80% said the Outcome Project did not improve residency training.

Whenever I have questioned the rationale for the Outcome Project, I have been told that it was based on “educational principles.” Such educational principles are an inadequate and unacceptable substitute for data. The ACGME should hold itself to the same stringent requirements for self assessment data that it applies to urology residency training programs. The academic urological community should demand the same standards from our accreditation organizations that we demand from ourselves.

Richard J. Macchia, M.D
Chairperson & SUNY Downstate
Distinguished Teaching Professor
Department of Urology, SUNY Downstate Medical School
Brooklyn, New York, USA

Do you agree with what you just read? Or maybe you disagree. Let us know and we will publish your thoughts as a Letter to the Editor.

4139 © The Canadian Journal of Urology™; 15(4); August 2008