An article in the 5/29/06 issue of BusinessWeek entitled "Medical Guesswork" is a great story about "evidence-based medicine". Here are some choice quotes:
With a groundbreaking computer simulation, Eddy showed that the conventional approach to treating diabetes did little to prevent the heart attacks and strokes that are complications of the disease. In contrast, a simple regimen of aspirin and generic drugs to lower blood pressure and cholesterol sent the rate of such incidents plunging.
…
For Eddy, this is one small step toward solving the thorniest riddle in medicine — a dark secret he has spent his career exposing. "The problem is that we don’t know what we are doing," he says.
…
Indeed, when he began taking on medicine’s sacred cows, Eddy liked to cite a figure that only 15% of what doctors did was backed by hard evidence.
…
"We don’t have the evidence [that treatments work], and we are not investing very much in getting the evidence," says Dr. Stephen C. Schoenbaum, executive vice-president of the Commonwealth Fund and former president of Harvard Pilgrim Health Care Inc.
…
What’s required is a revolution called "evidence-based medicine," says Eddy, a heart surgeon turned mathematician and health-care economist. … The goal of this movement is to pierce the fog that envelops the practice of medicine — a state of ignorance for which doctors cannot really be blamed.
…
But he soon became troubled. He began to ask if there was actual evidence to support what doctors were doing. The answer, he was surprised to hear, was no. Doctors decided whether or not to put a patient in intensive care or use a combination of drugs based on their best judgment and on rules and traditions handed down over the years, as opposed to real scientific proof.
…
Even when common treatments are proved to be dubious, physicians don’t rush to change their practice. They may still firmly believe in the treatment — or in the dollars it brings in.
Thank goodness we don’t have problems like this in information security. Can you imagine how awful it would be?
There’s a whole section on this in Blink, by Malcom Gladwell, about using actual science to determine who should be admitted to the ICU for observation or treatment of a potential heart attack.