Evidence-based clinical practice guidelines are a primary vehicle for overcoming the healthcare knowledge void and increasing quality through knowledge. They are implemented most effectively in high-fidelity healthcare systems. Following is a discussion of key issues surrounding the development and use of evolving practice guidelines.

What Evidence-Based Practice Guidelines Are

Practice guidelines (also called clinical guidelines) are central to patients’ plan of care.[1] They are, in effect, knowledge models describing a preferred process of care. These guidelines provide recommendations for prevention, treatment, and maintenance of all nontrivial illnesses, conditions, disorders and other healthcare problems causing distress, discomfort, debilitation, and quality of life decline. They are part of disease management and quality improvement programs.

When they are “evidence based”, it means the guidelines are validated through rigorous scientific studies, which recommend the most appropriate clinical choices, treatments, and care of people with specific diseases and conditions. These recommendations are based on the best available evidence and usually reflect a consensus that has been reached or approximated by experts on the care of a particular patient population. The guidelines are related to clinical protocols and pathways, in that the protocols are detailed procedures for how to implement guidelines, and pathways are broader frameworks for organizing the care of patient populations; protocols focus on what is to be done and pathways focus on who should do it, and when it should be done.[2]

Use of such guidelines (along with their associated protocols and pathways) is consistent with evidence based medicine (EBM) activities, which encourage clinicians to base decisions on sound scientific knowledge, not just individual experience or opinion.[3] EBM involves using current best evidence in making decisions about the care of individual patients, and requires the active integration of clinical knowledge, external clinical evidence from the literature, and individual patient needs and wishes.[4] One thorny issue is how to avoid political and ideological biases when determining what evidence to use as the basis for establishing the guidelines.[5,6]

At minimum, information contained in a guideline should include a description of when, why and how to use the guideline; the potential risks, benefits, and costs; a summary of the evidence-based literature supporting the guideline’s cost-effectiveness along with links to the actual studies; as well as alternative approaches to consider.

In addition, guidelines should:
  • Describe appropriate care based on the best available scientific evidence and broad consensus
  • Reduce inappropriate variation in practice
  • Provide a more rational basis for referral
  • Provide a focus for continuing education
  • Promote efficient use of resources
  • Act as focus for quality control, including audit; and highlight shortcomings of existing literature and suggest appropriate future research.[7]
    And the guidelines should be accompanied by consumer versions in lay language and be widely publicized and available to inform patients and the public about what their clinicians should be doing.[8]

Tailoring and Evolving the Guidelines

To be useful, guidelines should be specific enough to have different versions for patients with the same clinical problems depending on their clinician, hospital, location, available resources, patient characteristics and preferences, etc. For example, a recent study found that a moderately high total cholesterol level is associated with higher survival in certain patients with heart failure.[9]

They should also be constantly evaluated and revised based on research and sharing of anecdotal observations and lessons learned via a continuous quality improvement process. If guidelines do not continually evolve, they are “a record of the past, and little more—they should have an expiration date.”[10]

Disease Management and Practice Guidelines

Disease Management is a system of coordinated healthcare interventions types of problems. Use of evidence-based practice guidelines — which include patient self-management education and outcomes measurement, along with treatment recommendations — is a key component of disease management.

Ideally, disease management programs:
  • Support the physician or practitioner-patient relationship and plan of care
  • Emphasize prevention of exacerbations and complications utilizing evidence-based practice guidelines and patient empowerment strategies
  • Evaluate clinical, humanistic, and economic outcomes on an on-going basis with the goal of improving overall health.

Its components should include:
  • Population identification processes
  • Evidence-based practice guidelines
  • Collaborative practice models to include physician and support-service providers
  • Patient self-management education (may include primary prevention, behavior modification programs, and compliance/surveillance)
  • Process and outcomes measurement, evaluation, and management
  • Routine reporting/feedback loop (may include communication with patient, physician, health plan and ancillary providers practice profiling).

Note: Full service disease management programs must include all six components. Programs consisting of fewer components are called “disease management support services.”[11]

Developing Practice Guidelines through Consensus Conferences and Systemic Reviews

Consensus Conferences

Consensus conferences work on developing practice guidelines by group consensus. The process involves identifying and selecting a significant healthcare problem for a patient population for which there is adequate scientific research; a methodology for evaluating available information; examination of any existing guidelines; open discussion of all information and opinions; assessment of benefits, risks and costs of guideline implementation; establishment of guideline meeting the approval criteria; and continuous re-examination of the guideline in light of new research with changes in the consensus panel membership.[12,13]

Systemic Reviews

A systematic review summarizes the healthcare literature using specific methods to perform a thorough literature search and critical examination of individual studies. Its purpose is to identify the valid and useful evidence, and then apply appropriate techniques, such as quantitative meta-analysis and certain qualitative reviews, to combine the valid studies.

According to the Cochrane Collaboration’s handbook for systematic reviewers, each review should contain these main sections:
  • Background
  • Objectives
  • Methods of the review
  • Results
  • Conclusion and discussion.

And the steps for preparing and maintaining a systematic review are:
  • Formulating a problem
  • Locating and selecting studies
  • Critical appraisal of studies
  • Collecting data
  • Analyzing and presenting results
  • Interpreting results
  • Improving and updating reviews.

The value of a high-quality systematic review is the presentation of a balanced and impartial summary of the research with due consideration of any flaws in the evidence, which is especially useful when conflicting findings or substantial uncertainty exists.[14]

Quality Improvement (QI) Programs

There are numerous quality improvement programs being implemented in the healthcare industry. They typically use performance measures to assess quality and practice guidelines to improve quality. These measures and guidelines can differ greatly, however.

The Need for Specialty Measurements

One of the limitations of performance measures in use today is that they are very narrow in scope. Many are process metrics, which simply determine if particular interventions are being implemented, others include clinical outcomes, which determine if the interventions helped patients get better or control chronic conditions.

Many areas of healthcare, however, have neither. This is because providers from different healthcare disciplines and specialties require different types of data to evaluate what they do. For example, it is foolish to measure the quality of mental healthcare services with data appropriate for evaluating cardiologists’ performance; and the same is true for a podiatrist, dentist, chiropractor, etc. — each need measures aligned with their particular goals and the requirement of their QI programs. While specialty-unique measures are used in some QI programs, others do not. And in some QI programs, only claims (i.e., insurance-related) data are used.[15]

Using Practice Guidelines in QI Programs

Quality improvement (QI) programs develop and use practice guidelines. QI programs involve:
  • Establishing collaborative teams to work on the quality improvement process
  • Setting time-specific and measurable improvement aims for specific population of patients
  • Defining quantitative measures used to identify the specific changes that lead to improvements
  • Testing the changes in the real work setting using the “Plan-Do-Study-Act” method — plan a change to how things are done, try doing things the new way, observe the results, and act on what is learned; this is the scientific method used for action-oriented learning
  • Continuing testing and refining the changes until and establish the ones shown to improve quality as recommended guidelines
  • Spreading the guidelines within and between organizations after they are implemented successfully on a small scale (e.g., for a pilot population or an entire unit)
  • Continuing collecting outcomes data on the guidelines from the controlled clinical trials and real-world research (discussed below), and study the data to make further refinements for continually improving care quality. [16]

Practice Guidelines and Quality Measures

In order to evaluate the impact of practice guidelines on quality improvement, measures of quality must be used. The two classes of measures used for quality improvement are care process and outcome measures. Process measures are used to evaluate compliance with guidelines, and outcomes measures are used to evaluate the value of the guidelines; but watch out for the possible pitfalls.[17]

Care Process Measures

Process measures of care quality assess whether providers follow predefined procedures for specific patient types, with specific health problems and needs. These procedures are treatments, interventions, prescriptions, etc., which are supported by scientific evidence, or which have limited scientific support, but practitioners can justify their use. The procedures are incorporated into practice guidelines setting particular standards of care, and they become part of a patient’s plan of care. There are many practice guidelines using care process measures.[18]

Clinical Outcome Measures

Clinical outcomes measures of care assess the clinical results of episodes of care for particular patient types. There are numerous care outcome measures available, which can be used to evaluate the value of particular guidelines, and new ones are created regularly. They include mortality rates, complications (of disease or of medical care), safety (e.g., avoidable adverse events due to errors), biomedical measures (e.g., results of laboratory tests and vital signs), disease specific symptoms (severity, duration, and frequency, both physical and psychological/cognitive/emotional), quality of life (e.g., pain/discomfort, treatment side-effects, mood, energy level, family and social interactions, sexual function, ability to work, and ability to keep up with routine daily activities, functional status/independence), length of stay (duration of treatment), cost of care, patient satisfaction with care rendered, and access to care.[19,20,21,22,23]

Such measures are crucial for continuous quality improvement by providing essential data for researchers and feedback for practitioners. They help refine evidence-based guidelines over time by identifying procedures and systems that work well, and by providing insights into areas where there are opportunities for improvement.

Risk-adjusted clinical outcomes[24] are also vital for making performance transparent, so consumers and purchasers can evaluate quality of care. “For clinicians and patients, outcomes research provides evidence about benefits, risks, and results of treatments so they can make more informed decisions...For health care managers and purchasers, outcomes research can identify potentially effective strategies they can implement to improve the quality and value of care...Historically, clinicians have relied primarily on traditional biomedical measures, such as the results of laboratory tests, to determine whether a health intervention is necessary and whether it is successful. Researchers have discovered, however, that when they use only these measures, they miss many of the outcomes that matter most to patients. Hence, outcomes research also measures how people function and their experiences with care…No longer just the domain of a small cadre of researchers, outcomes research has altered the culture of clinical practice and health care research by changing how we assess the end results of health care services. In doing so, it has provided the foundation for measuring the quality of care. The results of…outcomes research are becoming part of the 'report cards' that purchasers and consumers can use to assess the quality of care in health plans. For public programs such as Medicaid and Medicare, outcomes research provides policymakers with the tools to monitor and improve quality both in traditional settings and under managed care. Outcomes research is the key to knowing not only what quality of care we can achieve, but how we can achieve it.”[25]

Note that in addition to being useful for assess care treatment results, outcome measures can also be used to evaluate the effectiveness of health promotion (wellness/prevention) programs and activities. In this case, they would measure such things as changes in lifestyle (e.g., tobacco use, food choices, physical activity, use of alcohol and illicit drugs, access to appropriate health services, patient knowledge through education, community participation, as well as changes in health environments, policies, and practices.[26]

Patient Ratings

Patient ratings of overall quality of care are unlikely to be good indicator of the actual technical quality of care. Instead, these ratings tend to reflect patients' satisfaction with the communication they have with their healthcare providers, which, while important, are not valid measure of clinical effectiveness.[27]

Potential Pitfalls of Quality Measurement

No mater what quality measures are used, there are complex issues to be resolved, such as:
  • At what point is there sufficient confidence in an evidence-based practice guideline that there is no longer any need to spend time or money on the continuous evaluation of its reliable and validity?
  • When is a definition of quality too narrow, e.g., by focusing on cost or symptom reduction, but not considering prevention, recurrence, coordination and continuity of care, or the patient-physician relationship?
  • How do you measure quality when resources are scarce and optimal care for the community may require less than “the best” care for its individual members (e.g., delegating office nurses to perform certain activities that physicians used to do)?
  • What is the best way to measure quality if outcomes are more strongly affected by patient compliance than by physician orders? This may occur, for example, if certain providers have personalities that trigger greater patient compliance, and visa versa.
  • Is it poor quality care if a provider follows the recommended practice guideline, but the patient is atypical and responds poorly? [28]

Another concern is the use of claims (administrative) data to measure providers' performance. Although they offer some useful measures, they are grossly inadequate metrics for incentives, transparency of cost & effectiveness, and continuous quality improvement. They should, therefore, be augmented with detailed clinical outcomes data that (a) offer more valid measures of performance, and (b) enable researchers to establish and evolve evidence-based practice guidelines.

Furthermore, assessing care quality using process data may not be valid since they do not necessarily reflect care outcomes.[29]
» Blog link: Performance measures only have a little of the answer

In addition, there are concerns about Quality Improvement Organizations (QIOs). QIOs are private contractors paid about $300 million a year by Medicare to work with providers to improve care and investigate complaints. They are under increased scrutiny from Congress for excessive salaries, potential conflicts of interest, leaving patients in the dark, and providing no benefit to improving the care quality.[30]
This points to the need for people with integrity to represent the consumer in any quality improvement initiative.

Methods Used for Validating and Improving Practice Guidelines

The two prominent ways guidelines are validated and updated are through controlled clinical trials[31] and real-world research.[32] Outcomes research is a method of investigation that can be used with either to assess scientific evidence of treatment cost-effectiveness. Sharing anecdotal data and lessons learned are also important methods for improving practice guidelines.

Controlled Clinical Trials

Controlled clinical trials types of research controls conditions, narrowly defines patient problems, and studies very specific patient types. The advantage of controlled clinical trials is that it can demonstrate guidelines cost-effectiveness very clearly without having to consider many external variables and biases. The disadvantages of this research are that it uses an artificial representation of the environment and thus may give results that differ from true real world effectiveness, thereby limiting the generalizability of the research results. It also gives little useful information about comparative effectiveness of alternate treatments since the studies researching different guidelines are likely to have different conditions making the results incompatible. Another possible problem occurs when political pressure drives experimentation leading to unreliable results; this can happen when research funding is tied to finding specific results.

Real-World Research

Real-world research, on the other hand, it is conducted in natural settings of everyday clinical practice. As such, it has the advantages of good generalization and can address the impact of factors in existing in real-life contexts. This means the result of using guidelines real-world research reflect what happens in everyday clinical practice, whereas controlled clinical trials may not be generalizable to actual practice settings with similar types of patients and patient problems. Since real-world research is less controlled, however, the results of measurements tend to be less precise than with controlled clinical trials, and the research results may be influenced by things not known or identified.

Bottom line is that both controlled clinical trials and real-world research has value and should be used based on the particular situation.

Outcome Research

Outcome research seeks to understand the end results of particular healthcare practices and interventions, such as symptom reduction, quality of life, and mortality. Outcome research has become the key to developing better ways to monitor and improve the quality of care[33] by supplying scientific evidence about healthcare cost-effectiveness to support clinical decisions. It is key in the creation and evaluation of evidence-based practice guidelines.

Outcome researchers are interested in studying such things as: (a) average effectiveness (as opposed to efficacy) of treatments due to, for instance, community practice patterns and patient adherence; (b) individual preferences, symptom control and impact on health-related quality of life, for instance due to the effectiveness or side-effect characteristics of a drug, or a more convenient regime; (c) estimated long-term outcomes for, say, chronic medications, which commonly extend months or years beyond clinical trial data; (d) cost-effectiveness information to help decision-makers determine best value for money; and (e) budget impact analysis to help decision-makers evaluate whether they can afford the addition of a new technology.[34]

The Cochrane Collaboration[35] is an example of an international organization that synthesizes clinical evidence for use in healthcare decision-making, and may other groups focused on outcomes exist.[36]

Observational Research, Anecdotal Data, and Lessons Learned

In addition to formal research studies, data from observation research (called clinical practice improvement) — which discovers associations among patient characteristics, process steps, and outcomes — are useful in convincing organizations to try promising interventions. And social science approaches, such as narrative studies (telling stories about clinical experiences), can enhance learning.[37] Also, providers sharing anecdotal data and lessons learned can be helpful in evaluating an intervention’s cost-effectiveness. For example, this feedback can help yield hypotheses to test using other methods, identify unexpected pitfalls and benefits, and stimulate discussions to emerge innovative ideas.

What to do when Evidence is in Dispute

Decisions about whether to establish or maintain a practice guideline can be difficult, even when scientific evidence supporting its value exists. Disputes over the evidence can arise from conflicts in the other spheres that influence decisions, such as the values, preferences, and circumstances of individuals and the communities they represent. The evidence itself can also be subject to debate. In either case, a formal evidence-based approach can help separate questions of evidence from the other important considerations, which often reveals surprising consensus on the scientific evidence that has been masked by fundamental differences of opinion about what outcomes are most important and what actions are appropriate in the face of imperfect evidence.

Such conflicts can be resolved by focusing on identifying the most important outcomes, evaluating the quality of evidence,and assessing the trade-offs involved, so these six questions can be answered: (1) What is the ultimate goal, and how does the guideline achieve those ends? (2) How good is the evidence that the guideline can improve important outcomes? (3) How good is the evidence that the guideline will work in my setting? (4) How do the potential benefits compare with the possible harms or costs of the guideline? (5) What constitutes "good enough" evidence for using the guideline? (6) What other considerations are relevant to making such decisions?[38]

Guidelines are Important, but Don’t Forget Bedside Manner

Providers possess qualities that cannot be readily measure, but which affect the outcomes of practice guidelines, and should therefore by addressed. According to a Stanford School of Medicine Dean’s newsletter, “We can and must, of course, not lose sight of educating our students and trainees to learn, to listen, to reach out and to connect to their patients. Some have called this the ‘art-of-medicine’ others ‘bedside manner’. From my perspective, it is the fundamental underpinning of what makes a great physician.

Exceptional scientific knowledge, along with a critical and analytic approach to clinical care that is evidence based and data-driven is essential. But unless these skills are coupled with a caring and compassionate manner, the value of the patient encounter is diminished. Importantly the patient feels less well served and perception of the physician as a ‘healer’ is altered.”[39]

And research supports this, finding that patients judge a good doctor based on bedside manner rather than technical knowledge and skills. The qualities patients attribute to an “ideal” doctor include honesty, compassion and respectfulness, while their worst experiences are a provider’s arrogance, dismissive attitude and “callousness” in discussing their condition.[40]

Links to blog posts:
» Importance of patient-provider relationship

Disseminating, Updating and Accessing Practice Guidelines

As practice guidelines are developed and validated, they should be made available to all stakeholder by storing them in electronic libraries, along with any instructional and reference materials needed to learn about and understand the guidelines, and to use them effectively. The guidelines should be:
  • Modified and refined on a continual basis in light of new research findings, so they evolve based on new scientific evidence.
  • Tailored to the needs of each stakeholder. For example, consumers would have access to guidelines written in a simplified, non-technical way they can comprehend, while providers would receive technical clinical information.
  • Accessible by all stakeholders quickly and easily, through web sites and specially designed software tools.

In addition, there needs to be a very efficient and effective way to disseminate updated clinical knowledge to providers. Since practice guidelines can change rapidly with evolving clinical trials, after-market surveillance, and new classes of drugs and procedures, a technology must be used that actively delivers (“pushes”) the new and refined guidelines to all clinicians immediately.[37]

Next: Tactic - Offer Consumer Education and Wellness Programs

Footnotes and References

[1] A plan of care is a treatment plan by a healthcare provider, which establishes the goals and details of procedures, medications, therapies, interventions, etc. designed to meet the particular needs of a patient for a particular episode of care. The Quality though Knowledge model specifies incorporating practice guideline or clinical pathways in the plan of care, along with methods and measures for evaluating their efficacy.
[2] March, A. (2006). Facilitating Implementation of Evidence-Based Guidelines in Hospital Settings: Learning From Trauma Centers, The Commonwealth Fund. Available at http://www.cmwf.org/publications/publications_show.htm?doc_id=378879
[3] The medical knowledge crisis and its solution through knowledge management. (2000). OpenClinical. Available at http://www.openclinical.org/whitepaper.html
[4] Evidence-Based Knowledge Portal at the Vanderbilt University Medical Center’s Eskind Biomedical Library. Available at http://www.mc.vanderbilt.edu/biolib/ebmportal/ebm_principles/intro_ebm_principles/ebmdefinition.htm?call=ebm
[5] Healy, B. (Sep. 2006).Who Says What's Best? U.S. News and World Report. Available at http://www.usnews.com/usnews/health/articles/060903/11healy.htm
[6] Holmes, D., et al. (2006). Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. Int J Evid Based Healthc; 4: 180–186. Available at http://www.ucl.ac.uk/Pharmacology/dc-bits/holmes-deconstruction-ebhc-06.pdf
[7] OpenClinical web site - http://www.openclinical.org/guidelines.html
[8] Woolf, S. H., et al. (1999). Potential benefits, limitations, and harms of clinical guidelines. BMJ 318 (20). Available at http://bmj.bmjjournals.com/cgi/reprint/318/7182/527.pdf
[9] Reuters (Sep 20, 2006). Elevated cholesterol may benefit failing hearts. Available at http://go.reuters.com/newsArticle.jhtml?type=healthNews&storyID=13549321&src=rss/healthNews
[10] Gawande, A (2004). The Bell Curve. The New Yorker. Available at http://www.newyorker.com/printables/fact/041206fa_fact
[11] See Disease Management Association of America at http://www.dmaa.org/definition.html
[12] Verstraete, M. (2002) How to increase the impact of clinical guidelines on medical practice. European Review; 10: 545-553 Cambridge University Press
[13] Consensus Conferences held in the Royal College of Physicians. Available at The Royal College of Physicians of Edinburgh’s web site at http://www.rcpe.ac.uk/education/standards/consensus/index.php
[14] Lavis, J. N., et al. (2006).Working Within and Beyond the Cochrane Collaboration to Make Systematic Reviews More Useful to Healthcare Managers and Policy Makers. Healthcare Policy; 1(2):21–33. Available at http://www.longwoods.com/product.php?productid=17872&cat=&page=1
[15] Andis Robeznieks. (March 22, 2006) Need for specialty measurements seen: CMS leader. http://www.modernhealthcare.com/article.cms?articleId=39169 .
[16] See the Institute for Healthcare Improvement at http://www.ihi.org/IHI/Topics/Improvement/ImprovementMethods/HowToImprove/
[17] For examples of quality measures incorporating both process and clinical outcome measures, see the Doctor's Office Quality - Information Technology (DOQ-IT) site at http://www.doqit.org/dcs/ContentServer?cid=1111783079915&pagename=DOQIT%2FDOQITPage%2FPageTemplate&c=DOQITPage
[18] For examples of clinical guidelines with care processes measures, see http://www.guideline.gov/browse/guideline_index.aspx and http://www.ahrq.gov/clinic/cpgonline.htm.
[19] Irish Clearing House on Health Outcomes at http://www.ich.ie/measures.htm
[20] Examples of clinical outcomes for acute care, psychiatric care, long term care, and home care at the International Quality Indicator Project (IQIP) web site at http://www.internationalqip.com/indicators.aspx
[21] Examples of clinical outcomes for hospital are JCAHO’s ORYX measures; see http://www.npic.org/Oryx_Measures.html and http://www.premierinc.com/all/informatics/services/oryx/noncoremeasures.jsp
[22] University of Oklahoma College of Allied Health Department of Rehabilitation Science at http://moon.ouhsc.edu/dthompso/CDM/outcomes.htm
[23] AcademyHealth. Health Outcomes Core Library Project - The National Information Center on Health Services Research and Health Care Technology, National Library of Medicine at http://www.academyhealth.org/publications/healthoutcomesreport.htm
[24] Outcomes data are risk-adjusted to account for the differences between patients treated by different providers so the outcomes data can be used to compare the performance of providers. This is necessary since some providers treat patient who are older or sicker than the patients of other providers. The risk factors include patient demographics (e.g., gender and age) and clinical characteristics (e.g., type of illness, severity of symptoms, and coexisting problems) that might influence the outcome care.
[25] Outcomes Research. Fact Sheet. AHRQ Publication No. 00-P011, March 2000. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/outfact.htm
[26] Irish Clearing House on Health Outcomes. About Outcomes. Available at http://www.ich.ie/about_outcomes.htm
[27] Chang, J.T., et al. (2006). Patients' Global Ratings of Their Health Care Are Not Associated with the Technical Quality of Their Care. Annals of Internal Medicine; 144:665-672. Abstract available at http://www.annals.org/cgi/content/abstract/144/9/665
[28] Donabedian, A. (2005). Evaluating the Quality of Medical Care. The Milbank Quarterly 83, 691–729. Available at http://www.milbank.org/quarterly/830416donabedian.pdf
[29] HealthDay (July 5, 2006). Hospital Ratings Don't Fully Reflect Patient Outcomes. Available at http://health.yahoo.com/news/_164028
[30] Gaul, G.M. (March 7, 2006). Senate Report Faults Medicare's Handling of Complaints. The Washington Post. Available at http://www.washingtonpost.com/wp-dyn/content/article/2006/03/06/AR2006030601386.html
[31] The gold standard of research includes methods such as randomization, control group, randomized controlled trial (RCT), cohort study, case-control study, cross-sectional study, crossover study design, double-blind, and intention-to-treat analysis. See the Evidence-Based Knowledge Portal at the Vanderbilt University Medical Center’s Eskind Biomedical Library for more about the major types of studies and their advantages and disadvantes. Available at http://www.mc.vanderbilt.edu/biolib/ebmportal/ebm_principles/intro_ebm_principles/ebmdefinition.htm?call=ebm
[32] Real-world research includes field experiments, field studies, survey studies, case studies.
[33]Outcomes Research. Fact Sheet. AHRQ Publication No. 00-P011, March 2000. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/outfact.htm
[34] Where does Outcomes Research fit into Evidence-Based Health Care Decision-Making? International Society for Pharmacoeconomics and Outcomes Research http://www.ispor.org/councils/Decision_Making.asp
[35] Cochrane Collaboration, www.cochrane.org
[36] Groups focused on outcomes research include the Centers for Disease Control and Prevention (CDC), the Agency for Health Research and Quality (AHRQ), the Centers for Medicare and Medicaid, Society for Medical Decision-Making (SMDM), the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) and Academy Health, National Institute for Clinical Excellence (NICE), the Canadian Coordinating Office of Technology Assessment (CCOHTA), Australian’s Pharmacy Benefit Advisory Committee (PBAC), International Network of Agencies for Health Technology Assessment (INAHTA)
[37] Foubister, V. (January 2006). Quality Matters: Evaluating Evidence for Quality Improvement, The Commonwealth Fund. Vol. 15. Available at http://www.cmwf.org/publications/publications_show.htm?doc_id=339493
[38] Atkins, D. et al. (2005). Making Policy When the Evidence is in Dispute. Health Affairs; 24 (1), 102-113. Available at http://content.healthaffairs.org/cgi/content/abstract/24/1/102
[39] The Dean's Newsletter - Stanford School of Medicine. (November 10, 2003) Available at http://deansnewsletter.stanford.edu/archive/11_10_03.html
[40] Doctor's bedside manner still what counts: Compassion and respect outweigh technical skills, patient survey finds. Reuters: March 23, 2006.
[41] Shojania, K. G., & Grimshaw, J. M. (2005). Evidence-based quality improvement: The state of the science. Health Affairs, 24(1), 138-150. Available at http://content.healthaffairs

external image meter.asp?site=s24wellness