INTRODUCTION
In contrast to ordering more tests and offering more services, there is an increasing emphasis on the quality of healthcare delivered. This is driven by the recognition that healthcare delivery itself can be dangerous. However, exploding costs are also driving new payment models that financially incentivize stakeholders to produce outcomes rather than simply offer more services. Quality improvement in healthcare is a broad subject, but our team has focused on the use of data (e.g., from the electronic health record system) and iteration. Of course, there are a multitude of methods and frameworks to encourage quality improvement iteration including Lean, Six Sigma, and Total Quality Management among others.
The Phrase Health team often adopts the Plan-Do-Study-Act (PDSA) framework given its simplicity. The name, PDSA, is an acronym for the steps in a circular process of iteration. The first step, “Plan”, involves activities like setting goals and establishing methods for data collection. The next step, “Do”, entails actually implementing the plan, collecting data, and identifying issues.The next step, “Study”, incorporates activities around the analysis of data and comparing it with the hypotheses from the planning stage. Finally, the “Act” step concludes by determining the need for an additional iteration or, if goals are met, incorporating the changes systemwide. Of course, the most important consideration is about pursuing additional iterations. Projects are bound to require multiple iterations and variations before meeting their desired goals.
WIth this PDSA methodology in mind, it raises the question of how PDSA has been implemented in the literature. In 2013, a team from London sought answers by conducting a systematic review of the literature. Their findings highlight the state of adoption of this methodology in quality improvement.
APPROACH
A systematic review of the literature requires a broad search of all available publications on a specific topic. This is a similar approach to the Journal Club article previously published on the effectiveness of interruptive electronic health record (EHR) alerts. In the case of the PDSA review, the team used the following search terms in a variety of article databases: 'PDSA', 'PDCA', 'Deming Cycle', 'Deming Circle', 'Deming Wheel' and 'Shewhart Cycle'.
A set of standardized inclusion principles were used and the articles were independently reviewed by three researchers. These are the criteria that they used:
The researchers attempted to map integral parts of the PDSA cycle into observable data elements that they could identify in a study. For example, some of the key parts included: “iterative cycles”, “small-scale testing”, and “documentation.” In the case of “iterative cycles,” the researchers could assess inclusion of “were multiple cycles used” or “when isolated cycles were used were future actions based on the 'Act' stage?”
Key Features of PDSA Framework
Table 2 from the original paper
It’s important to recognize that the researchers didn’t aim to evaluate if the PDSA approach was successful, but rather was PDSA used. As a result, the researchers acknowledged that this may introduce publication bias in the number of studies published given unsuccessful implementations are less likely to be published.
RESULTS
The researchers found 409 unique articles based on their search. After further investigation into the abstracts and/or full texts, the team settled on 73 papers that met the inclusion criteria. The researchers provided a broad set of descriptive statistics and findings from these available studies. Below, many of these have been mapped back to the key features of the authors' PDSA framework presented earlier. (All data derived from Table 2 from the original paper.)
Iterative Cycles
Description: To achieve an iterative approach, multiple PDSA cycles must occur. Lessons learned from one cycle link and inform cycles that follow. Depending on the knowledge gained from a PDSA cycle, the following cycle may seek to modify, expand, adopt or abandon a change that was tested
How Measured: Were multiple cycles used? Were multiple cycles linked to one another (ie, does the 'act' stage of one cycle inform the 'plan' stage of the cycle that follows)? When isolated cycles were used were future actions postulated in the 'act' stage?
Included: 49 studies
The authors found a total of 49 studies that referenced any iterative analysis. Of these, only 14 used successive iterations. Another 33 used unlinked cycles. The final 2 actually used the incorrect order of PDSA!
Prediction-Based Testing of Change
Description: A prediction of the outcome of a change is developed in the 'plan' stage of a cycle. This change is then tested and examined by comparison of results with the prediction.
How Measured: Was a change tested? Was an explicit prediction articulated?
Included: 47 studies
The PDSA cycle follows the tenants of the scientific method. During the 'plan' stage, a hypothesis is developed and then tested through the cycle. A total of 30 studies used a single cycle to test a change. Another 3 used the framework to simple analyze the data. Of those that actually used successive iterations, 8 iteratively tested their hypotheses. On the other hand, 5 of these iterative PDSA studies started with data collection cycles and then successive testing cycles.
Small-Scale Testing
Description: As certainty of success of a test of change is not guaranteed, PDSAs start small in scale and build in scale as confidence grows. This allows the change to be adapted according to feedback, minimizes risk and facilitates rapid change and learning
How Measured: Sample size per cycle? Temporal duration of cycles? Number of changes tested per cycle? Did sequential cycles increase scale of testing?
Included: 47 studies
The authors refer to "scale" in a few ways. These include number of included subjects, duration of study, and complexity. The size was difficult to compare as some studies reported this for the entire project. Only 27 studies included sample sizes for at least one PDSA cycle. Of the studies with a single PDSA cycle, the range was 7 to 2079 (mean=323.33, standard deviation=533.60). The first cycle of the iterative studies ranged from 1 to 34 (mean=16.75, standard deviation=11.47). A single PDSA cycle ranged from 2 weeks to 5 years (mean=11.91 months, standard deviation=12.81). For articles that mentioned successive cycles, total project lengths covered from 1 day to 4 years (mean=20.38 months, standard deviation=20.39). Finally, 22 of the articles tried to test more than one change in at least a single cycle.
Documentation
Description: Documentation is crucial to support local learning and transferability of learning to other settings
How Measured: How thoroughly was the application of the PDSA method detailed in the reports? Was each stage of the PDSA cycles documented?
Included: 73 studies
Since there needed to be some mentioned and detail about the cycles, all of the 73 included studies qualified for documentation. However, there were 16 publications that included no details around the PDSA cycles whatsoever. On the other hand, 47 articles described details from each stage of the PDSA cycle.
Data Over Time
Description: Data over time increases understanding regarding the variation inherent in a complex healthcare system. Use of data over time is necessary to understand the impact of a change on the process or outcome of interest.
How Measured: Was data collected over time? Were statistics used to test the effect of changes and/or understand variation?
Included: 47 studies
Of course, data can be either quantitative of qualitative. Objective quantitative data is more easily measured and easier to analyze change. Several studies used quantitative techniques. However, the researchers broke down the qualitative data assessment into four types of categories:
Of note, none of the publications used statistical process control (SPC) methods to assess change. Only 11 used some statistical tests to assess the data.
The study also mentioned the geographic distribution. The studies came from: USA (n=46), UK (n=13), Canada (n=3), Australia (n=3), Netherlands (n=2). Additionally, the authors mention a broad array of clinical settings. However, the two most common were surgery (n=6) and pain management (n=6).
DISCUSSION
This systematic review was published in 2013, so the field has continued to evolve since then. However, there is a large amount of variation in the reporting of quality improvement efforts using PDSA. This is ironic because the reduction of variation is a key goal when aiming to deliver reliable results. Overall, the researchers found that assessing compliance with any part of the PDSA cycle was difficult because of a lack of standard reporting procedures.
Only two of the publications (bolded in references below) demonstrated all integral components of the PDSA framework. This shows a lack of rigor or available resources when applying the framework. Either way, it may reflect poorly on the results from these efforts. The authors note, “The theoretical framework presented in this paper highlights the complexity of PDSA cycles and the underpinning knowledge required for correct application.” As a result, the researchers warn about the interpretability of the results of many studies that reference PDSA as they may not truly adopt it appropriately.
One of the key features of the authors PDSA framework is “the use of data over time.” Surprisingly, only 7 papers used monthly or more frequent data as part of their PDSA approach. As data is key in evaluating effectiveness, it’s impossible to assess directionality of efforts without some available quantifiable information. Access to self-service data tools can be an important factor in encouraging quality improvement using PDSA or other frameworks.
EVALUATING QUALITY IMPROVEMENT USING PDSA
Phrase Health provides tools to analyze and manage EHR interventions. If tools like EHR alerts or order sets are part of your quality improvement efforts, the self-service access to dashboards can lower the bar to data accessibility. Additionally, Phrase Health enables quality improvement teams to construct entire projects using key driver diagrams to map out interventions, process measures, and outcome measures.
Journal Club Source
Taylor MJ, McNicholas C, Nicolay C, et al. Systematic review of the application of the plan–do–study–act method to improve quality in healthcare. BMJ Quality & Safety 2014;23:290-298.
Bader MK, Palmer S, Stalcup C, Shaver T. Using a FOCUS-PDCA Quality Improvement Model for Applying the Severe Traumatic Brain Injury Guidelines to Practice: Process and Outcomes. Worldviews on Evidence-based Nursing presents the archives of Online Journal of Knowledge Synthesis for Nursing. 2002;E9(1):97-100.
Baker DL. Successful performance improvement. AORN journal. 2002;75(4):825.
Barry S, Hughes G, Lawton-Smith S. Small is beautiful. Mental health today (Brighton, England). 2006:30.
Beger D, Messenger F, Roth S. Self-Administered Medication Packet for Patients Experiencing a Vaginal Birth. Journal of Nursing Care Quality. 1999;13(4):47-59.
Bittle MJ, Charache P, Wassilchalk DM. Performance Improvement: Registration-Associated Patient Misidentification in an Academic Medical Center: Causes and Corrections. Joint Commission Journal on Quality and Patient Safety. 2007;33(1):25-33.
Boesch RP, Myers C, Garrett T, Nie AM, Thomas N, Chima A, et al. Prevention of Tracheostomy-related Pressure Ulcers in Children. Pediatrics. 2012;129(3):e792-e7.
Boyd S, Aggarwal I, Davey P, Logan M, Nathwani D. Peripheral intravenous catheters: the road to quality improvement and safer patient care. Journal of Hospital Infection. [doi: 10.1016/j.jhin.2010.09.011]. 2011;77(1):37-41.
Brown A. Redesigning patient services. Nursing Management - UK. 2006;13(2):26-30.
Buckley JD, Joyce B, Garcia AJ, Jordan J, Scher E. Linking Residency Training Effectiveness to Clinical Outcomes: A Quality Improvement Approach. Joint Commission Journal on Quality and Patient Safety. 2010;36(5):203-8.
Buhr GT, White HK. Quality Improvement Initiative for Chronic Pain Assessment and Management in the Nursing Home: A Pilot Study. Journal of the American Medical Directors Association. [doi: 10.1016/j.jamda.2005.11.002]. 2006;7(4):246-53.
Campbell S, MacDonald M, Carr B, Anderson D, MacKinley R, Cairns S. Bridging the gap between primary and secondary care: use of a clinical pathway for the investigation and management of deep vein thrombosis. Journal of health services research & policy. 2008;13(suppl 1):15-9.
Caswell DR, Williams JP, Vallejo M, Zaroda T, McNair N, Keckeisen M, et al. Improving pain management in critical care. The Joint Commission journal on quality improvement. 1996;22(10):702-12.
Chen M, Deng, Jin-Hua, Zhou, Fu-De, Wang M, et al. Improving the management of anemia in hemodialysis patients by implementing the continuous quality improvement program. Basel, SUISSE: Karger; 2006.
Christie P, Robinson H. Using a communication framework at handover to boost patient outcomes. Nurs Times. 2009 Dec 1-7;105(47):13-5.
Curran E, Bunyan D. Using a PDSA cycle of improvement to increase preparedness for, and management of, norovirus in NHS Scotland. Journal of Hospital Infection. 2012.
Dobrzanska L, Young L, Patterson C. Piloting stroke rehabilitation in a community hospital. Nursing times. 2006;102(43):30-1.
Dover N. Caring for patients in the right place at the right time. Emergency nurse: the journal of the RCN Accident and Emergency Nursing Association. 2012;20(3):30.
Dunn SL, Shattuck SR, Baird L, Mau J, Bakker D. Developing a nursing model of care? Try focus groups. Nursing Management. 2011;42(8):24- 6.
Eckhart J, Gilbert P. Improved Coumadin therapy using a continuous quality improvement process. Clin Lab Manage Rev. 1996 MarApr;10(2):153-6.
Eisenberg P, Painter JD. Intravascular therapy process improvement in a multihospital system: don't get stuck with substandard care. Clin Nurse Spec. 2002 Jul;16(4):182-6.
Esmail R, Kirby A, Inkson T, Boiteau P. Quality improvement in the ICU. A Canadian perspective. Journal of critical care. 2005;20(1):74.
Feehery PA, Allen S, Bey J. Flushing 101: using a FOCUS-PDCA quality improvement model to reduce catheter occlusions with standardized protocols. Journal of Vascular Access Devices. 2003;8(2):38-45.
Fernandes S, Benjamin EE, Edwards G. Using evidence to reduce the rate of episiotomy in a Dubai hospital. Evidence-based Midwifery. 2009;7(2):60.
Flynt G, Caraway C. Using OASIS data to improve skin care. Home Healthc Nurse. 2002 Apr;20(4):263-6.
Gillaspie M. Better Pain Management After Total Joint Replacement Surgery: A Quality Improvement Approach. Orthopaedic Nursing. 2010;29(1):20-4 10.1097/NOR.0b013e3181c8cd32.
Gordon DB, Jones HD, Goshman LM, Foley DK, Bland SE. A Quality Improvement Approach to Reducing Use of Meperidine. Joint Commission Journal on Quality and Patient Safety. 2000;26(12):686-99.
Gordon DB, Rees SM, McCausland MP, Pellino TA, Sanford-Ring S, Smith-Helmenstine J, et al. Improving Reassessment and Documentation of Pain Management. Joint Commission Journal on Quality and Patient Safety. 2008;34(9):509-17.
Gray J, Eden G, Williams M. Developing the public health role of a front line clinical service: integrating stop smoking advice into routine podiatry services†. Journal of Public Health. 2007 June 1, 2007;29(2):118-22.
Hallett N, Hewison A. How to address the physical needs of clients in a mental health setting. Nursing management (Harrow, London, England: 1994). 2012;18(10):30.
Hoskins EJ, Sayger SA, Westman JS. Quality Improvement in Patient Distribution at a Major University Student Health Center. Journal of American College Health. [doi: 10.1080/07448480209603449]. 2002 2002/05/01;50(6):303-8.
Isouard G. A quality management intervention to improve clinical laboratory use in acute myocardial infarction. The Medical journal of Australia. 1999;170(1):11-4.
Johnson P, Raterink G. Implementation of a diabetes clinic-in-a-clinic project in a family practice setting: using the plan, do, study, act model. J Clin Nurs. 2009;18(14):2096-103.
Koll BS, Straub TA, Jalon HS, Block R, Heller KS, Ruiz RE. The CLABs collaborative: a regionwide effort to improve the quality of care in hospitals. Joint Commission Journal on Quality and Patient Safety. 2008;34(12):713-23.
Leone AF, Standoli F, Hirth V. Implementing a Pain Management Program in a Long-Term Care Facility Using a Quality Improvement Approach. Journal of the American Medical Directors Association. 2009;10(1):67-73.
Lynch-Jordan AM, Kashikar-Zuck S, Crosby LE, Lopez WL, Smolyansky BH, Parkins IS, et al. Applying Quality Improvement Methods to Implement a Measurement System for Chronic Pain-Related Disability. Journal of Pediatric Psychology. 2010 January 1, 2010;35(1):32-41.
Manfredi SR, Canziani ME, Draibe SA, Dalboni MA, Andreolli MC, Watanabe R, et al. A model for improving quality in nephrology settings. Nephrol Nurs J. 2003 Jun;30(3):295-9.
Marang-van de Mheen P, Stadlander M, Kievit J. Adverse outcomes in surgical patients: implementation of a nationwide reporting system. Quality and Safety in Health Care. 2006;15(5):320-4.
Marcellus L, Harrison A, MacKinnon K. Quality Improvement for Neonatal Nurses, Part II: Using a PDSA Quality Improvement Cycle Approach to Implement an Oral Feeding Progression Guideline for Premature Infants. Neonatal Network: The Journal of Neonatal Nursing. 2012;31(4):215-22.
McPharlin M, Shepard AD, Kiell CS, Nypaver TJ. FOCUS-PDCA: A Quality Improvement Tool to Improve Efficiency in the Vascular Laboratory. Journal of Vascular Technology. 1993;17(5):243-6.
Meehan CD, Silvestri A, Street ED. Improving blood glucose monitoring in a hospital setting using the PDCA approach. Journal of Nursing Care Quality. 1993;7(4):56-63.
Miano B, Wood W. Implementation of the i.v. push method of antibiotic administration using the FOCUS/PDCA approach. Home Healthc Nurse. 1998 Dec;16(12):831-7.
Miller LJ, Clark GB. Quality improvement in the cutaneous micrographic surgery laboratory. Clin Lab Manage Rev. 1994 Nov-Dec;8(6):574-6, 8-86, 88-92.
Moran S. Improving palliative care. Nurs Manag (Harrow). 2009 May;16(2):14-7.
Nakayama DK, Bushey TN, Hubbard I, Cole D, Brown A, Grant TM, et al. Using a Plan-Do-Study-Act Cycle to Introduce a New OR Service Line. AORN journal. 2010;92(3):335-43.
Nayeri ND, Zargar MT. An investigation into the effects of quality improvement method on patients' satisfaction: a semi experimental research in Iran. Acta Medica Iranica. 2011;49(1).
New SW, Gutierrez L. Quality improvement in the ambulatory surgical setting. Nurs Clin North Am. 1997 Jun;32(2):477-88.
Nicotra D, Ulrich C. Process improvement plan for the reduction of nosocomial pneumonia in patients on ventilators. Journal of Nursing Care Quality. 1996;10(4):18.
Olenginski TP, Newman ED, Hummel JL, Hummer M. Development and evaluation of a vertebral fracture assessment program using IVA and its integration with mobile DXA. Journal of Clinical Densitometry. 2006;9(1):72-7.
Oyler J, Vinci L, Johnson J, Arora V. Teaching Internal Medicine Residents to Sustain Their Improvement Through the Quality Assessment and Improvement Curriculum. J Gen Intern Med. 2011;26(2):221-5.
Pace NM, Long JB, Elerding S, Lim H, Kelly M, Reed J, et al. Performance Model Anchors Successful Nutrition Support Protocol. Nutrition in Clinical Practice. 1997 December 1, 1997;12(6):274-9.
Porter C, Greenfield C, Larson A, Gilles M. Improving GP diabetes management--A PDSA audit cycle in Western Australia. Aust Fam Physician. 2009;38(11):939.
Pronovost PJ, Morlock L, Davis RO, Cunningham T, Paine L, Scheulen J. Using Online and Offline Change Models to Improve ICU Access and Revenues. Joint Commission Journal on Quality and Patient Safety. 2000;26(1):5-17.
Provance L, Alvis D, Silfen E. Quality improvement and public health-tetanus immunization in the emergency department. American Journal of Medical Quality. 1994;9(4):165-71.
Reid D, Leyland J, Gill L. Does client self-booking reduce ‘did not attends’ (DNAs) in a counselling service? Counselling and Psychotherapy Research. [doi: 10.1080/14733140500510275]. 2005 2005/12/01;5(4):291-4.
Reid D, Glascott G, Woods D. Improving referral information in community mental health. Nurs Times. 2005 Oct 18-24;101(42):34-5.
Robarts S, Kennedy D, MacLeod AM, Findlay H, Gollish J. A framework for the development and implementation of an advanced practice role for physiotherapists that improves access and quality of care for patients. Healthcare quarterly (Toronto, Ont). 2008;11(2):67.
Sanchez I. Implementation of a Diabetic Visual Foot Assessment in a Primary Care Setting. The Internet Journal of Advanced Nursing Practice. 2009;10(2).
Saxena S, Ramer L, Shulman IA. A comprehensive assessment program to improve blood‐administering practices using the FOCUS–PDCA model. Transfusion. 2004;44(9):1350-6.
Simon NV, Heaps KP, Chodroff CH. Improving the processes of care and outcomes in obstetrics/gynecology. The Joint Commission journal on quality improvement. 1997;23(9):485-97.
Sorokin R, Gottlieb JE. Enhancing Patient Safety During Feeding-Tube Insertion: A Review of More Than 2000 Insertions. Journal of Parenteral and Enteral Nutrition. 2006 SEPTEMBER-OCTOBER 2006;30(5):440-5.
Stadt J, Molare E. Best practices: that improved patient outcomes and agency operational performance. Home Healthc Nurse. 2005 Sep;23(9):587-93.
Stevens DP, Bowen JL, Johnson JK, Woods DM, Provost LP, Holman HR, et al. A multi-institutional quality improvement initiative to transform education for chronic illness care in resident continuity practices. J Gen Intern Med. 2010;25:574-80.
Sumrall D, Douglas J. Achieving appropriate prophylactic antibiotic administration while simultaneously implementing an automated anesthesia record. Ochsner J. 2011 Spring;11(1):34-6.
Tea C, Ellison M, Feghali F. Proactive patient rounding to increase customer service and satisfaction on an orthopaedic unit. Orthopaedic Nursing. 2008;27(4):233-40.
Thakkar K, Gilchrist M, Dickinson E, Benn J, Franklin BD, Jacklin A, et al. A quality improvement programme to increase compliance with an anti-infective prescribing policy. Journal of Antimicrobial Chemotherapy. 2011;66(8):1916-20.
Tomolo AM, Lawrence RH, Aron DC. A case study of translating ACGME practice-based learning and improvement requirements into reality: systems quality improvement projects as the key component to a comprehensive curriculum. Postgraduate Medical Journal. 2009 October 1, 2009;85(1008):530-7.
Torkki PM, Alho AI, Peltokorpi AV, Torkki MI, Kallio PE. Managing urgent surgery as a process: Case study of a trauma center. International Journal of Technology Assessment in Health Care. 2006;22(02):255-60.
van Tiel FH, Elenbaas TWO, Voskuilen BMAM, Herczeg J, Verheggen FW, Mochtar B, et al. Plan-do-study-act cycles as an instrument for improvement of compliance with infection control measures in care of patients after cardiothoracic surgery. Journal of Hospital Infection. [doi: 10.1016/j.jhin.2005.05.016]. 2006;62(1):64-70.
Varkey P, Sathananthan A, Scheifer A, Bhagra S, Fujiyoshi A, Tom A, et al. Using quality-improvement techniques to enhance patient education and counselling of diagnosis and management. Quality in Primary Care. 2009;17(3):205-13.
Wheatland B, Porter C, Gilles M, Greenfield C, Larson A. Initiating a PDSA cycle-Improving management of diabetes in rural WA. Aust Fam Physician. 2006;35(8):650.
Wojciechowski E, Cichowski K. A Case Review: Designing a New Patient Education System. The Internet Journal of Advanced Nursing Practice. 2007;8(2).
Wolfenden J, Dunn A, Holmes A, Davies C, Buchan J. Track and trigger system for use in community hospitals. Nursing standard. 2010;24(45):35-9.
Zack J. Zeroing in on zero tolerance for central line-associated bacteremia. Am J Infect Control. 2008;36(10):S176.