Percent Orders from Order Sets as a Marker of Clinical Standardization

Healthcare organizations looking to decrease care variation, improve efficiency, and augment quality care delivery track how orders are placed among practitioners. In particular, they can analyze how many orders are placed from curated order bundles like an order set. Quality improvement (QI) and informatics professionals, especially, know that metrics related to uptake of specialty-approved order sets can reflect standardized, high-quality care. In particular, the percent of all orders placed from an order bundle divided by all orders placed can guide order bundle creation and workflow optimizations. This metric reveals the adoption of standardized ordering processes and how well aligned they are with clinical workflows.

APPROACHES TO ANALYSIS: PROJECT-BASED VS. ORGANIZATION-WIDE

An analysis of lab and medication ordering can be done in two ways: project-based or organization-wide.

Tracking on a per-project basis how an order bundle is used, and running reports about which orders are being placed, provides an opportunity to understand if the electronic tools are being used as intended. The analysis can answers questions like:

• Is this order set being used?
• Is it being used by the right providers?
• Are the right orders being placed for the right patient?
  
  

Although it’s difficult to track this project-based information at scale without outside help, organizations will often pick a few priority initiatives like sepsis or venous thromboembolism (VTE) to understand if and how well their order build is used.

It’s less well known how order set adoption could be studied at an organizational level to assess how well a healthcare system is driving standardization. As a part of our Phrase Health EHR Benchmarking Study, we pushed organizations to evaluate how they’re standardizing care through order set usage as a whole. We helped them identify potential opportunities by comparing specialties, provider types, and many other views of their data.

We wanted to give these organizations a sense about which service lines might have the best opportunity to drive adoption compared to other organizations. This organization-wide view of data helps executives and quality improvement professionals answer questions like:

• How well are we pushing standardized processes among certain specialties and provider types?
• Do certain specialties or provider types need more education around the benefits of order sets?
• Do certain specialties or provider types need more resources to help build order sets that can be adopted to their workflow?

BENCHMARKING ANALYSIS

Our Phrase Health team compared overall order set and order panel adoption among our clients in Q1 2020 versus Q1 2021. We saw an increase from 33% in Q1 2020 to 37% in Q1 2021. One potential explanation for this change is that the COVID-19 pandemic developed an opportunity for standardized care among COVID-positive patients, which represented a relatively significant volume of overall patients compared to other periods. Phrase analysis determined that the greatest increase in orders placed from order sets took place in the Emergency departments (+59%). Furthermore, there were significant utilizations of COVID-19-related order sets for Phrase clients. We believe this supports the hypothesis that health systems encouraging standardization for COVID patients was an important reason for adoption increasing as a whole.

Overall Order Set and Order Panel Adoption Q1 '20 vs Q1 '21

We also looked for differences in order bundle adoption by specialty across these organizations, noting the highest and lowest levels. Average order set and order panel adoption varied by specialty, but tended to not exceed 50%.

Average % of Orders from Order Sets and Order Panels Q1 2021

This specialty comparison shows that OB/GYN and surgery tend to protocolize care resulting in more standardized ordering. By comparison, infectious disease, rheumatology, and other specialties are less standardized, but it’s unclear why. Though it’s difficult to generalize to a specialty, one would think that specialties with less widely adopted practice-based guidelines or a higher variety of patient disease treatment options may not fit well with standardized workflows. Additionally, this could be due to training and knowledge about the existence of available ordering bundles. Alternatively, some might not have the volume of patients, vocal champions, or other compelling reasons to spend resources on developing standardized ordering workflows.

COMPARING ORGANIZATIONS BY SPECIALTY

While average order set adoption in Q1 2021 occurs less than half of the time across most specialties, there is high variation in adoption across organizations. For instance, gastroenterology (GI) orders placed from order sets ranged from 4% to 70% among Phrase Health clients.

Highest Ordering Difference Across Specialties

Lowest Ordering Difference Across Specialties

This pattern highlights specialties where health systems have similar levels of standardized ordering. Further analysis would be needed to uncover the reasons why.

EXAMPLES FROM HEALTHCARE SYSTEMS

The following is an example of a healthcare system we guided through an order set uptake review, and the results.

EXAMPLE: DRIVING DOWN SEPSIS MORTALITY

A pediatric health system in a major U.S. city had a goal of driving down mortality from sepsis in the NICU. As part of this initiative, they reviewed the percent of orders placed from order sets for sepsis and sought to increase it. They were successful in driving order set compliance and considered that success a key reason they reduced sepsis mortality during that time. Sepsis order set compliance increased from 73% in July 2018 to more than 85% in July 2019.

FURTHER CONSIDERATIONS

While these metrics show promise for gaining insights and improving healthcare delivery, there are pitfalls to consider before using them to evaluate quality improvement performance. We’ll highlight two in particular here.

WHO USES THE ORDER SET?

At an individual level, although an organization might be able to roughly assess if an order set is adopted or not, it also needs to evaluate if an order set was used to target the patient population in question. For example, although a post-operative pain management order set was used several hundred times in the last month, was it being used for patients who recently came out of surgery? Some clinicians utilize order sets as a workaround for orders they’re not allowed to place individually, like expensive medications. Without a method or tool for linking utilization to outcomes, this would appear to artificially inflate utilization numbers.

PATIENT COMPLEXITY

Comparing health systems at an aggregated level is valuable to understand overall order bundle adoption, but results can be misleading without the right context. For example, if a hospital is renowned for a gastrointestinal subspecialty, it might rank low in adoption of order sets. However, this may be because the complexity of the treatment for that hospital’s patients is not covered by widely accepted guidelines and requires significant variation in care on a case-by-case basis. In this case, this workflow may necessitate personalized ordering lists and other shortcuts that aim to improve clinician efficiency rather than standardization.

SUMMARY

As health systems look for ways to improve the quality of patient care and clinician efficiency, the percent of all orders placed from an order bundle divided by all orders placed is a useful metric.

It highlights success in standardizing treatment for certain conditions, or comparing the level of standardization between specialties. Our benchmarking analysis between health systems showed variation in standardization between similar specialties. Further digging into these disparities could bring up information about the tools and approaches to standardization that are working best and why.

Care must be taken to consider context, however, including in specialties that lack widely accepted best practice guidelines or when patient complexity frequently deviates from these established guidelines.

Our Phrase Health team always encourages the effective use of EHR data to improve how health systems treat patients and ensure the highest level of care from providers.

ADDITIONAL PHRASE HEALTH REPORTS

Explore our EHR benchmarking study →

Explore our analysis of COVID data →