Impact Assessment of CPOEs and CDSS: Systematic Review and Meta-Analysis

Welcome back! I say that to both you and myself. It’s the start of a new school year. While this blog was abandoned in the second half of last academic year, this year I am hoping to put more into this.

One of my subjects this year is MI 227: Clinical and Laboratory Information Systems. We were tasked to find an article the adoption or use of any of the following: (1) EMR system; (2) CPOE system; (3) medication administration system; (4) telemedicine system; (5) telehealth system; (6) PHR, or; (7) other clinical or laboratory system or application.

I came across this article by Prgomet, et. al published in the Journal of Medical Informatics Association entitled:

Screen Shot 2017-08-10 at 10.30.54 PM.png

The article’s main objective was to determine the impact of computerized provider data entry (CPOE) systems and clinical decision support systems (CDSS) on the medication errors, length of stay (LOS), and ICU mortalities through a systematic review and meta-analysis. The authors searched for journals published between January 2000 and 2016 and focused on those that used commercial CPOE and CDSS. They reasoned that homegrown softwares were more likely to demonstrate positive effects on safety and quality of care. At the same time, however, they are becoming more increasingly difficult for organizations to maintain and it is likely that almost all future system implementations will involve commercial systems. Twenty studies fit the inclusion criteria. Based on the EPHPP quality assessment tool, thirteen (13) studies were rated as having moderate methodological quality while seven (7) were rated as weak.

The overall results were as follows:

  • Significant reduction of medication error rate by 85% with introduction of CPOE (pooled RR: 0.15, 95% CI,, 0.03-0.80, P=0.03)
  • No significant change in ICU LOS following the introduction of CPOE (pooled mean difference: -0.01, 95% CI, -0.81-0.60, P=0.7)
  • Evidence of significant reduction in ICU mortality by 12% following introduction of CPOE (pooled RR: 0.89, 95% CI, 0.78 – 0.99, P=0.04)
  • No significant association between CPOE introduction and hospital mortality (pooled RR: 1.17, 95% CI, 0.53-2.54, P=0.6)

Several points were tackled in the discussion portion. One is the small sample sizes in the studies involved, such that they may not be powered to detect a true effect. The authors recommended future studies assessing impact of CPOE and CDSS to include larger sample sizes, so that they will be sufficiently powered to accurately detect clinically relevant rate of change in important indicators following implementation of CPOE and/or CDSS. Such systems are a big investment, and establishing a good business case for the continued use should be evidence-based as well.

Another is the need to monitor outcomes once such systems are implemented, because while CPOEs and CDSS address some problems of paper-based systems, they also introduce new challenges. Physicians have always been notorious for having illegible handwriting, and this is one of the things CPOEs address. However, they may also be attributed to system-related errors, which are not present in paper-based system. Example of which are duplicate prescriptions and erroneous selection from dropdown menu. Aside from such errors, other identified outcomes from the use and implementation of CPOEs include the following: order delays due to inability to “pre-register” patients into the system, increased time to enter orders, reduction of staff interaction, and delays in medication administration. Because the switch from a paper-based system is expected to bring about several changes, it is therefore imperative for any institution to make a follow-up on the effects of the implementation and address the new challenges along the way.

Yet another point of discussion for this article was the lack of standardized definitions.. For example, “medication prescription error” had different operational definitions in the studies included. Missing weight or signature constituted an error in some studies, while these were not considered in others. For systematic reviews and meta-analyses to have a more robust conclusion, it would be ideal if the studies involved are more homogenous – and that includes measuring the same outcomes the same way.

Said learnings and recommendations above could be applied to the local setting. However, the problem is the fact that paper-based systems are still the more common practice here. As a nurse and a physician, I was able to train into two hospitals representing opposite poles in terms of healthcare in the Philippines – Philippine General Hospital and St. Luke’s Medical Center. I was a student nurse in PGH from 2005 to 2009 and there was no CPOE nor CDSS at the time. At present, the main hospital still uses paper-based system. SLMC, on the other hand, has more advanced HIS. However, the practice was more of a hybrid – physicians used paper-based systems which are then encoded by either the pharmacists or the nurses. If there were potential drug-drug interactions or contraindications, it was the pharmacist’s responsibility to alert the physician who requested the order. Currently, I work in a private multinational company and we have our own EMR with CPOE and CDSS capabilities. However, since our system is not interoperable with any pharmacies or outside outside clinics in the Philippines, the EMR serves more as a repository of data. Laboratory, diagnostic, and medication orders are encoded into the system after the patient is seen and after paper prescriptions or requests have been given to the patient. I am not privy to how other hospitals or companies work, and my search for literature specific to outcomes measurement after CPOE/CDSS implementation in the Philippines did not yield any results. I believe this is both bad and good. Bad – because it means there is much to be done when it comes to the adoption, implementation, and evaluation of such systems. Good – because it also means that we can benchmark on what other countries have done. Hopefully we can perform more standardized and large-scale impact assessments and consequently generate robust studies for which we can safely draw conclusions from.

Reference:

  • Mirela Prgomet, Ling Li, Zahra Niazkhani, Andrew Georgiou, Johanna I Westbrook; Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis, Journal of the American Medical Informatics Association, Volume 24, Issue 2, 1 March 2017, Pages 413–422, https://doi.org/10.1093/jamia/ocw145

 

Advertisements