Impact Assessment of CPOEs and CDSS: Systematic Review and Meta-Analysis

Welcome back! I say that to both you and myself. It’s the start of a new school year. While this blog was abandoned in the second half of last academic year, this year I am hoping to put more into this.

One of my subjects this year is MI 227: Clinical and Laboratory Information Systems. We were tasked to find an article the adoption or use of any of the following: (1) EMR system; (2) CPOE system; (3) medication administration system; (4) telemedicine system; (5) telehealth system; (6) PHR, or; (7) other clinical or laboratory system or application.

I came across this article by Prgomet, et. al published in the Journal of Medical Informatics Association entitled:

The article’s main objective was to determine the impact of computerized provider data entry (CPOE) systems and clinical decision support systems (CDSS) on the medication errors, length of stay (LOS), and ICU mortalities through a systematic review and meta-analysis. The authors searched for journals published between January 2000 and 2016 and focused on those that used commercial CPOE and CDSS. They reasoned that homegrown softwares were more likely to demonstrate positive effects on safety and quality of care. At the same time, however, they are becoming more increasingly difficult for organizations to maintain and it is likely that almost all future system implementations will involve commercial systems. Twenty studies fit the inclusion criteria. Based on the EPHPP quality assessment tool, thirteen (13) studies were rated as having moderate methodological quality while seven (7) were rated as weak.

The overall results were as follows:

• Significant reduction of medication error rate by 85% with introduction of CPOE (pooled RR: 0.15, 95% CI,, 0.03-0.80, P=0.03)
• No significant change in ICU LOS following the introduction of CPOE (pooled mean difference: -0.01, 95% CI, -0.81-0.60, P=0.7)
• Evidence of significant reduction in ICU mortality by 12% following introduction of CPOE (pooled RR: 0.89, 95% CI, 0.78 – 0.99, P=0.04)
• No significant association between CPOE introduction and hospital mortality (pooled RR: 1.17, 95% CI, 0.53-2.54, P=0.6)

Several points were tackled in the discussion portion. One is the small sample sizes in the studies involved, such that they may not be powered to detect a true effect. The authors recommended future studies assessing impact of CPOE and CDSS to include larger sample sizes, so that they will be sufficiently powered to accurately detect clinically relevant rate of change in important indicators following implementation of CPOE and/or CDSS. Such systems are a big investment, and establishing a good business case for the continued use should be evidence-based as well.

Another is the need to monitor outcomes once such systems are implemented, because while CPOEs and CDSS address some problems of paper-based systems, they also introduce new challenges. Physicians have always been notorious for having illegible handwriting, and this is one of the things CPOEs address. However, they may also be attributed to system-related errors, which are not present in paper-based system. Example of which are duplicate prescriptions and erroneous selection from dropdown menu. Aside from such errors, other identified outcomes from the use and implementation of CPOEs include the following: order delays due to inability to “pre-register” patients into the system, increased time to enter orders, reduction of staff interaction, and delays in medication administration. Because the switch from a paper-based system is expected to bring about several changes, it is therefore imperative for any institution to make a follow-up on the effects of the implementation and address the new challenges along the way.

Yet another point of discussion for this article was the lack of standardized definitions.. For example, “medication prescription error” had different operational definitions in the studies included. Missing weight or signature constituted an error in some studies, while these were not considered in others. For systematic reviews and meta-analyses to have a more robust conclusion, it would be ideal if the studies involved are more homogenous – and that includes measuring the same outcomes the same way.

Said learnings and recommendations above could be applied to the local setting. However, the problem is the fact that paper-based systems are still the more common practice here. As a nurse and a physician, I was able to train into two hospitals representing opposite poles in terms of healthcare in the Philippines – Philippine General Hospital and St. Luke’s Medical Center. I was a student nurse in PGH from 2005 to 2009 and there was no CPOE nor CDSS at the time. At present, the main hospital still uses paper-based system. SLMC, on the other hand, has more advanced HIS. However, the practice was more of a hybrid – physicians used paper-based systems which are then encoded by either the pharmacists or the nurses. If there were potential drug-drug interactions or contraindications, it was the pharmacist’s responsibility to alert the physician who requested the order. Currently, I work in a private multinational company and we have our own EMR with CPOE and CDSS capabilities. However, since our system is not interoperable with any pharmacies or outside outside clinics in the Philippines, the EMR serves more as a repository of data. Laboratory, diagnostic, and medication orders are encoded into the system after the patient is seen and after paper prescriptions or requests have been given to the patient. I am not privy to how other hospitals or companies work, and my search for literature specific to outcomes measurement after CPOE/CDSS implementation in the Philippines did not yield any results. I believe this is both bad and good. Bad – because it means there is much to be done when it comes to the adoption, implementation, and evaluation of such systems. Good – because it also means that we can benchmark on what other countries have done. Hopefully we can perform more standardized and large-scale impact assessments and consequently generate robust studies for which we can safely draw conclusions from.

—

Reference:

• Mirela Prgomet, Ling Li, Zahra Niazkhani, Andrew Georgiou, Johanna I Westbrook; Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis, Journal of the American Medical Informatics Association, Volume 24, Issue 2, 1 March 2017, Pages 413–422, https://doi.org/10.1093/jamia/ocw145

 

Clinical Decision Support Systems – How Can They Impact Healthcare?

Let’s skip the intro and dive right into this week’s assignment. The driving question that we needed to answer was…

“How can Clinical Decision Support Systems (CDSS) improve the quality of healthcare?”

Aside from the question above, we were tasked to think of a clinical scenario and suggest a clinical decision support system embedded within CHITS to address this.

A clinical decision support system or CDSS, according to Jaspers et. al (2010) is a tool that provides “clinicians or patients with computer-generated clinical knowledge and patient-related information, intelligently filtered or presented at appropriate times, to enhance patient care.” Another definition was provided by Souza et. al (2011) which describes CDSS as “computerized matching of an individual patient’s characteristics with a knowledge base that then provides patient specific recommendations to healthcare providers.” Common examples of CDSS are those that check for drug-drug interactions among the medications of a patient or those that check for possible drug-disease precautions or contraindications, reminder and recall systems for immunizations, prompts for  clinicians to initiate smoking cessation interventions, blood pressure screening, or laboratory/radiology testing among others. CDSS can exist independent of electronic medical records (EMRs) or may be incorporated into them. The idea is that CDSS will provide decision support to users at the time they make decisions, which should result in improved quality of care. For example, it can alert a physician that his/her medication of choice has a drug-drug interaction with a patient’s existing medication. Or it could also alert the healthcare practitioner (HCP) that his/her choice of medication can potentially aggravate a patient’s condition. Other examples are reminders such as the need for mammographic screening in women older than 40 or 45 years old, or the recommended vaccinations for an infant who is 14 weeks old. In the paper by Bates et. al (2003), they identified 10 “commandments” for an effective CDSS. I will enumerate and briefly explain them below.

1. Speed is everything. As mentioned above, the overall goal of CDSS is to provide decision support at the time the decision is being made. Bates et. al has found that the speed of an information system is the parameter that users value most.  A good IS, therefore, will less likely be seen as helpful if it takes time for decision support to be shown or accessed. I imagine this would be more applicable to settings wherein there is a long queue of patients who need attention.
2. Anticipate needs and deliver in real time. This pertains to making the necessary information available to clinicians at the time they need it. An example of which is emphasizing high potassium levels in a patient receiving a potassium-sparing diuretic in a patient with hypertension or congestive heart failure.
3. Fit into the user’s workflow. In one of my earlier blogs, I mentioned that one of involvement of the end users as early as the planning or design phase is one of the determinants for success of a health information system. Part of involving users is making sure that the proposed IS will work well with their current workflow. A new software (or CDSS in particular), even when equipped with excellent guidelines, will likely meet resistance when it significantly changes the workflow of the end users (unless the workflow is the actual problem and it NEEDS to be changed).
4. Little things can make a big difference. This sounds cliché to me but its applicability, even to CDSS, cannot be ignored. Little things in the CDSS can spell the success or failure of the system. For example, requiring a physician to enter his/her diagnosis using ICD codes (with a drop-down button) as opposed to entering it in free text will better help in ensuring that diagnoses for the patients are entered in a standard way. If part of the CDSS, then, is checking for drug-disease interactions, the system will be more accurate in delivering such information because the diagnoses are standardized.
5. Recognize that physicians will strongly resist stopping. For example, your CDSS tells the physician that no further tests are recommended for the patient. However, the physician has been used to ordering a chest x-ray for similar cases. Since the CDSS only mentioned to stop but did not offer an alternative, the physician will most likely override the said recommendation and continue to order the chest x-ray. The same is applicable for medications, such that giving no or an undesirable alternative to the physician prescribing the medication will most likely make him/her prescribe the medication he had intended to.  
6. Changing direction is easier than stopping. This is in relation to the previous number. An example of change would be changing the default dose and frequency of a drug when a physician orders it. Unless the physician has specific reason to change the dose and/or frequency, he/she will be more likely to use the suggested/default dose/frequency.
7. Simple interventions work best. An alternative to this statement would be less is more. The most important guidelines for a medication, for example, should fit in the HCP’s screen. It might be helpful to have it linked to a more comprehensive or complete list of guidelines, which should also be readily accessible to the clinician with a few clicks.
8. Ask for additional information only when you really need it. In CDSS that are integrated into the EMR, there are patient data that are already available for use by the CDSS to be able to give decision support. If the data is insufficient, the HCP will be asked to enter additional data. In stand-alone CDSS, most data will have to be entered by the HCP. According to Bates, et. al, “the likelihood of success in implementing a computerized guideline is inversely proportional to the number of extra data elements needed.” Therefore, it would be best if the least amount of data is asked of the HCP to be able to provide meaningful decision support, as a system that requires an HCP to enter several data will be something that a clinician is less likely to use, especially considering the usually limited amount of time HCPs have in dealing with their patients.
9. Monitor impact, get feedback, and respond. How likely are clinicians to accept the suggestions of the CDSS? How often do they overlook the suggestions? How do their actions impact process flow, or more importantly, patient care and patient outcomes? It is essential to know the answers to these, and make corrective actions even mid-course to improve the CDSS.
10. Manage and maintain your knowledge-based systems. CDSS suggestions are most often based on local/international guidelines. Since medicine is dynamic (and so are the guidelines), CDSS should also be able to keep up with the changes. A decision support with outdated guidelines will defeat the purpose. It is also helpful to track the response of the users to the CDSS and evaluate the reports on a regular basis.

Now that we know more about clinical decision support systems, let us discuss CHITS. CHITS stands for Community Health Information Tracking System. It was developed by the National Telehealth Center (NTHC) to improve health information management at the rural health unit level. It was described as an “EMR for the health workers, by the health workers” since it was developed alongside health workers and was designed in such a way that it features a workflow that is similar to what is currently being employed in local health centers. It uses an open source software making it more flexible to the needs of both the RHU and the DOH. It was built to improve data-gathering at the RHU level to be able to generate more accurate reports, which will be later used to plan programs and effect changes on both local and national public health levels.

I was a student nurse rotating in the local health centers (in Metro Manila and Batangas) from 2006-2009 and I was a medical intern doing community rotation from 2012-2014. I have never encountered CHITS in any of the health centers I rotated at. There is limited information on what CHITS is and what is available so far (based on my internet search), so I will assume that my suggestions below are still non-existent.

In this scenario, barangay health worker (BHW) Dela Cruz is on duty at a rural health center. It’s vaccination day and Mommy Anna comes in with her children Karen and Nina, who are 2 years old and 9 months old, respectively. BHW Dela Cruz  briefly interviews Mommy Anna and gets pertinent data regarding her children. During the interview, she found out that Mommy Anna had just moved into the barangay and that her daughters have never received any type of vaccination. She was then asked by the mother which vaccines her daughters should receive. In the brief period that Dela Cruz has been a BHW, she has never actually encountered an infant or toddler who is naive to any type of vaccination.

I think that a useful CDSS in this scenario would be something that guides a healthcare worker (HCW) on the vaccinations a patient, particularly a pediatric patient, should receive. The minimum data required would be the patient’s birthday (the age should be computed automatically). When the software determines that the patient is a pediatric patient, the recommended vaccinations for patient based on age should be displayed. The HCW can then enter the date the vaccines were received (if any and if they were received elsewhere). It is at this point that any previous reactions to vaccines should also be documented. If there are none, as in the scenario above, the CDSS should show the recommended vaccines arranged by priority. The basis for the recommendation would be the guidelines based on DOH’s expanded program on immunization (EPI). Contraindications for vaccines in general or for a specific vaccine could also be shown. A trained barangay health worker (BHW) should be able to do a simple assessment, enter the birthdate, and identify which vaccinations a patient should receive. Once the vaccine has been administered, it is documented in the software. Ideally, if the vaccine is part of a series, the date for the next dose should be shown so that the BHW could inform the patient (or his/her guardian) on when to come back. The same CDSS should also be able to generate reports such as the type and quantity of vaccinations given on a daily, weekly, monthly, quarterly or annual basis or how many children are considered fully immunized. These types of reports could be married with statistics from the local government to determine the vaccine coverage in the area to assist in procurement of additional vaccines and to help plan for program that are related to vaccination.

Again, I have to admit that I am not aware if this is already being done in the RHUs. Considering the lack of healthcare professionals in the rural health units, particularly the geographically isolated and disadvantaged areas, this type of CDSS would be very helpful to the RHU healthcare workers and in helping attain the main goal of DOH’s EPI, which is to reduce the mortality and morbidity among children against the most common vaccine-preventable diseases.

I personally am using a vaccination application on my smart phone that was created by the Centers for Disease Control (CDC). It has recommendations based on age group or based on special populations (pregnant, person with HIV, etc.). As a standalone app, it has been very helpful to my in my practice and for my own personal use. Something similar could be incorporated to CHITS to help implement the immunization program of DOH.

That is it for week 11 of HI 201! Can you think of other clinical decision support systems that could benefit people especially in the remote areas where there is often limited access to healthcare? What do you think are pitfalls of CDSS?

As always, let me know in the comments below. I would love to hear and learn from you!

Love,
Eve

---

References:

• Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality by Bates, et. al (2003)
• Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings by Jaspers, et. al (2011)
• Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes by Souza (2011)
• Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study by Ash, et. al (2012)
• Community Health Information Tracking System by National Telehealth Center
• Expanded Program on Immunization by the DOH