New Study Evaluates the Quality of Medical Apps

Smartphones are everywhere and the number of apps that you can use on them is almost limitless. Just in terms of healthcare and medications, there are tens of thousands of apps-- including many that target medication-related problems.

But just how good are they? There are currently no standards or tests for quality that healthcare-related apps must pass, which means there is the risk that they may end up misleading consumers or providing unreliable information to healthcare professionals.

A team of researchers in Singapore set out to create a quality assessment tool that can be used to evaluate mobile apps that target medication-related problems. They then used this tool to assess 59 free and purchased or paid apps. Their findings were published in the Journal of Managed Care and Specialty Pharmacy in October.

The 59 apps that were evaluated had been screened from 400 apps related to health or medicine that were available through either the Apple ITunes Store or the Google Play Store, which offers apps for the Android operating system. The 400 apps were the top 100 paid and top 100 free apps available through Apple and the top 100 paid and top 100 free apps for Android. The apps were intended for either clinicians or consumers-- or both. The apps that were chosen offered monitoring features, medication information, dose calculators, medication records, or medication interaction checkers. The researchers created a tool that evaluated them for appropriateness, reliability, usability, and privacy.

The study found that paid Apple apps had higher overall quality compared to the free apps (58.1% versus 55.8%; P = 0.962), and that paid Android apps also had higher overall quality than free ones (59.4% versus 59.1%; P = 0.937), said Kevin Yi-Lwern Yap, BSc (Pharm), PhD, Lecturer in the Department of Pharmacy, Faculty of Science, at the National University of Singapore and an author of the study.  

Apps with an interaction checker and monitoring features had the highest and lowest overall qualities, 66.9% and 54.8%, respectively. Paid apps generally scored higher for usability than free apps (P = 0.006), but scored lower for privacy (P = 0.003). “Free apps scored higher for privacy than paid apps in this study because they are more likely to include privacy policies,” Yap noted in an email interview. “A possible reason could be that most paid apps were usually developed by individuals rather than organizations, hence they are less likely to take users’ privacy into account during app development.”

Blood pressure and heart rate monitoring apps had the highest overall quality scores, at 67.7%. Apps that monitored visual, hearing, and temperature changes scored the lowest, at 35.5%.

One of the important findings of the study was that the same app varied in functionality between the Apple and Android platforms, Yap noted. For example, the Android version of Epocrates contains an IV-drip rate calculator that was not available with the Apple version.

The study included a Table of the top scoring apps among those tested for each category. However, apps for smartphones are continually updated and enhanced and new ones are also continually being released, Yap pointed out. “Therefore there is a need to frequently assess the quality of apps to ensure that they remain within certain standards.”

Such validation can be done with a quality assessment tool like that developed by Yap and his colleagues or through a third-party certification system. One such system is run by the Health on the Net Foundation (https://www.healthonnet.org), a non-governmental organization devoted to identifying reliable sources of health information. “Such evaluations can help filter out useful and reliable apps from app stores,” Yap said.

The quality assessment tool that the Singapore group developed asks a series of questions about many aspects of each app, ranging from how well each does the job it was created to do to how reliable or unbiased it is--which includes whether the app states the source of funding or conflicts of interest. One section also evaluates the apps’ design, (eg, such as how intuitive it is to use, whether the design is well organized, and if the type is of a readable size). Each answer is scored with a 0, 1, or 2.

“We hope this tool is a more objective means for clinicians and reviewers to assess the quality of medical apps that are available before they use them in their practice settings or recommend them to patients,” Yap said. Potential developers of health care apps can also use this tool as a guide to create apps of a better quality standard for appropriate use in the health care setting.”

Valerie DeBenedette is a medical news writer in Putnam County, N.Y.