Abstract:Background: There are several mobile health (mHealth) apps in mobile app stores. These apps enter the business-to-customer market with limited controls. Both, apps that users use autonomously and those designed to be recommended by practitioners require an end-user validation to minimize the risk of using apps that are ineffective or harmful. Prior studies have reviewed the most relevant aspects in a tool designed for assessing mHealth app quality, and different options have been developed for this purpose. However, the psychometric properties of the mHealth quality measurement tools, that is, the validity and reliability of the tools for their purpose, also need to be studied. The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative has developed tools for selecting the most suitable measurement instrument for health outcomes, and one of the main fields of study was their psychometric properties. Objective: This study aims to address and psychometrically analyze, following the COSMIN guideline, the quality of the tools that are used to measure the quality of mHealth apps. Methods: From February 1, 2019, to December 31, 2019, 2 reviewers searched PubMed and Embase databases, identifying mHealth app quality measurement tools and all the validation studies associated with each of them. For inclusion, the studies had to be meant to validate a tool designed to assess mHealth apps. Studies that used these tools for the assessment of mHealth apps but did not include any psychometric validation were excluded. The measurement tools were analyzed according to the 10 psychometric properties described in the COSMIN guideline. The dimensions and items analyzed in each tool were also analyzed. Results: The initial search showed 3372 articles. Only 10 finally met the inclusion criteria and were chosen for analysis in this review, analyzing 8 measurement tools. Of these tools, 4 validated ≥5 psychometric properties defined in the COSMIN guideline. Although some of the tools only measure the usability dimension, other tools provide information such as engagement, esthetics, or functionality. Furthermore, 2 measurement tools, Mobile App Rating Scale and mHealth Apps Usability Questionnaire, have a user version, as well as a professional version. Conclusions: The Health Information Technology Usability Evaluation Scale and the Measurement Scales for Perceived Usefulness and Perceived Ease of Use were the most validated tools, but they were very focused on usability. The Mobile App Rating Scale showed a moderate number of validated psychometric properties, measures a significant number of quality dimensions, and has been validated in a large number of mHealth apps, and its use is widespread. It is suggested that the continuation of the validation of this tool in other psychometric properties could provide an appropriate option for evaluating the quality of mHealth apps.

Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT)

Application of Mixed Methods in Health Services Management Research: A Systematic Review

Physicians found an interactive tool displaying structured evidence summaries for multiple comparisons understandable and useful: a qualitative user testing study

A mixed methods analysis of existing assessment and evaluation tools (AETs) for mental health applications

The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review.

Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis

Tools for Evaluating the Content, Efficacy, and Usability of Mobile Health Apps According to the Consensus-Based Standards for the Selection of Health Measurement Instruments: Systematic Review

Comparison of evaluation methods for improving the usability of a Spanish mHealth tool

Optimising the value of the critical appraisal skills programme (CASP) tool for quality appraisal in qualitative evidence synthesis

Creation of an online inventory for choosing critical appraisal tools

Validation of the Mobile Application Rating Scale (MARS)

Comparison of the validity, perceived usefulness and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability: a study protocol

Usability Assessment Methods for Mobile Apps for Physical Rehabilitation: Umbrella Review

Quality and risk of bias appraisals of systematic reviews are inconsistent across reviewers and centers

Development of quality assessment tool for systematic reviews and meta-analyses of real-world studies: a Delphi consensus survey

The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study

Systematic review of applications and properties of the NoMAD instrument for assessing implementation outcomes: Study protocol

Comparison of the validity, perceived usefulness, and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability

AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

Essential concepts for effective mixed methods research in the health professions: AMEE Guide No. 173

A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing