Abstract:<p>Method comparison studies are essential for development in medical and clinical fields. These studies often compare a cheaper, faster, or less invasive measuring method with a widely used one to see if they have sufficient agreement for interchangeable use. Moreover, unlike simply reading measurements from devices, eg, reading body temperature from a thermometer, the response measurement in many clinical and medical assessments is impacted not only by the measuring device but also by the rater. For example, widespread inconsistencies are commonly observed among raters in psychological or cognitive assessment studies due to different characteristics such as rater training and experience, especially in large‐scale assessment studies when many raters are employed. This paper proposes a model‐based approach to assess agreement of two measuring methods for paired repeated binary measurements under the scenario where the agreement between two measuring methods and the agreement among raters are required to be studied simultaneously. Based upon the generalized linear mixed models (GLMMs), the decision on the adequacy of interchangeable use is made by testing the equality of fixed effects of methods. Approaches for assessing method agreement, such as the Bland‐Altman diagram and Cohen's kappa, are also developed for repeated binary measurements based upon the latent variables in GLMMs. We assess our novel model‐based approach by simulation studies and a real clinical application, in which patients are evaluated repeatedly for delirium with two validated screening methods. Both the simulation studies and the real data analyses demonstrate that our proposed approach can effectively assess method agreement.</p>

Assessing Agreement Between Two Methods of Quantitative Measurements: Exact Test Procedure and Sample Size Calculation

Sample Size for Assessing Agreement Between Two Methods of Measurement by Bland-Altman Method.

Sample Size for AssessingAgreement between TwoMethods ofMeasurement byBland − Altman Method

Assessing agreement between two measurement systems: An alternative to the limits of agreement approach

Statistical Analysis of Noninferiority Trials with a Rate Ratio in Small-Sample Matched-Pair Designs.

Statistical methods for conducting agreement (comparison of clinical tests) and precision (repeatability or reproducibility) studies in optometry and ophthalmology

Determining Sample Size in Binary Measurement System

Assessing method agreement for paired repeated binary measurements administered by multiple raters

Unconditional Exact Tests for Equivalence or Noninferiority for Paired Binary Endpoints

A Primer on the Use of Equivalence Testing for Evaluating Measurement Agreement

Statistical models for assessing agreement for quantitative data with heterogeneous random raters and replicate measurements

Sample Size Determination for Establishing Equivalence/Noninferiority Via Ratio of Two Proportions in Matched–Pair Design

On Power and Sample Size Computation for Multiple Testing Procedures

Exact Tests for Comparing Two Paired Proportions with Incomplete Data

A noniterative sample size procedure for tests based on t distributions

Testing two variances for superiority/non-inferiority and equivalence: Using the exhaustion algorithm for sample size allocation with cost

Review of sample size determination methods for the intraclass correlation coefficient in the one-way analysis of variance model

Sample size determination for establishing equivalence/noninferiority via ratio of two proportions in matched-pair design.

Testing the Equality of Proportions for Correlated Otolaryngologic Data

A Note on Sample Size Calculation for Mean Comparisons Based on Noncentral T-Statistics.

Finite sample corrections for average equivalence testing