Abstract:1. Researchers frequently take repeated measurements of individuals in a sample with the goal of quantifying the proportion of the total variation that can be attributed to variation among individuals vs. variation among measurements within individuals. The proportion of the variation attributed to variation among individuals is known as repeatability and is most frequently estimated as the intraclass correlation coefficient (ICC). The goal of our study is to provide guidelines for determining the sample size (number of individuals and number of measurements per individual) required to accurately estimate the ICC. 2. We report a range of ICCs from the literature and estimate 95% confidence intervals for these estimates. We introduce a predictive equation derived by Bonett (2002), and we test the assumptions of this equation through simulation. Finally, we create an R statistical package for the planning of experiments and estimation of ICCs. 3. Repeatability estimates were reported in 1·5% of the articles published in the journals surveyed. Repeatabilities tended to be highest when the ICC was used to estimate measurement error and lowest when it was used to estimate repeatability of behavioural and physiological traits. Few authors report confidence intervals, but our estimated 95% confidence intervals for published ICCs generally indicated a low level of precision associated with these estimates. This survey demonstrates the need for a protocol to estimate repeatability. 4. Analysis of the predictions from Bonett’s equation over a range of sample sizes, expected repeatabilities and desired confidence interval widths yields both analytical and intuitive guidelines for designing experiments to estimate repeatability. However, we find a tendency for the confidence interval to be underestimated by the equation when ICCs are high and overestimated when ICCs and the number of measurements per individual are low. 5. The sample size to use when estimating repeatability is a question pitting investigator effort against expected precision of the estimate. We offer guidelines that apply over a wide variety of ecological and evolutionary studies estimating repeatability, measurement error or heritability. Additionally, we provide the R package, icc, to facilitate analyses and determine the most economic use of resources when planning experiments to estimate repeatability.

Sample size formulas for estimating intraclass correlation coefficients with precision and assurance

Review of sample size determination methods for the intraclass correlation coefficient in the one-way analysis of variance model

Revisiting sample size planning for receiver operating characteristic studies: A confidence interval approach with precision and assurance

Calculation of sample size guaranteeing the required width of the empirical confidence interval with predefined probability

Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra-Class Correlation Coefficient in Inter-Rater Reliability Studies

Practical Sample Size Calculations for Surveillance and Diagnostic Investigations

Determining Sample Size in Binary Measurement System

Sample size estimation in diagnostic test studies of biomedical informatics

Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve

Using the confidence interval confidently

Hybrid sample size calculations for cluster randomised trials using assurance

Cronbach's alpha reliability: Interval estimation, hypothesis testing, and sample size planning

Sample Size Determination in Survey Research

Guidelines for estimating repeatability

Sample size determination for interval estimation of the prevalence of a sensitive attribute under non‐randomized response models

Sample size planning for estimating the global win probability with precision and assurance

Sample Size Guideline for Correlation Analysis

Sample Size Determination for Interval Estimation of the Prevalence of a Sensitive Attribute Under Randomized Response Models

REQUIRED SAMPLE SIZE FOR CORRELATION ANALYSIS

Revisiting the Relationship Between Correlation Coefficient, Confidence Level, and Sample Size

Binomial Confidence Intervals for Rare Events: Importance of Defining Margin of Error Relative to Magnitude of Proportion