Abstract:1. Researchers frequently take repeated measurements of individuals in a sample with the goal of quantifying the proportion of the total variation that can be attributed to variation among individuals vs. variation among measurements within individuals. The proportion of the variation attributed to variation among individuals is known as repeatability and is most frequently estimated as the intraclass correlation coefficient (ICC). The goal of our study is to provide guidelines for determining the sample size (number of individuals and number of measurements per individual) required to accurately estimate the ICC. 2. We report a range of ICCs from the literature and estimate 95% confidence intervals for these estimates. We introduce a predictive equation derived by Bonett (2002), and we test the assumptions of this equation through simulation. Finally, we create an R statistical package for the planning of experiments and estimation of ICCs. 3. Repeatability estimates were reported in 1·5% of the articles published in the journals surveyed. Repeatabilities tended to be highest when the ICC was used to estimate measurement error and lowest when it was used to estimate repeatability of behavioural and physiological traits. Few authors report confidence intervals, but our estimated 95% confidence intervals for published ICCs generally indicated a low level of precision associated with these estimates. This survey demonstrates the need for a protocol to estimate repeatability. 4. Analysis of the predictions from Bonett’s equation over a range of sample sizes, expected repeatabilities and desired confidence interval widths yields both analytical and intuitive guidelines for designing experiments to estimate repeatability. However, we find a tendency for the confidence interval to be underestimated by the equation when ICCs are high and overestimated when ICCs and the number of measurements per individual are low. 5. The sample size to use when estimating repeatability is a question pitting investigator effort against expected precision of the estimate. We offer guidelines that apply over a wide variety of ecological and evolutionary studies estimating repeatability, measurement error or heritability. Additionally, we provide the R package, icc, to facilitate analyses and determine the most economic use of resources when planning experiments to estimate repeatability.

A Review on the Theory and Application of ICCs

Intraclass correlation – A discussion and demonstration of basic features

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling

Approaches for Quantifying the ICC in Multilevel Logistic Models: A Didactic Demonstration

Rank Intraclass Correlation for Clustered Data

Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging.

Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra-Class Correlation Coefficient in Inter-Rater Reliability Studies

Measure Observability by the Generalized Informational Correlation.

Interrater reliability for multilevel data: A generalizability theory approach.

An inventory for assessing interpersonal communication competence of college students

Research on the Influences of English News Online Reading on Chinese College Students’ Intercultural Competence

Guidelines for estimating repeatability

Assumptions behind Intercoder Reliability Indices

Intercultural Communicative Competence Teaching and Assessment Based on Modern Information Technology

Liberal-Conservative Hierarchies of Intercoder Reliability Estimators

Human biomarker interpretation: the importance of intra-class correlation coefficients (ICC) and their calculations based on mixed models, ANOVA, and variance estimates

Intercoder Reliability for Use in Qualitative Research and Evaluation

An approach for selecting and using a method of inter-coder reliability in information management research

The Principles and Applications of ICCD

Chinese University Students’ Intercultural Competence