Experimental Tests of Subjective Bayesian Methods

Yuelin Li,David H. Krantz
DOI: https://doi.org/10.1007/bf03395509
2005-01-01
Abstract:We evaluated Samaniego and Reneau's 1994 novel weight method for eliciting subjective probability estimates. Experiment 1 replicated their experiment (subjects weighed their prior estimate against 10 new observations), with an additional weight judgment against 50 observations. In Experiment 2, subjects gave prior estimates to questions in a domain more familiar to them. In Experiment 3, subjects weighed priors against samples of 10 or 50 on a modified scale. We find substantial individual differences in sensitivity to sample size in this task, and also evidence for implicit learning: Subjects who first judged N = 50 subsequently place lower weights on N = 10. People cannot evaluate the quality of their prior knowledge accurately, nor can they consistently produce calibrated priors and marginal posteriors. These results do not support the use of the weight-judgment method for parameter estimation, but do encourage its continued development and improvement.
What problem does this paper attempt to address?