Criterion placement threatens the construct validity of neural measures of consciousness

Johannes Jacobus Fahrenfort,Philippa Anne Johnson,Niels A Kloosterman,Timo Stein,Simon van Gaal
DOI: https://doi.org/10.1101/2024.02.22.581517
2024-08-10
Abstract:How consciousness arises from brain activity has been a topic of intense scientific research for decades. But how does one identify the neural basis of something that is intrinsically personal and subjective? A hallmark approach has been to ask observers to judge stimuli as 'seen' (conscious) and 'unseen' (unconscious) and use post hoc sorting of neural measurements based these judgments. Unfortunately, cognitive and response biases are known to strongly affect how observers place their criterion for judging stimuli as 'seen' vs. 'unseen', thereby confounding neural measures of consciousness. Surprisingly however, the effect of conservative and liberal criterion placement on neural measures of unconscious and conscious processing has never been explicitly investigated. Here we use simulations and electrophysiological brain measurements to show that conservative criterion placement has an unintuitive consequence: rather than selectively providing a cautious estimate of conscious processing, it inflates effect sizes in neural measures of both conscious and unconscious processing, while liberal criterion placement does the reverse. After showing this in simulation, we performed decoding analyses on two electroencephalography studies that employ common subjective indicators of conscious awareness, in which we experimentally manipulated the response criterion. The results confirm that the predicted confounding effects of criterion placement on neural measures of unconscious and conscious processing occur in empirical data, while further showing that the most widely used subjective scale, the Perceptual Awareness Scale (PAS), does not guard against criterion confounds. Follow up simulations explicate how the experimental context determines whether the relative confounding effect of criterion placement is larger in neural measures of either conscious or unconscious processing. We conclude that criterion placement threatens the construct validity of neural measures of conscious and unconscious processing.
Neuroscience
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How does criterion placement in subjective measurement affect the construct validity of neural consciousness measurement?** Specifically, the author focuses on when studying consciousness, participants judge whether the stimulus is "seen" (conscious) or "not seen" (unconscious) based on their own perception. Such judgments are easily influenced by cognitive and response biases, leading to confusion in the results of neural measurements. However, the specific effects of conservative and liberal criterion placement on neural measurements of unconscious and conscious processing have never been clearly investigated. ### Main problems 1. **Effects of criterion placement**: - Does a conservative criterion placement conservatively estimate conscious processing, or does it simultaneously magnify the effects of both conscious and unconscious processing? - Does a liberal criterion placement have the opposite effect? 2. **Experimental verification**: - Use simulation and electroencephalogram (EEG) measurements to verify the specific effects of criterion placement on neural measurements. - Confirm whether widely - used subjective scales (such as the Perceptual Awareness Scale, PAS) can avoid criterion confusion. ### Methods - **Simulation experiment**: Based on Signal Detection Theory (SDT), analyze the influence on neural measurements by simulating the internal signal distributions under different criterion placements. - **Empirical study**: Conducted two EEG experiments, manipulated the decision criteria of participants, and analyzed their influence on neural measurements. ### Results - **Simulation results**: Conservative criterion placement will exaggerate the neural measurement effects of both conscious and unconscious processing, while liberal criterion placement will reduce these effects. - **Empirical results**: The two experiments confirmed the above - mentioned simulation results and further indicated that PAS cannot completely avoid criterion confusion. ### Conclusion Criterion placement threatens the construct validity of neural measurements in conscious and unconscious processing. Therefore, when using subjective measurements, potential biases caused by criterion placement must be considered to ensure the reliability and validity of research results. ### Formula explanation In Signal Detection Theory, the relationship between internal signal strength and response can be expressed by the following formula: \[ d'=\frac{\mu_S - \mu_N}{\sigma} \] where: - \( \mu_S \) is the mean of the signal - plus - noise distribution - \( \mu_N \) is the mean of the pure - noise distribution - \( \sigma \) is the standard deviation of the noise distribution The position of the response criterion can be expressed as: \[ c =-\frac{z(\text{criterion})\cdot\sigma}{\sqrt{2}} \] where \( z(\text{criterion}) \) is the standard score position. By adjusting \( c \), the conservatism or liberality of the response can be changed, thereby affecting subsequent neural measurement results.