[Rheumatoid polyarthritis and deafness. Apropos of a case of oto-arthritis].

M. Hamidou,G. Morlock,R. Chalard,C. Cadène

Abstract:

What problem does this paper attempt to address?

A Suboptimal Model Explained the Spatiotemporal Control of Human Eye Movements in Visual Search

Yunhui Zhou,Yuguo Yu

DOI: https://doi.org/10.1101/779652

2019-01-01

bioRxiv

Abstract:Efficient eye movement planning in both space and time is vital for visual perception. Previous studies showed that human eye movements were consistent with an optimal searcher during visual search; however, we found that such a model could not fully explain the spatial control of eye movements and it completely ignored the temporal control aspect. Here, we first measured the temporal course of human visibility map, and then measured eye movement dynamics during a visual search task. We further built a continuous-time eye movement model that determined both fixation duration and location. When the participants’ saccade amplitude bias and memory capacity limitations were introduced, this model could describe multiple aspects of human visual search behavior much better than the optimal searcher with fixed fixation duration. Interestingly, this model predicted that the human visual memory capacity is approximately 8 fixations. The results showed that human performed suboptimal in visual search, and implied that human may use a mixed control strategy to select fixation location. This strategy may have adaptive advantages considering the limitations of the human brain and statistics of the natural environment.
Waldo Reveals Cultural Differences in Return Fixations

Patrick Luthold,Junpeng Lao,Lingnan He,Xinyue Zhou,Roberto Caldara

DOI: https://doi.org/10.1080/13506285.2018.1561567

2018-01-01

Visual Cognition

Abstract:ABSTRACT Humans routinely perform visual search towards targets to adapt to the environment. These sequences of ballistic eye movements are shaped by a combination of top–down and bottom–up factors. Recent research documented that human observers display cultural-specific fixation patterns in a range of visual processing tasks. In particular, eye movement strategies extracting information from faces clearly differs between Western Caucasian (WC) and East Asian (EA) observers. However, whether such cultural differences are also present for visual scene processing remains debated. To this aim, we recorded the eye movements of WC and EA observers while they were solving visual search problems parametrically varying in difficulty: Where’s Waldo. Both groups had a comparable familiarity with the Waldo books reaching a comparable level of accuracy in target detection. Both cultural groups also showed a comparable temporal effect on inhibition of return, with longer fixation durations when saccades were performed to a return location compared to other locations. Westerners, however, located Waldo faster than Easterners. Interestingly, this modulation of speed was likely related to differences occurring on the low-level mechanisms of spatial inhibition of return, with EA observers returning more often to previously visited locations than the WC observers. This suboptimal eye movement strategy in the Easterners might be engendered by their cultural perceptual bias consisting in⁠ a greater use of extra-foveal information. Overall, our data point towards the existence of a subtle, but significant difference in the processing of visual scenes across observers from different cultures during active visual search.
Energy-Efficient Visual Search by Eye Movement and Low-Latency Spiking Neural Network

Yunhui Zhou,Dongqi Han,Yuguo Yu

2023-10-10

Abstract:Human vision incorporates non-uniform resolution retina, efficient eye movement strategy, and spiking neural network (SNN) to balance the requirements in visual field size, visual resolution, energy cost, and inference latency. These properties have inspired interest in developing human-like computer vision. However, existing models haven't fully incorporated the three features of human vision, and their learned eye movement strategies haven't been compared with human's strategy, making the models' behavior difficult to interpret. Here, we carry out experiments to examine human visual search behaviors and establish the first SNN-based visual search model. The model combines an artificial retina with spiking feature extraction, memory, and saccade decision modules, and it employs population coding for fast and efficient saccade decisions. The model can learn either a human-like or a near-optimal fixation strategy, outperform humans in search speed and accuracy, and achieve high energy efficiency through short saccade decision latency and sparse activation. It also suggests that the human search strategy is suboptimal in terms of search speed. Our work connects modeling of vision in neuroscience and machine learning and sheds light on developing more energy-efficient computer vision algorithms.

Neural and Evolutionary Computing,Computer Vision and Pattern Recognition,Neurons and Cognition
Optimal Visual Search with Highly Heuristic Decision Rules

Anqi Zhang,Wilson S. Geisler

2024-09-26

Abstract:Visual search is a fundamental natural task for humans and other animals. We investigated the decision processes humans use when searching briefly presented displays having well-separated potential target-object locations. Performance was compared with the Bayesian-optimal decision process under the assumption that the information from the different potential target locations is statistically independent. Surprisingly, humans performed slightly better than optimal, despite humans' substantial loss of sensitivity in the fovea, and the implausibility of the human brain replicating the optimal computations. We show that three factors can quantitatively explain these seemingly paradoxical results. Most importantly, simple and fixed heuristic decision rules reach near optimal search performance. Secondly, foveal neglect primarily affects only the central potential target location. Finally, spatially correlated neural noise causes search performance to exceed that predicted for independent noise. These findings have far-reaching implications for understanding visual search tasks and other identification tasks in humans and other animals.

Neurons and Cognition,Computer Vision and Pattern Recognition,Applications
Control of fixation duration during visual search task execution

Alexander Vasilyev

DOI: https://doi.org/10.48550/arXiv.2004.10832

2020-04-22

Neurons and Cognition

Abstract:We study the ability of human observer to control fixation duration during execution of visual search tasks. We conducted the eye-tracking experiments with natural and synthetic images and found the dependency of fixation duration on difficulty of the task and the lengths of preceding and succeeding saccades. In order to explain it, we developed the novel control model of human eye-movements that incorporates continuous-time decision making, observation and update of belief state. This model is based on Partially Observable Markov Decision Process with delay in observation and saccade execution that accounts for a delay between eye and cortex. We validated the computational model through comparison of statistical properties of simulated and experimental eye-movement trajectories.
A normalization model of visual search predicts single trial human fixations in an object search task

Thomas Miconi,Laura Groomes,Gabriel Kreiman

DOI: https://doi.org/10.48550/arXiv.1404.6453

2014-04-25

Abstract:When searching for an object in a scene, how does the brain decide where to look next? Theories of visual search suggest the existence of a global attentional map, computed by integrating bottom-up visual information with top-down, target-specific signals. Where, when and how this integration is performed remains unclear. Here we describe a simple mechanistic model of visual search that is consistent with neurophysiological and neuroanatomical constraints, can localize target objects in complex scenes, and predicts single-trial human behavior in a search task among complex objects. This model posits that target-specific modulation is applied at every point of a retinotopic area selective for complex visual features and implements local normalization through divisive inhibition. The combination of multiplicative modulation and divisive normalization creates an attentional map in which aggregate activity at any location tracks the correlation between input and target features, with relative and controllable independence from bottom-up saliency. We first show that this model can localize objects in both composite images and natural scenes and demonstrate the importance of normalization for successful search. We next show that this model can predict human fixations on single trials, including error and target-absent trials. We argue that this simple model captures non-trivial properties of the attentional system that guides visual search in humans.

Neurons and Cognition
Simulating human saccadic scanpaths on natural images

Wei Wang, Cheng Chen, Yizhou Wang, Tingting Jiang, Fang Fang, Yuan Yao

DOI: https://doi.org/10.1109/CVPR.2011.5995423

2011-01-01

Computer Vision and Pattern Recognition

Abstract:Human saccade is a dynamic process of information pursuit. Based on the principle of information maximization, we propose a computational model to simulate human saccadic scanpaths on natural images. The model integrates three related factors as driven forces to guide eye movements sequentially - reference sensory responses, fovea-periphery resolution discrepancy, and visual working memory. For each eye movement, we compute three multi-band filter response maps as a coherent representation for the three factors. The three filter response maps are combined into multi-band residual filter response maps, on which we compute residual perceptual information (RPI) at each location. The RPI map is a dynamic saliency map varying along with eye movements. The next fixation is selected as the location with the maximal RPI value. On a natural image dataset, we compare the saccadic scanpaths generated by the proposed model and several other visual saliency-based models against human eye movement data. Experimental results demonstrate that the proposed model achieves the best prediction accuracy on both static fixation locations and dynamic scanpaths.
Predicting Visual Fixations

Matthias Kümmerer,Matthias Bethge

DOI: https://doi.org/10.1146/annurev-vision-120822-072528

2023-09-15

Abstract:As we navigate and behave in the world, we are constantly deciding, a few times per second, where to look next. The outcomes of these decisions in response to visual input are comparatively easy to measure as trajectories of eye movements, offering insight into many unconscious and conscious visual and cognitive processes. In this article, we review recent advances in predicting where we look. We focus on evaluating and comparing models: How can we consistently measure how well models predict eye movements, and how can we judge the contribution of different mechanisms? Probabilistic models facilitate a unified approach to fixation prediction that allows us to use explainable information explained to compare different models across different settings, such as static and video saliency, as well as scanpath prediction. We review how the large variety of saliency maps and scanpath models can be translated into this unifying framework, how much different factors contribute, and how we can select the most informative examples for model comparison. We conclude that the universal scale of information gain offers a powerful tool for the inspection of candidate mechanisms and experimental design that helps us understand the continual decision-making process that determines where we look.
A DNA reference laboratory for identification of isolates of Echinococcus (hydatid disease) organisms from Europe.

D. McManus

DOI: https://doi.org/10.1016/0035-9203(87)90193-3

1987-06-01

Transactions of the Royal Society of Tropical Medicine and Hygiene

Abstract:
[Effect of hypo- and hyperventilation on the electro-encephalogram].

F. Sakai,A. Sakuma,Y. Otsuka,Y. Saji,H. Kumagai

Abstract:
How do eyes and brain search a randomly structured uninformative scene? Exploiting a basic interplay of attention and memory

Lorenzo Resca,Pamela M. Greenwood,Terrance D. Keech

DOI: https://doi.org/10.48550/arXiv.2004.02873

2020-04-07

Abstract:We tracked the eye movements of seven young and seven older adults performing a conjunctive visual search task similar to that performed by two highly trained monkeys in an original influential study of Motter and Belky (1998a, 1998b). We obtained results consistent with theirs regarding elements of perception, selection, attention and object recognition, but we found a much greater role played by long-range memory. A design inadequacy in the original Motter-Belky study is not sufficient to explain such discrepancy, nor is the high level of training of their monkeys. Perhaps monkeys and humans do not use mnemonic resources compatibly already in basic visual search tasks, contrary to a common expectation, further supported by cortical representation studies. We also found age-related differences in various measures of eye movements, consistently indicating slightly reduced conspicuity areas for the older adults, hence, correspondingly reduced processing and memory capacities. However, because of sample size and age differential limitations, statistically significant differences were found only for a few variables, most notably overall reaction times. Results reported here provide the basis for demonstrating the formation of spiraling or circulating patterns in the eye movement trajectories and for developing corresponding computational models and simulations.

Neurons and Cognition
Gaze-based Human Intention Prediction in the Hybrid Foraging Search Task

Yunxian Pan,Jie Xu

DOI: https://doi.org/10.1016/j.neucom.2024.127648

IF: 6

2024-01-01

Neurocomputing

Abstract:An agent’s ability to predict a human’s intention can facilitate the effectiveness of a human-agent team. The aim of this study was to explore the feasibility of predicting human intention in the hybrid foraging search task using eye gaze data and machine learning. In the hybrid foraging search paradigm, the human performs searches in patches and decides to stay and exploit a patch or to leave and explore another patch. Researchers have modeled the “patch-leaving problem” on the collective level using the marginal value theorem. However, few have attempted to predict the exact moment when the human will leave. In the current study, 40 participants performed the hybrid foraging search task while eye gaze data were collected with an eye-tracker. The leaving intention was associated with larger pupil size, shorter average fixation duration, larger number of fixations, longer saccade amplitude, and faster saccade velocity compared to the searching intention. A cross-subject machine learning model with an artificial neural network algorithm was able to predict whether a participant would leave the current patch with up to 78% accuracy with a 2-second data analysis window. The length of the data analysis window did not significantly affect prediction accuracy. Furthermore, the earlier the behavior prediction was made, the lower the reliability of the prediction. These results demonstrate that eye gaze features and machine learning algorithms are useful in predicting human intention in visual search tasks.
Modeling Rational Adaptation of Visual Search to Hierarchical Structures

Saku Sourulahti,Christian P Janssen,Jussi PP Jokinen

2024-09-14

Abstract:Efficient attention deployment in visual search is limited by human visual memory, yet this limitation can be offset by exploiting the environment's structure. This paper introduces a computational cognitive model that simulates how the human visual system uses visual hierarchies to prevent refixations in sequential attention deployment. The model adopts computational rationality, positing behaviors as adaptations to cognitive constraints and environmental structures. In contrast to earlier models that predict search performance for hierarchical information, our model does not include predefined assumptions about particular search strategies. Instead, our model's search strategy emerges as a result of adapting to the environment through reinforcement learning algorithms. In an experiment with human participants we test the model's prediction that structured environments reduce visual search times compared to random tasks. Our model's predictions correspond well with human search performance across various set sizes for both structured and unstructured visual layouts. Our work improves understanding of the adaptive nature of visual search in hierarchically structured environments and informs the design of optimized search spaces.

Human-Computer Interaction
A computational account of transsaccadic attentional allocation based on visual gain fields

William J Harrison,Imogen Stead,Thomas S A Wallis,Peter J Bex,Jason B Mattingley

DOI: https://doi.org/10.1073/pnas.2316608121

2024-07-02

Abstract:Coordination of goal-directed behavior depends on the brain's ability to recover the locations of relevant objects in the world. In humans, the visual system encodes the spatial organization of sensory inputs, but neurons in early visual areas map objects according to their retinal positions, rather than where they are in the world. How the brain computes world-referenced spatial information across eye movements has been widely researched and debated. Here, we tested whether shifts of covert attention are sufficiently precise in space and time to track an object's real-world location across eye movements. We found that observers' attentional selectivity is remarkably precise and is barely perturbed by the execution of saccades. Inspired by recent neurophysiological discoveries, we developed an observer model that rapidly estimates the real-world locations of objects and allocates attention within this reference frame. The model recapitulates the human data and provides a parsimonious explanation for previously reported phenomena in which observers allocate attention to task-irrelevant locations across eye movements. Our findings reveal that visual attention operates in real-world coordinates, which can be computed rapidly at the earliest stages of cortical processing.
Predicting Human Saccadic Scanpaths Based on Iterative Representation Learning

Chen Xia,Junwei Han,Fei Qi,Guangming Shi

DOI: https://doi.org/10.1109/tip.2019.2897966

IF: 10.6

2019-01-01

IEEE Transactions on Image Processing

Abstract:Visual attention is a dynamic process of scene exploration and information acquisition. However, existing research on attention modeling has concentrated on estimating static salient locations. In contrast, dynamic attributes presented by saccade have not been well explored in previous attention models. In this paper, we address the problem of saccadic scanpath prediction by introducing an iterative representation learning framework. Within the framework, saccade can be interpreted as an iterative process of predicting one fixation according to the current representation and updating the representation based on the gaze shift. In the predicting phase, we propose a Bayesian definition of saccade to combine the influence of perceptual residual and spatial location on the selection of fixations. In implementation, we compute the representation error of an autoencoder-based network to measure perceptual residuals of each area. Simultaneously, we integrate saccade amplitude and center-weighted mechanism to model the influence of spatial location. Based on estimating the influence of two parts, the final fixation is defined as the point with the largest posterior probability of gaze shift. In the updating phase, we update the representation pattern for the subsequent calculation by retraining the network with samples extracted around the current fixation. In the experiments, the proposed model can replicate the fundamental properties of psychophysics in visual search. In addition, it can achieve superior performance on several benchmark eye-tracking data sets.
Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts

Florian Kadner,Tobias Thomas,David Hoppe,Constantin A. Rothkopf

DOI: https://doi.org/10.1109/WACV56688.2023.00214

2024-02-18

Abstract:The human prioritization of image regions can be modeled in a time invariant fashion with saliency maps or sequentially with scanpath models. However, while both types of models have steadily improved on several benchmarks and datasets, there is still a considerable gap in predicting human gaze. Here, we leverage two recent developments to reduce this gap: theoretical analyses establishing a principled framework for predicting the next gaze target and the empirical measurement of the human cost for gaze switches independently of image content. We introduce an algorithm in the framework of sequential decision making, which converts any static saliency map into a sequence of dynamic history-dependent value maps, which are recomputed after each gaze shift. These maps are based on 1) a saliency map provided by an arbitrary saliency model, 2) the recently measured human cost function quantifying preferences in magnitude and direction of eye movements, and 3) a sequential exploration bonus, which changes with each subsequent gaze shift. The parameters of the spatial extent and temporal decay of this exploration bonus are estimated from human gaze data. The relative contributions of these three components were optimized on the MIT1003 dataset for the NSS score and are sufficient to significantly outperform predictions of the next gaze target on NSS and AUC scores for five state of the art saliency models on three image data sets. Thus, we provide an implementation of human gaze preferences, which can be used to improve arbitrary saliency models' predictions of humans' next gaze targets.

Computer Vision and Pattern Recognition
Cortical dynamics of contextually cued attentive visual learning and search: Spatial and object evidence accumulation.

Tsung-Ren Huang,Stephen Grossberg

DOI: https://doi.org/10.1037/a0020664

IF: 8.2473

2010-01-01

Psychological Review

Abstract:How do humans use target-predictive contextual information to facilitate visual search? How are consistently paired scenic objects and positions learned and used to more efficiently guide search in familiar scenes? For example, humans can learn that a certain combination of objects may define a context for a kitchen and trigger a more efficient search for a typical object, such as a sink, in that context. The ARTSCENE Search model is developed to illustrate the neural mechanisms of such memory-based context learning and guidance and to explain challenging behavioral data on positive-negative, spatial-object, and local-distant cueing effects during visual search, as well as related neuroanatomical, neurophysiological, and neuroimaging data. The model proposes how global scene layout at a first glance rapidly forms a hypothesis about the target location. This hypothesis is then incrementally refined as a scene is scanned with saccadic eye movements. The model simulates the interactive dynamics of object and spatial contextual cueing and attention in the cortical What and Where streams starting from early visual areas through medial temporal lobe to prefrontal cortex. After learning, model dorsolateral prefrontal cortex (area 46) primes possible target locations in posterior parietal cortex based on goal-modulated percepts of spatial scene gist that are represented in parahippocampal cortex. Model ventral prefrontal cortex (area 47/12) primes possible target identities in inferior temporal cortex based on the history of viewed objects represented in perirhinal cortex.

psychology, multidisciplinary
Eye Movement Evidence for the V1 Saliency Hypothesis and the Central-peripheral Dichotomy Theory in an Anomalous Visual Search Task.

Junhao Liang,Severin Maher,Zhaoping Li

DOI: https://doi.org/10.1016/j.visres.2023.108308

IF: 1.8

2023-01-01

Vision Research

Abstract:Typically, searching for a target among uniformly tilted non-targets is easier when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH) - that V1 creates a saliency map to guide attention exogenously - predicts exactly the opposite in a special case: each target or non-target is a pair of equally-sized disks, a homo-pair of two disks of the same color, black or white, or a hetero-pair of two disks of the opposite color; the inter-disk displacement defines its orientation. This prediction - parallel advantage - was supported by the finding that parallel targets require shorter reaction times (RTs) to report targets' locations. Furthermore, it is stronger for targets further from the center of search images, as predicted by the Centralperipheral Dichotomy (CPD) theory entailing that saliency effects are stronger in peripheral than in central vision. However, the parallel advantage could arise from a shorter time required to recognize - rather than to shift attention to - the parallel target. By gaze tracking, the present study confirms that the parallel advantage is solely due to the RTs for the gaze to reach the target. Furthermore, when the gaze is sufficiently far from the target during search, saccade to a parallel, rather than perpendicular, target is more likely, demonstrating the Central-peripheral Dichotomy more directly. Parallel advantage is stronger among observers encouraged to let their search be guided by spontaneous gaze shifts, which are presumably guided by bottom-up saliency rather than top-down factors.
Look Twice: A Generalist Computational Model Predicts Return Fixations across Tasks and Species

Mengmi Zhang,Marcelo Armendariz,Will Xiao,Olivia Rose,Katarina Bendtz,Margaret Livingstone,Carlos Ponce,Gabriel Kreiman

DOI: https://doi.org/10.1371/journal.pcbi.1010654

2022-10-14

Abstract:Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.

Computer Vision and Pattern Recognition
Visual search as an embodied process: The effects of perspective change and external reference on search performance

Huiyuan Zhang,Jing Samantha Pan

DOI: https://doi.org/10.1167/jov.22.10.13

2022-09-02

Abstract:Traditional visual search tasks in the laboratories typically involve looking for targets in 2D displays with exemplar views of objects. In real life, visual search commonly entails 3D objects in 3D spaces with nonperpendicular viewing and relative motions between observers and search array items, both of which lead to transformations of objects' projected images in lawful but unpredicted ways. Furthermore, observers often do not have to memorize a target before searching, but may refer to it while searching, for example, holding a picture of someone while looking for them from a crowd. Extending the traditional visual search task, in this study, we investigated the effects of image transformation as a result of perspective change yielded by discrete viewing angle change (Experiment 1) or continuous rotation of the search array (Experiment 2) and of having external references on visual search performance. Results showed that when searching from 3D objects with a non-zero viewing angle, performance was similar to searching from 2D exemplar views of objects; when searching for 3D targets from rotating arrays in virtual reality, performance was similar to searching from stationary arrays. In general, discrete or continuous perspective change did not affect the search outcomes in terms of accuracy, response time, and self-rated confidence, or the search process in terms of eye movement patterns. Therefore, visual search does not require the exact match of retinal images. Additionally, being able to see the target during the search improved search accuracy and observers' confidence. It increased search time because, as revealed by the eye movements, observers actively checked back on the reference target. Thus, visual search is an embodied process that involves real-time information exchange between the observers and the environment.

[Rheumatoid polyarthritis and deafness. Apropos of a case of oto-arthritis].

A Suboptimal Model Explained the Spatiotemporal Control of Human Eye Movements in Visual Search

Waldo Reveals Cultural Differences in Return Fixations

Energy-Efficient Visual Search by Eye Movement and Low-Latency Spiking Neural Network

Optimal Visual Search with Highly Heuristic Decision Rules

Control of fixation duration during visual search task execution

A normalization model of visual search predicts single trial human fixations in an object search task

Simulating human saccadic scanpaths on natural images

Predicting Visual Fixations

A DNA reference laboratory for identification of isolates of Echinococcus (hydatid disease) organisms from Europe.

[Effect of hypo- and hyperventilation on the electro-encephalogram].

How do eyes and brain search a randomly structured uninformative scene? Exploiting a basic interplay of attention and memory

Gaze-based Human Intention Prediction in the Hybrid Foraging Search Task

Modeling Rational Adaptation of Visual Search to Hierarchical Structures

A computational account of transsaccadic attentional allocation based on visual gain fields

Predicting Human Saccadic Scanpaths Based on Iterative Representation Learning

Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts

Cortical dynamics of contextually cued attentive visual learning and search: Spatial and object evidence accumulation.

Eye Movement Evidence for the V1 Saliency Hypothesis and the Central-peripheral Dichotomy Theory in an Anomalous Visual Search Task.

Look Twice: A Generalist Computational Model Predicts Return Fixations across Tasks and Species

Visual search as an embodied process: The effects of perspective change and external reference on search performance