Watching the watchers: camera identification and characterization using retro-reflections

Trevor Seets,Alec Epstein,Andreas Velten

DOI: https://doi.org/10.1364/oe.520545

IF: 3.8

2024-03-29

Optics Express

Abstract:Trevor Seets, Alec Epstein, Andreas Velten A focused imaging system such as a camera will reflect light directly back at a light source in a retro-reflection (RR) or cat-eye ... [Opt. Express 32, 13836-13850 (2024)]

optics

What problem does this paper attempt to address?

A Single-Camera Gaze Tracking System under Natural Light

Feng Xiao,Dandan Zheng,Kejie Huang,Yue Qiu,Haibin Shen

DOI: https://doi.org/10.16910/jemr.11.4.5

IF: 1.3489

2018-01-01

Journal of Eye Movement Research

Abstract:Gaze tracking is a human-computer interaction technology, and it has been widely studied in the academic and industrial fields. However, constrained by the performance of the specific sensors and algorithms, it has not been popularized for everyone. This paper proposes a single-camera gaze tracking system under natural light to enable its versatility. The iris center and anchor point are the most crucial factors for the accuracy of the system. The accurate iris center is detected by the simple active contour snakuscule, which is initialized by the prior knowledge of eye anatomical dimensions. After that, a novel anchor point is computed by the stable facial landmarks. Next, second-order mapping functions use the eye vectors and the head pose to estimate the points of regard. Finally, the gaze errors are improved by implementing a weight coefficient on the points of regard of the left and right eyes. The feature position of the iris center achieves an accuracy of 98.87% on the GI4E database when the normalized error is lower than 0.05. The accuracy of the gaze tracking method is superior to the-state-of-the-art appearance-based and feature-based methods on the EYEDIAP database.
A new class of Ornstein transformations with singular spectrum

El Houcein El Abdalaoui,François Parreau,A. A. Prikhod'Ko

DOI: https://doi.org/10.48550/arXiv.math/0502274

2005-02-14

Abstract:It is shown that for any family of probability measures in Ornstein type constructions the corresponding transformation has almost surely a singular spectrum. This is a new generalization of Bourgain's theorem, the same result is proved for Rudolph's construction.

Dynamical Systems
Recent Advances on Apposition Compound Eye Cameras

Y. M. Song,Y. Xie,V. Malyarchuk,J. Xiao,I. Jung,K.-J. Choi,Z. Liu,H. Park,C. Lu,R.-H. Kim,R. Li,K. B. Crozier,Y. Huang,J. A. Rogers

DOI: https://doi.org/10.1364/cleo_at.2013.ath5a.5

2013-01-01

Abstract:This work presents digital cameras that resemble, both structurally and functionally, apposition-type compound eyes of daylight insect. Hemispherical imagers with 180 artificial ommatidia yield 160 degree field of view and nearly infinite depth-of-field.
Whole-viewing-angle cat's-eye retroreflector as a target of laser trackers

Toshiyuki Takatsuji,Mitsuo Goto,Sonko Osawa,Ruimin Yin,Tomizo Kurosawa

DOI: https://doi.org/10.1088/0957-0233/10/7/403

IF: 2.398

1999-01-01

Measurement Science and Technology

Abstract:A spherical cat's-eye retroreflector made from a glass material having a refractive index of two was developed. The geometrical sphericity of the cat's eye and the sphericity examined optically were approximately 100 nm and 500 nm respectively. The discrepancy between these two values is discussed. This optical device is very versatile in terms of applications since it is free from viewing angle restrictions.

engineering, multidisciplinary,instruments & instrumentation
Arthropod Eye-Inspired Digital Camera With Unique Imaging Characteristics

Jianliang Xiao,Young Min Song,Yizhu Xie,Viktor Malyarchuk,Inhwa Jung,Ki-Joong Choi,Zhuangjian Liu,Hyunsung Park,Chaofeng Lu,Rak-Hwan Kim,Rui Li,Kenneth B. Crozier,Yonggang Huang,John A. Rogers

DOI: https://doi.org/10.1117/12.2050820

2014-01-01

Abstract:In nature, arthropods have a remarkably sophisticated class of imaging systems, with a hemispherical geometry, a wide-angle field of view, low aberrations, high acuity to motion and an infinite depth of field. There are great interests in building systems with similar geometries and properties due to numerous potential applications. However, the established semiconductor sensor technologies and optics are essentially planar, which experience great challenges in building such systems with hemispherical, compound apposition layouts. With the recent advancement of stretchable optoelectronics, we have successfully developed strategies to build a fully functional artificial apposition compound eye camera by combining optics, materials and mechanics principles. The strategies start with fabricating stretchable arrays of thin silicon photodetectors and elastomeric optical elements in planar geometries, which are then precisely aligned and integrated, and elastically transformed to hemispherical shapes. This imaging device demonstrates nearly full hemispherical shape (about 160 degrees), with densely packed artificial ommatidia. The number of ommatidia (180) is comparable to those of the eyes of fire ants and bark beetles. We have illustrated key features of operation of compound eyes through experimental imaging results and quantitative ray-tracing-based simulations. The general strategies shown in this development could be applicable to other compound eye devices, such as those inspired by moths and lacewings (refracting superposition eyes), lobster and shrimp (reflecting superposition eyes), and houseflies (neural superposition eyes).
Geometric properties of multiple reflections in catadioptric camera with two planar mirrors

Xianghua Ying,Kun Peng,Ren Ren,Hongbin Zha

DOI: https://doi.org/10.1109/CVPR.2010.5540088

2010-01-01

Abstract:A catadioptric system consisting of a pinhole camera and two planar mirrors is deeply investigated in this paper. The two mirrors combine to form a corner and face-to-face with the pinhole. Their relative pose is unknown. An object will be reflected in the mirror corner one-time or multiple-times. Using the pinhole, we may take an image containing the object and its reflections, i.e., simultaneously imaging multiple views of an object by a single camera. We discovered that each 3D point and its reflections lie on a circle. We call the point set composed of a 3D point and its reflections, a Reflection Point Group (RPG), and the circle related to a RPG is called a RPG circle. All RPG circles are parallel to one another. Furthermore, each RPG can be partitioned into two separate subgroups. Shape formed by all points in a subgroup is invariant with respect to location of the 3D point. From these geometric properties, two calibration approaches can be utilized: One is based on parallel circles; the other is using 2D homographies among invariant shapes. Experiments validate our approaches.
Dynamically Tunable Hemispherical Electronic Eye Camera System with Adjustable Zoom Capability

Inhwa Jung,Jianliang Xiao,Viktor Malyarchuk,Chaofeng Lu,Ming Li,Zhuangjian Liu,Jongseung Yoon,Yonggang Huang,John A. Rogers

DOI: https://doi.org/10.1073/pnas.1015440108

2011-01-01

Abstract:Imaging systems that exploit arrays of photodetectors in curvilinear layouts are attractive due to their ability to match the strongly nonplanar image surfaces (i.e., Petzval surfaces) that form with simple lenses, thereby creating new design options. Recent work has yielded significant progress in the realization of such "eyeball" cameras, including examples of fully functional silicon devices capable of collecting realistic images. Although these systems provide advantages compared to those with conventional, planar designs, their fixed detector curvature renders them incompatible with changes in the Petzval surface that accompany variable zoom achieved with simple lenses. This paper describes a class of digital imaging device that overcomes this limitation, through the use of photodetector arrays on thin elastomeric membranes, capable of reversible deformation into hemispherical shapes with radii of curvature that can be adjusted dynamically, via hydraulics. Combining this type of detector with a similarly tunable, fluidic plano-convex lens yields a hemispherical camera with variable zoom and excellent imaging characteristics. Systematic experimental and theoretical studies of the mechanics and optics reveal all underlying principles of operation. This type of technology could be useful for night-vision surveillance, endoscopic imaging, and other areas that require compact cameras with simple zoom optics and wide-angle fields of view.
Digital Cameras with Designs Inspired by the Arthropod Eye

Young Min Song,Yizhu Xie,Viktor Malyarchuk,Jianliang Xiao,Inhwa Jung,Ki-Joong Choi,Zhuangjian Liu,Hyunsung Park,Chaofeng Lu,Rak-Hwan Kim,Rui Li,Kenneth B. Crozier,Yonggang Huang,John A. Rogers

DOI: https://doi.org/10.1038/nature12083

IF: 64.8

2013-01-01

Nature

Abstract:Digital cameras with layouts inspired by the compound, hemispherical designs of arthropod eyes have been built by combining elastomeric optical elements with deformable arrays of thin silicon photodetectors.
Self-Calibration of Catadioptric Camera with Two Planar Mirrors from Silhouettes

Xianghua Ying,Kun Peng,Yongbo Hou,Sheng Guan,Jing Kong,Hongbin Zha

DOI: https://doi.org/10.1109/tpami.2012.195

IF: 23.6

2013-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:If an object is interreflected between two planar mirrors, we may take an image containing both the object and its multiple reflections, i.e., simultaneously imaging multiple views of an object by a single pinhole camera. This paper emphasizes the problem of recovering both the intrinsic and extrinsic parameters of the camera using multiple silhouettes from one single image. View pairs among views in a single image can be divided into two kinds by the relationship between the two views in the pair: reflected by some mirror (real or virtual) and in a circular motion. Epipoles in the first kind of pairs can be easily determined from intersections of common tangent lines of silhouettes. Based on the projective properties of these epipoles, efficient methods are proposed to recover both the imaged circular points and the included angle between two mirrors. Epipoles in the second kind of pairs can be recovered simultaneously with the projection of intersection line between two mirrors by solving a simple 1D optimization problem using the consistency constraint of epipolar tangent lines. Fundamental matrices among views in a single image are all recovered. Using the estimated intrinsic and extrinsic parameters of the camera, a euclidean reconstruction can be obtained. Experiments validate the proposed approach.
Mechanics of Tunable Hemispherical Electronic Eye Camera Systems That Combine Rigid Device Elements with Soft Elastomers

Chaofeng Lu,Ming Li,Jianliang Xiao,Inhwa Jung,Jian Wu,Yonggang Huang,Keh-Chih Hwang,John A. Rogers

DOI: https://doi.org/10.1115/1.4023962

2013-01-01

Journal of Applied Mechanics

Abstract:A tunable hemispherical imaging system with zoom capability was recently developed by exploiting heterogeneous integration of rigid silicon photodetectors on soft, elastomeric supports, in designs that can facilitate tunable curvature for both the lens and detector. This paper reports analytical mechanics models for the soft materials aspects of the tunable lenses and detector surfaces used in such devices. The results provide analytical expressions for the strain distributions, apex heights and detector positions, and have been validated by the experiments and finite element analysis. More broadly, they represent important design tools for advanced cameras that combine hard and soft materials into nonplanar layouts with adjustable geometries.
Simulation Of Imaging And Refocusing For Light Field Photography

Han Huang,Xiaotong Li,Zhaofeng Cen

DOI: https://doi.org/10.1117/12.2037128

2013-01-01

Abstract:The entire process of image formation and refocusing of light field photography with microlens array is simulated using the in-house software OTS based on real ray tracing. The optical system consists of a photographic lens and the microlens array which are specifically designed. Objects are placed in different distance before the lens, and images of them on the detector plane are got by imaging simulation algorithm. Utilizing the color, intensity and the corresponding direction information of light rays on the image plane, pictures focusing at different depth in the image space are calculated, so that we can refocus at different objects in different distance. The paper also details the basic requirement of optical system and this simulation process is a fundamental work of research on design method and performance of lens using in light field photography.
High-speed target tracking control system based on short-time rotational reflection imaging

Haoran Liu,Jiehu Kang,Luyuan Feng,Jianwei Zhang,Fei Fan,Zhen Zhang,Zefeng Sun,Bin Wu

DOI: https://doi.org/10.1364/ao.514800

IF: 1.9

2024-01-26

Applied Optics

Abstract:Haoran Liu, Jiehu Kang, Luyuan Feng, Jianwei Zhang, Fei Fan, Zhen Zhang, Zefeng Sun, Bin Wu High-speed tracking technology has wide applications in the military and aerospace industry. However, existing approaches, such as camera ... [Appl. Opt. 63, 1094-1104 (2024)]

optics
Real-time ghost image characterization for panoramic annular lenses

Yiqi Liao,Fang Ke,Yiru Pan,Shaohua Gao,Jian Bai,Kaiwei Wang

DOI: https://doi.org/10.1364/oe.525648

IF: 3.8

2024-06-08

Optics Express

Abstract:Yiqi Liao, Fang Ke, Yiru Pan, Shaohua Gao, Jian Bai, Kaiwei Wang As a common but troublesome branch of stray light, ghost images in catadioptric optical systems greatly limit the performances of such ... [Opt. Express 32, 22998-23014 (2024)]

optics
Computational rephotography

Soonmin Bae,Aseem Agarwala,Frédo Durand

DOI: https://doi.org/10.1145/1805964.1805968

IF: 6.2

2010-06-01

ACM Transactions on Graphics

Abstract:Rephotographers aim to recapture an existing photograph from the same viewpoint. A historical photograph paired with a well-aligned modern rephotograph can serve as a remarkable visualization of the passage of time. However, the task of rephotography is tedious and often imprecise, because reproducing the viewpoint of the original photograph is challenging. The rephotographer must disambiguate between the six degrees of freedom of 3D translation and rotation, and the confounding similarity between the effects of camera zoom and dolly. We present a real-time estimation and visualization technique for rephotography that helps users reach a desired viewpoint during capture. The input to our technique is a reference image taken from the desired viewpoint. The user moves through the scene with a camera and follows our visualization to reach the desired viewpoint. We employ computer vision techniques to compute the relative viewpoint difference. We guide 3D movement using two 2D arrows. We demonstrate the success of our technique by rephotographing historical images and conducting user studies.

computer science, software engineering
Retinomorphic Sensing

Zhaodong Kang,Jianing Li,Lin Zhu,Yonghong Tian

DOI: https://doi.org/10.1145/3474085.3479237

2021-01-01

Abstract:Conventional frame-based cameras for multimedia computing have encountered important challenges in high-speed and extreme light scenarios. However, how to design a novel paradigm for visual perception that overcomes the disadvantages of conventional cameras still remains an open issue. In this paper, we propose a novel solution, namely retinomorphic sensing, which integrates fovea-like and peripheral-like sampling mechanisms to generate asynchronous visual streams using a unified representation as the retina does. Technically, our encoder incorporates an interaction controller to switch flexibly between dynamic and static sensing. Then, the decoder effectively extracts dynamic events for machine vision and reconstructs visual textures for human vision. The results show that our strategy enables it to sense dynamic events and visual textures meanwhile reduce data redundancy. We further build a prototype hybrid camera system to verify this strategy on vision tasks such as image reconstruction and object detection. We believe that this novel paradigm will provide insight into future multimedia computing. The code can be available at https://github.com/acmmm2021-bni-retinomorphic/retinomorphic-sensing.
Recent Event Camera Innovations: A Survey

Bharatesh Chakravarthi,Aayush Atul Verma,Kostas Daniilidis,Cornelia Fermuller,Yezhou Yang

2024-08-27

Abstract:Event-based vision, inspired by the human visual system, offers transformative capabilities such as low latency, high dynamic range, and reduced power consumption. This paper presents a comprehensive survey of event cameras, tracing their evolution over time. It introduces the fundamental principles of event cameras, compares them with traditional frame cameras, and highlights their unique characteristics and operational differences. The survey covers various event camera models from leading manufacturers, key technological milestones, and influential research contributions. It explores diverse application areas across different domains and discusses essential real-world and synthetic datasets for research advancement. Additionally, the role of event camera simulators in testing and development is discussed. This survey aims to consolidate the current state of event cameras and inspire further innovation in this rapidly evolving field. To support the research community, a GitHub page (<a class="link-external link-https" href="https://github.com/chakravarthi589/Event-based-Vision_Resources" rel="external noopener nofollow">this https URL</a>) categorizes past and future research articles and consolidates valuable resources.

Computer Vision and Pattern Recognition
Seeing the World through Your Eyes

Hadi Alzayer,Kevin Zhang,Brandon Feng,Christopher Metzler,Jia-Bin Huang

2024-03-03

Abstract:The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like. By imaging the eyes of a moving person, we can collect multiple views of a scene outside the camera's direct line of sight through the reflections in the eyes. In this paper, we reconstruct a 3D scene beyond the camera's line of sight using portrait images containing eye reflections. This task is challenging due to 1) the difficulty of accurately estimating eye poses and 2) the entangled appearance of the eye iris and the scene reflections. Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer's eye iris texture. We further propose a simple regularization prior on the iris texture pattern to improve reconstruction quality. Through various experiments on synthetic and real-world captures featuring people with varied eye colors, we demonstrate the feasibility of our approach to recover 3D scenes using eye reflections.

Computer Vision and Pattern Recognition
Freeform Lens Design to Eliminate Retroreflection for Optical Systems.

Sifeng He,Yuan Meng,Mali Gong

DOI: https://doi.org/10.1364/ao.57.001218

IF: 1.9

2018-01-01

Applied Optics

Abstract:This paper presents an effective approach to alleviate the cat-eye effect by inserting a freeform single lens into the original optical system. By shifting the image of the given optical system transversely on the original image plane, the reflected beams originating from the sensor are subsequently blocked by the optical aperture, substantially eliminating retroreflections from the system. The influence of incident angle on retroreflected beams is analyzed in detail, and the optimal image translation distance for completely eliminating the cat-eye effect is also proposed via numerical simulations. According to the translation requirement, a freeform single lens comprised of two different freeform surfaces is designed for the given system as the only translation element. The design recipe is elaborated on the basis of phase pupil function. Furthermore, a design example is demonstrated to verify the feasibility of our proposal. After optimization with optical design software, the modulation transfer function (MTF) of the modified system after translation is close to the diffraction limit, and the cat-eye reflected beam could be eliminated thoroughly.
Kaleidoscopic imaging

Ilya Reshetouski

Abstract:Kaleidoscopes have a great potential in computational photography as a tool for redistributing light rays. In time-of-flight imaging the concept of the kaleidoscope is also useful when dealing with the reconstruction of the geometry that causes multiple reflections. This work is a step towards opening new possibilities for the use of mirror systems as well as towards making their use more practical. The focus of this work is the analysis of planar kaleidoscope systems to enable their practical applicability in 3D imaging tasks. We analyse important practical properties of mirror systems and develop a theoretical toolbox for dealing with planar kaleidoscopes. Based on this theoretical toolbox we explore the use of planar kaleidoscopes for multiview imaging and for the acquisition of 3D objects. The knowledge of the mirrors positions is crucial for these multi-view applications. On the other hand, the reconstruction of the geometry of a mirror room from time-offlight measurements is also an important problem. We therefore employ the developed tools for solving this problem using multiple observations of a single scene point.

Engineering,Computer Science,Physics
CamRadar

Ziwei Liu,Feng Lin,Chao Wang,Yijie Shen,Zhongjie Ba,Li Lü,Wenyao Xu,Kui Ren

DOI: https://doi.org/10.1145/3569505

2022-01-01

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies

Abstract:Hidden cameras in sensitive locations have become an increasing threat to personal privacy all over the world. Because the camera is small and camouflaged, it is difficult to detect the presence of the camera with naked eyes. Existing works on this subject have either only covered using wireless transmission to detect cameras, or using other methods which are cumbersome in practical use. In this paper, we introduce a new direction that leverages the unintentional electromagnetic (EM) emanations of the camera to detect it. We first find that the digital output of the camera's image sensor will be amplitude-modulated to the EM emanations of the camera's clock. Thus, changes in the scope of the camera will directly cause changes in the camera's EM emanations, which constitutes a unique characteristic for a hidden camera. Based on this, we propose a novel camera detection system named CamRadar, which can filter out potential camera EM emanations from numerous EM signals quickly and achieve accurate hidden camera detection. Benefitting from the camera's EM emanations, CamRadar will not be limited by the camera transmission types or the detection angle. Our extensive real-world experiments using CamRadar and 19 hidden cameras show that CamRadar achieves a fast detection (in 16.75s) with a detection rate of 93.23% as well as a low false positive rate of 3.95%.

Watching the watchers: camera identification and characterization using retro-reflections

A Single-Camera Gaze Tracking System under Natural Light

A new class of Ornstein transformations with singular spectrum

Recent Advances on Apposition Compound Eye Cameras

Whole-viewing-angle cat's-eye retroreflector as a target of laser trackers

Arthropod Eye-Inspired Digital Camera With Unique Imaging Characteristics

Geometric properties of multiple reflections in catadioptric camera with two planar mirrors

Dynamically Tunable Hemispherical Electronic Eye Camera System with Adjustable Zoom Capability

Digital Cameras with Designs Inspired by the Arthropod Eye

Self-Calibration of Catadioptric Camera with Two Planar Mirrors from Silhouettes

Mechanics of Tunable Hemispherical Electronic Eye Camera Systems That Combine Rigid Device Elements with Soft Elastomers

Simulation Of Imaging And Refocusing For Light Field Photography

High-speed target tracking control system based on short-time rotational reflection imaging

Real-time ghost image characterization for panoramic annular lenses

Computational rephotography

Retinomorphic Sensing

Recent Event Camera Innovations: A Survey

Seeing the World through Your Eyes

Freeform Lens Design to Eliminate Retroreflection for Optical Systems.

Kaleidoscopic imaging

CamRadar