Abstract:The astronomy community has witnessed an explosive growth in the use of deep-learning techniques based on neural networks since the mid-2010s. The widespread adoption of these nature-inspired technologies has helped astronomers tackle previously insurmountable problems and provided an unprecedented opportunity for new discoveries. However, one of the primary tools of today's optical astronomy is neither natural nor efficient: their photo-sensing devices. Specifically, the modern CCD camera - like that of the cutting-edge Rubin Observatory - requires an internal clock to regularly expose the sensor to light, consumes a large amount of energy and information bandwidth, and has a limited dynamic range. On the contrary, biological eyes lack an internal clock and a shutter, have much higher pixel density but consume significantly less energy and bandwidth, and can adapt to bright and low light conditions. Inspired by the nature of the eyes, M. Mahowald and C. Mead introduced the revolutionary concept of a silicon retina sensor in 1991. Also known as event-based cameras (EBCs), these types of devices operate in a vastly different way compared to conventional CCD-based imaging sensors. EBCs mimic the operating principles of optic nerves and continuously produce a stream of events, with each event generated only when a pixel detects a change in light intensity. EBCs do not have fixed exposure times, have high dynamic range, require low power for operation, and can capture high-speed phenomena. These properties are important requirements for Cherenkov telescopes as well as other high-speed optical astronomy. This work presents the opportunities and challenges of using EBCs in those cases, and proposes a low-cost approach to experimentally assess the feasibility of this innovative technique.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are as follows: Currently, light - sensing devices (such as CCD cameras) used in Atmospheric Cherenkov Telescopes (IACTs) and fast optical astronomy have problems such as low efficiency, high energy consumption, and limited dynamic range. Specifically, modern CCD cameras require an internal clock to expose the sensor periodically, consuming a large amount of energy and information bandwidth, and having a limited dynamic range. In contrast, biological eyes have no internal clock and shutter, have a higher pixel density, but consume significantly less energy and bandwidth, and are able to adapt to bright and dim lighting conditions. To solve these problems, the paper explores the possibility of using bio - inspired neuromorphic cameras (Event - Based Cameras, EBCs). EBCs imitate the working principle of retinal neurons and generate an event stream only when a change in pixel brightness is detected, thus having the following advantages: 1. **Low data volume**: EBCs only record pixels with brightness changes, reducing the data volume and improving the spatial resolution. 2. **High dynamic range**: EBCs can adaptively capture information from low - light to strong - light, with a dynamic range exceeding 100 dB. 3. **High temporal resolution**: The read - out rate of EBCs can reach 2 MHz to 1200 MHz, which is suitable for the observation of fast astronomical phenomena. 4. **Low voltage, low power consumption**: EBCs only consume power when processing changing pixels, usually only requiring a few volts of voltage and about 10 mW of power. 5. **Cost and availability**: EBCs are relatively inexpensive and there are already commercial products available for selection. In addition, the paper also discusses the challenges faced in applying EBCs to IACTs and fast optical astronomy, including sensitivity estimation, temporal resolution, data interpretation, and integration problems. Finally, the paper proposes a low - cost EBC - based IACT prototype design to evaluate the potential and limitations of this new technology. In summary, this paper aims to improve the light - sensing technology in Atmospheric Cherenkov Telescopes and fast optical astronomy by introducing bio - inspired neuromorphic cameras, thereby improving the observation efficiency and expanding the application range.

Neuromorphic cameras for Atmospheric Cherenkov Telescopes and fast optical astronomy: new paradigm, challenges and opportunities

Arthropod Eye-Inspired Digital Camera With Unique Imaging Characteristics

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception.

High-Speed, Photon Counting CCD Cameras for Astronomy

On the use of electron-multiplying CCDs for astronomical spectroscopy

Dynamically Tunable Hemispherical Electronic Eye Camera System with Adjustable Zoom Capability

Thin On-Sensor Nanophotonic Array Cameras

Neuromorphic Imaging with Super-Resolution

Microsaccade-inspired event camera for robotics

Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy

Seeing Motion at Nighttime with an Event Camera

Compact biologically inspired camera with computational compound eye

Retina-Inspired Object Motion Segmentation for Event-Cameras

Ultrafast machine vision with 2D material neural network image sensors

Electron Multiplying CCDs - New Technology for Low Light Level Imaging

Spectral Convolutional Neural Network Chip for In-sensor Edge Computing of Incoherent Natural Light

The NectarCAM camera project

Spatially Varying Nanophotonic Neural Networks

EDeNN: Event Decay Neural Networks for low latency vision

Generalized Event Cameras