Rip Current Detection in Nearshore Areas through UAV Video Analysis with Almost Local-Isometric Embedding Techniques on Sphere

Anchen Sun,Kaiqi Yang

2024-02-21

Abstract:Rip currents pose a significant danger to those who visit beaches, as they can swiftly pull swimmers away from shore. Detecting these currents currently relies on costly equipment and is challenging to implement on a larger scale. The advent of unmanned aerial vehicles (UAVs) and camera technology, however, has made monitoring near-shore regions more accessible and scalable. This paper proposes a new framework for detecting rip currents using video-based methods that leverage optical flow estimation, offshore direction calculation, earth camera projection with almost local-isometric embedding on the sphere, and temporal data fusion techniques. Through the analysis of videos from multiple beaches, including Palm Beach, Haulover, Ocean Reef Park, and South Beach, as well as YouTube footage, we demonstrate the efficacy of our approach, which aligns with human experts' annotations.

Image and Video Processing,Graphics,Multimedia

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to use unmanned aerial vehicle (UAV) video analysis technology to automatically detect rip currents in near - shore areas. Rip currents are powerful water currents that can quickly pull swimmers from the shore towards the sea, posing a significant threat to beach visitors. Currently, the detection of rip currents mainly relies on expensive equipment and is difficult to implement on a large scale. Therefore, the paper proposes a new framework based on UAV videos to detect rip currents through optical flow estimation, offshore direction calculation, earth - camera projection, and temporal data fusion techniques.

### Main contributions of the paper:
1. **The first fully - automated UAV - video - based rip - current detection framework**: This framework can detect rip currents using only UAV video data without human intervention.
2. **Introduction of a new modeling method of almost locally isometric embedding on the sphere**: This method can accurately map the captured pixels to the actual earth's surface.
3. **Introduction of temporal data fusion technology**: By fusing the temporal data in UAV videos, the robustness of rip - current detection is improved.
4. **Evaluation of the performance of different optical flow estimation methods**: Analyze the performance of various optical flow estimation methods in current speed estimation and rip - current detection, and determine the key factors for selecting appropriate methods.
5. **Experimental verification**: Use videos from different locations to conduct experiments and quantitatively and qualitatively evaluate the effectiveness of the proposed rip - current detection framework.

### Paper structure:
- **Introduction**: Introduce the hazards of rip currents and the importance of their detection, and summarize the existing detection methods and their limitations.
- **Related work**: Review model - based and video - based rip - current detection methods.
- **Proposed rip - current detection framework**: Detail each component of the framework, including optical flow estimation methods, seawater and wave segmentation, offshore direction calculation, and temporal data fusion techniques.
- **Experiments and evaluation**: Use video data from different locations to conduct experiments to verify the effectiveness of the framework.
- **Conclusions and future work**: Summarize the research results and discuss future improvement directions.

### Key technical details:
1. **Optical flow estimation methods**:
- **Problem formulation**: Optical flow \( V=(u, v) \) describes the pixel - level motion between adjacent frames in a video. The optical flow is determined by minimizing the displaced frame difference (DFD):
\[
D = I_x u+I_y v + I_t
\]
where \( I_x, I_y, I_t \) are the derivative matrices of the image intensity values along the \( x, y \) and time dimensions respectively.
- **Optimization problem**:
\[
\arg\min_{V\in E} F_d(V)+\gamma F_r(V)
\]
where \( E \) is the domain of the two - dimensional velocity field, \( F_d(V) \) is the motion modeling term, \( F_r(V) \) is the regularization term, and \( \gamma>0 \) is the Lagrange multiplier that balances the motion modeling and regularization terms.

2. **Seawater and wave segmentation**:
- Use a convolutional neural network (CNN) and U - Net architecture for seawater segmentation to generate a coastal area mask.
- Generate a wave mask through a predefined range in the HSV color space and combine it with the coastal area mask to remove noise.

3. **Offshore direction calculation**:
- Map the pixel points in the video to the actual earth's surface through the almost locally isometric embedding on the sphere technology, and calculate the offshore direction of each pixel point.
- Use Gaussian filtering and Canny edge detection to extract coastline information and calculate the distance matrix of each pixel point to the coastline.

4. **Rip - current detection**:
- Through temporal data fusion technology, use the velocity field and offshore direction information in multi - frame videos to detect rip - current areas.

Through these techniques, the paper provides an efficient and accurate rip - current detection method, which helps to reduce beach drowning accidents.