Frontiers of Deep Learning: From Novel Application to Real-World Deployment

Rui Xie
2024-07-19
Abstract:Deep learning continues to re-shape numerous fields, from natural language processing and imaging to data analytics and recommendation systems. This report studies two research papers that represent recent progress on deep learning from two largely different aspects: The first paper applied the transformer networks, which are typically used in language models, to improve the quality of synthetic aperture radar image by effectively reducing the speckle noise. The second paper presents an in-storage computing design solution to enable cost-efficient and high-performance implementations of deep learning recommendation systems. In addition to summarizing each paper in terms of motivation, key ideas and techniques, and evaluation results, this report also presents thoughts and discussions about possible future research directions. By carrying out in-depth study on these two representative papers and related references, this doctoral candidate has developed better understanding on the far-reaching impact and efficient implementation of deep learning models.
Machine Learning
What problem does this paper attempt to address?
The paper attempts to address two issues related to deep learning: 1. **Synthetic Aperture Radar (SAR) Image Denoising**: - **Background**: SAR images inherently contain speckle noise, which makes the images appear very grainy and obscures important details, making the images difficult to analyze. - **Objective**: To improve the clarity of SAR images and preserve details using a transformer architecture. - **Method**: A new method based on a transformer architecture is proposed to address the SAR image denoising problem. It captures local and global dependencies in the image through a self-attention mechanism, thereby reducing speckle noise and preserving image details. 2. **Efficient Implementation of Recommendation Systems**: - **Background**: With the development of deep neural networks, the scale of recommendation systems has been continuously increasing, and the demand for embedding layers has also increased, leading to storage bottlenecks in traditional implementations. - **Objective**: To improve the efficiency of recommendation system implementation, especially when handling large-scale datasets, through in-storage computing. - **Method**: The RM-SSD system is proposed, which leverages the computational capabilities of solid-state drives (SSD) to perform recommendation model inference calculations within the storage device, thereby reducing latency and increasing throughput. Through these two research directions, the paper aims to demonstrate the potential applications of deep learning in different fields and propose effective solutions to overcome existing technical challenges.