Abstract:While omni-directional sensors provide holistic representations typical deep learning frameworks reduce the benefits by introducing distortions and discontinuities as spherical data is supplied as planar input. On the other hand, recent spherical convolutional neural networks (CNNs) often require significant memory and parameters, thus enabling execution only at very low resolutions and shallow architectures. We propose HexNet, an orientation-aware deep learning framework for spherical signals, that allows for fast computation as we exploit standard planar network operations on an efficiently arranged projection of the sphere. Furthermore, we introduce a graph-based version for partial spheres, allowing us to compete at high-resolution with planar CNNs using residual network architectures. Our kernels operate on the tangent of the sphere and thus standard feature weights, pretrained on perspective data, can be transferred, enabling spherical pretraining on ImageNet. As our design is free of distortions and discontinuity, our orientation-aware CNN becomes a new state of the art for semantic segmentation on the recent 2D3DS dataset, and the omni-directional version of SYNTHIA introduced in this work. Moreover, we experimentally show the benefit of our spherical representation over standard images on the Cityscapes dataset by reducing distortion effects of planar CNNs. We implement object detection for the spherical domain. Rotation invariant classification and segmentation tasks are additionally presented for comparison to prior art.

Biologically Inspired Hexagonal Deep Learning for Hexagonal Image Generation

Hexagonal Image Processing in the Context of Machine Learning: Conception of a Biologically Inspired Hexagonal Deep Learning Framework

An efficient hexagonal image framework using pseudo hexagonal pixel for computer vision applications

HexCNN: A Framework for Native Hexagonal Convolutional Neural Networks

An Approach to Edge Detection on a Virtual Hexagonal Structure

Generative Creativity: Adversarial Learning For Bionic Design

HexagDLy - Processing hexagonally sampled data with CNNs in PyTorch

Biologically inspired deep residual networks for computer vision applications

Hexagonal image segmentation on spatially resolved transcriptomics

Resampling and super-resolution of hexagonally sampled images using deep learning

HexNet: An Orientation-Aware Deep Learning Framework for Omni-Directional Input

Generative NeuroEvolution for Deep Learning

Two-stage Generative Models of Simulating Training Data at the Voxel Level for Large-Scale Microscopy Bioimage Segmentation

Local Binary Patterns on Hexagonal Image Structure

Rotation-Invariant Convolution Networks with Hexagon-Based Kernels

Bi-cubic Interpolation for Image Conversion from Virtual Hexagonal to Square Structure.

Canny Edge Detection Using Bilateral Filter on Real Hexagonal Structure

Image Transformation on Hexagonal Structure Based on Conversion between 1D and 2D Coordinates

DL-Polycube: Deep learning enhanced polycube method for high-quality hexahedral mesh generation and volumetric spline construction

HexPlane: A Fast Representation for Dynamic Scenes

Canny edge detection on a virtual hexagonal image structure