Letter: Chemoimmunotherapy of adult acute leukaemia.

R. Peto,D. Galton

IF: 202.731

The Lancet

Abstract:

What problem does this paper attempt to address?

MS-NAS: Multi-scale Neural Architecture Search for Medical Image Segmentation

Xingang Yan,Weiwen Jiang,Yiyu Shi,Cheng Zhuo

DOI: https://doi.org/10.1007/978-3-030-59710-8_38

2020-01-01

Abstract:The recent breakthroughs of Neural Architecture Search (NAS) have motivated various applications in medical image segmentation. However, most existing work either simply rely on hyper-parameter tuning or stick to a fixed network backbone, thereby limiting the underlying search space to identify more efficient architecture. This paper presents a Multi-Scale NAS (MS-NAS) framework that is featured with multi-scale search space from network backbone to cell operation, and multi-scale fusion capability to fuse features with different sizes. To mitigate the computational overhead due to the larger search space, a partial channel connection scheme and a two-step decoding method are utilized to reduce computational overhead while maintaining optimization quality. Experimental results show that on various datasets for segmentation, MS-NAS outperforms the state-of-the-art methods and achieves 0.6–5.4% mIOU and 0.4–3.5% DSC improvements, while the computational resource consumption is reduced by 18.0–24.9%.
RT-DNAS: Real-Time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation

Qing Lu,Xiaowei Xu,Shunjie Dong,Cong Hao,Lei Yang,Cheng Zhuo,Yiyu Shi

DOI: https://doi.org/10.1007/978-3-031-16443-9_58

2022-01-01

Abstract:Accurately segmenting temporal frames of cine magnetic resonance imaging (MRI) is a crucial step in various real-time MRI guided cardiac interventions. To achieve fast and accurate visual assistance, there are strict requirements on the maximum latency and minimum throughput of the segmentation framework. State-of-the-art neural networks on this task are mostly hand-crafted to satisfy these constraints while achieving high accuracy. On the other hand, existing literature has demonstrated the power of neural architecture search (NAS) in automatically identifying the best neural architectures for various medical applications, within which differentiable NAS is a prevailing and efficient approach. However, they are mostly guided by accuracy, sometimes with computation complexity, but the importance of real-time constraints are overlooked. A major challenge is that such constraints are non-differentiable and thus are not compatible with the widely used differentiable NAS frameworks. In this paper, we present a strategy that can directly handle real-time constraints in differentiable NAS frameworks, named RT-DNAS. Experiments on extended 2017 MICCAI ACDC dataset show that compared with state-of-the-art manually and automatically designed architectures, RT-DNAS is able to identify neural architectures that can achieve better accuracy while satisfying the real-time constraints.
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation

Chenxi Liu,Liang-Chieh Chen,Florian Schroff,Hartwig Adam,Wei Hua,Alan L. Yuille,Li Fei-Fei

DOI: https://doi.org/10.1109/cvpr.2019.00017

2019-06-01

Abstract:Recently, Neural Architecture Search (NAS) has successfully identified neural network architectures that exceed human designed ones on large-scale image classification. In this paper, we study NAS for semantic image segmentation. Existing works often focus on searching the repeatable cell structure, while hand-designing the outer network structure that controls the spatial resolution changes. This choice simplifies the search space, but becomes increasingly problematic for dense image prediction which exhibits a lot more network level architectural variations. Therefore, we propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space. We present a network level search space that includes many popular designs, and develop a formulation that allows efficient gradient-based architecture search (3 P100 GPU days on Cityscapes images). We demonstrate the effectiveness of the proposed method on the challenging Cityscapes, PASCAL VOC 2012, and ADE20K datasets. Auto-DeepLab, our architecture searched specifically for semantic image segmentation, attains state-of-the-art performance without any ImageNet pretraining.1
DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation

Xiong Zhang,Hongmin Xu,Hong Mo,Jianchao Tan,Cheng Yang,Lei Wang,Wenqi Ren

DOI: https://doi.org/10.48550/arXiv.2003.11883

2021-03-27

Abstract:Neural Architecture Search (NAS) has shown great potentials in automatically designing scalable network architectures for dense image predictions. However, existing NAS algorithms usually compromise on restricted search space and search on proxy task to meet the achievable computational demands. To allow as wide as possible network architectures and avoid the gap between target and proxy dataset, we propose a Densely Connected NAS (DCNAS) framework, which directly searches the optimal network structures for the multi-scale representations of visual information, over a large-scale target dataset. Specifically, by connecting cells with each other using learnable weights, we introduce a densely connected search space to cover an abundance of mainstream network designs. Moreover, by combining both path-level and channel-level sampling strategies, we design a fusion module to reduce the memory consumption of ample search space. We demonstrate that the architecture obtained from our DCNAS algorithm achieves state-of-the-art performances on public semantic image segmentation benchmarks, including 84.3% on Cityscapes, and 86.9% on PASCAL VOC 2012. We also retain leading performances when evaluating the architecture on the more challenging ADE20K and Pascal Context dataset.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
NAS-Unet: Neural Architecture Search for Medical Image Segmentation

Yu Weng,Tianbao Zhou,Yujie Li,Xiaoyu Qiu

DOI: https://doi.org/10.1109/access.2019.2908991

IF: 3.9

2019-01-01

IEEE Access

Abstract:Neural architecture search (NAS) has significant progress in improving the accuracy of image classification. Recently, some works attempt to extend NAS to image segmentation which shows preliminary feasibility. However, all of them focus on searching architecture for semantic segmentation in natural scenes. In this paper, we design three types of primitive operation set on search space to automatically find two cell architecture DownSC and UpSC for semantic image segmentation especially medical image segmentation. Inspired by the U-net architecture and its variants successfully applied to various medical image segmentation, we propose NAS-Unet which is stacked by the same number of DownSC and UpSC on a U-like backbone network. The architectures of DownSC and UpSC updated simultaneously by a differential architecture strategy during the search stage. We demonstrate the good segmentation results of the proposed method on Promise12, Chaos, and ultrasound nerve datasets, which collected by magnetic resonance imaging, computed tomography, and ultrasound, respectively. Without any pretraining, our architecture searched on PASCAL VOC2012, attains better performances and much fewer parameters (about 0.8M) than U-net and one of its variants when evaluated on the above three types of medical image datasets.
C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation

Qihang Yu,Dong Yang,Holger Roth,Yutong Bai,Yixiao Zhang,Alan L. Yuille,Daguang Xu

DOI: https://doi.org/10.48550/arXiv.1912.09628

2020-04-20

Abstract:3D convolution neural networks (CNN) have been proved very successful in parsing organs or tumours in 3D medical images, but it remains sophisticated and time-consuming to choose or design proper 3D networks given different task contexts. Recently, Neural Architecture Search (NAS) is proposed to solve this problem by searching for the best network architecture automatically. However, the inconsistency between search stage and deployment stage often exists in NAS algorithms due to memory constraints and large search space, which could become more serious when applying NAS to some memory and time consuming tasks, such as 3D medical image segmentation. In this paper, we propose coarse-to-fine neural architecture search (C2FNAS) to automatically search a 3D segmentation network from scratch without inconsistency on network size or input size. Specifically, we divide the search procedure into two stages: 1) the coarse stage, where we search the macro-level topology of the network, i.e. how each convolution module is connected to other modules; 2) the fine stage, where we search at micro-level for operations in each cell based on previous searched macro-level topology. The coarse-to-fine manner divides the search procedure into two consecutive stages and meanwhile resolves the inconsistency. We evaluate our method on 10 public datasets from Medical Segmentation Decalthon (MSD) challenge, and achieve state-of-the-art performance with the network searched using one dataset, which demonstrates the effectiveness and generalization of our searched models.

Computer Vision and Pattern Recognition
Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation

Zhichao Lu,Ran Cheng,Shihua Huang,Haoming Zhang,Changxiao Qiu,Fan Yang

DOI: https://doi.org/10.48550/arXiv.2208.06820

2022-08-14

Abstract:The architectural advancements in deep neural networks have led to remarkable leap-forwards across a broad array of computer vision tasks. Instead of relying on human expertise, neural architecture search (NAS) has emerged as a promising avenue toward automating the design of architectures. While recent achievements in image classification have suggested opportunities, the promises of NAS have yet to be thoroughly assessed on more challenging tasks of semantic segmentation. The main challenges of applying NAS to semantic segmentation arise from two aspects: (i) high-resolution images to be processed; (ii) additional requirement of real-time inference speed (i.e., real-time semantic segmentation) for applications such as autonomous driving. To meet such challenges, we propose a surrogate-assisted multi-objective method in this paper. Through a series of customized prediction models, our method effectively transforms the original NAS task into an ordinary multi-objective optimization problem. Followed by a hierarchical pre-screening criterion for in-fill selection, our method progressively achieves a set of efficient architectures trading-off between segmentation accuracy and inference speed. Empirical evaluations on three benchmark datasets together with an application using Huawei Atlas 200 DK suggest that our method can identify architectures significantly outperforming existing state-of-the-art architectures designed both manually by human experts and automatically by other NAS methods.

Computer Vision and Pattern Recognition,Neural and Evolutionary Computing
V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation

Zhuotun Zhu,Chenxi Liu,Dong Yang,Alan Yuille,Daguang Xu

DOI: https://doi.org/10.1109/3dv.2019.00035

2019-09-01

Abstract:Deep learning algorithms, in particular 2D and 3D fully convolutional neural networks (FCNs), have rapidly become the mainstream methodology for volumetric medical image segmentation. However, 2D convolutions cannot fully leverage the rich spatial information along the third axis, while 3D convolutions suffer from the demanding computation and high GPU memory consumption. In this paper, we propose to automatically search the network architecture tailoring to volumetric medical image segmentation problem. Concretely, we formulate the structure learning as differentiable neural architecture search, and let the network itself choose between 2D, 3D or Pseudo-3D (P3D) convolutions at each layer. We evaluate our method on 3 public datasets, i.e., the NIH Pancreas dataset, the Lung and Pancreas dataset from the Medical Segmentation Decathlon (MSD) Challenge. Our method, named V-NAS, consistently outperforms other state-of-the-arts on the segmentation tasks of both normal organ (NIH Pancreas) and abnormal organs (MSD Lung tumors and MSD Pancreas tumors), which shows the power of chosen architecture. Moreover, the searched architecture on one dataset can be well generalized to other datasets, which demonstrates the robustness and practical use of our proposed method.
MedNAS: Multi-Scale Training-Free Neural Architecture Search for Medical Image Analysis

Yan Wang,Liangli Zhen,Jianwei Zhang,Miqing Li,Lei Zhang,Zizhou Wang,Yangqin Feng,Yu Xue,Xiao Wang,Zheng Chen,Tao Luo,Rich Siow Mong Goh,Yong Liu

DOI: https://doi.org/10.1109/tevc.2024.3352641

IF: 16.497

2024-01-01

IEEE Transactions on Evolutionary Computation

Abstract:Deep neural networks have demonstrated impressive results in medical image analysis, but designing suitable architectures for each specific task is expertise dependent and time consuming. Neural architecture search (NAS) offers an effective means of discovering architectures. It has been highly successful in numerous applications, particularly in natural image classification. Yet, medical images possess unique characteristics, such as small regions and a wide variety of lesion sizes, that differentiate them from natural images. Furthermore, most current NAS methods struggle with high computational costs, especially when dealing with high-resolution image datasets. In this article, we present a novel evolutionary NAS method called multiscale training-free neural architecture search (MSTF-NAS) to address these challenges. Specifically, to accommodate the broad range of lesion region sizes in disease diagnosis, we develop a new reduction cell search space that enables the search algorithm to explicitly identify the optimal scale combination for multiscale feature extraction. To overcome the issue of high computational costs, we utilize training-free indicators as performance measures for candidate architectures, which allows us to search for the optimal architecture more efficiently. More specifically, by considering the capability and simplicity of various networks, we formulate a multiobjective optimization problem that involves two training-free indicators and model complexity for candidate architectures. Extensive experiments on a large medical image benchmark and a publicly available breast cancer detection dataset are conducted. The empirical results demonstrate that our MSTF-NAS outperforms both human-designed architectures and current state-of-the-art NAS algorithms on both datasets, indicating the effectiveness of our proposed method.
DNAS: Decoupling Neural Architecture Search for High-Resolution Remote Sensing Image Semantic Segmentation

Yu Wang,Yansheng Li,Wei Chen,Yunzhou Li,Bo Dang

DOI: https://doi.org/10.3390/rs14163864

IF: 5

2022-08-10

Remote Sensing

Abstract:Deep learning methods, especially deep convolutional neural networks (DCNNs), have been widely used in high-resolution remote sensing image (HRSI) semantic segmentation. In literature, most successful DCNNs are artificially designed through a large number of experiments, which often consume lots of time and depend on rich domain knowledge. Recently, neural architecture search (NAS), as a direction for automatically designing network architectures, has achieved great success in different kinds of computer vision tasks. For HRSI semantic segmentation, NAS faces two major challenges: (1) The task's high complexity degree, which is caused by the pixel-by-pixel prediction demand in semantic segmentation, leads to a rapid expansion of the search space; (2) HRSI semantic segmentation often needs to exploit long-range dependency (i.e., a large spatial context), which means the NAS technique requires a lot of display memory in the optimization process and can be tough to converge. With the aforementioned considerations in mind, we propose a new decoupling NAS (DNAS) framework to automatically design the network architecture for HRSI semantic segmentation. In DNAS, a hierarchical search space with three levels is recommended: path-level, connection-level, and cell-level. To adapt to this hierarchical search space, we devised a new decoupling search optimization strategy to decrease the memory occupation. More specifically, the search optimization strategy consists of three stages: (1) a light super-net (i.e., the specific search space) in the path-level space is trained to get the optimal path coding; (2) we endowed the optimal path with various cross-layer connections and it is trained to obtain the connection coding; (3) the super-net, which is initialized by path coding and connection coding, is populated with kinds of concrete cell operators and the optimal cell operators are finally determined. It is worth noting that the well-designed search space can cover various network candidates and the optimization process can be done efficiently. Extensive experiments on the publicly open GID and FU datasets showed that our DNAS outperformed the state-of-the-art methods, including artificial networks and NAS methods.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Hierarchical Shared Architecture Search for Real-Time Semantic Segmentation of Remote Sensing Images

Wenna Wang,Lingyan Ran,Hanlin Yin,Mingjun Sun,Xiuwei Zhang,Yanning Zhang

DOI: https://doi.org/10.1109/tgrs.2024.3373493

IF: 8.2

2024-03-22

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Real-time semantic segmentation of remote-sensing images demands a trade-off between speed and accuracy, which makes it challenging. Apart from manually designed networks, researchers seek to adopt neural architecture search (NAS) to discover a real-time semantic segmentation model with optimal performance automatically. Most existing NAS methods stack up no more than two types of searched cells, omitting the characteristics of resolution variation. This article proposes the hierarchical shared architecture search (HAS) method to automatically build a real-time semantic segmentation model for remote sensing images. Our model contains a lightweight backbone and a multiscale feature fusion module. The lightweight backbone is carefully designed with low computational cost. The multiscale feature fusion module is searched using the NAS method, where only the blocks from the same layer share identical cells. Extensive experiments reveal that our searched real-time semantic segmentation model of remote sensing images achieves the state-of-the-art trade-off between accuracy and speed. Specifically, on the LoveDA, Potsdam, and Vaihingen datasets, the searched network achieves 54.5% mIoU, 87.8% mIoU, and 84.1% mIoU, respectively, with an inference speed of 132.7 FPS. Besides, our searched network achieves 72.6% mIoU at 164.0 FPS on the CityScapes dataset and 72.3% mIoU at 186.4 FPS on the CamVid dataset.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation

Woong Bae,Seungho Lee,Yeha Lee,Beomhee Park,Minki Chung,Kyu-Hwan Jung

DOI: https://doi.org/10.48550/arXiv.1909.00548

2019-09-02

Abstract:Neural Architecture Search (NAS), a framework which automates the task of designing neural networks, has recently been actively studied in the field of deep learning. However, there are only a few NAS methods suitable for 3D medical image segmentation. Medical 3D images are generally very large; thus it is difficult to apply previous NAS methods due to their GPU computational burden and long training time. We propose the resource-optimized neural architecture search method which can be applied to 3D medical segmentation tasks in a short training time (1.39 days for 1GB dataset) using a small amount of computation power (one RTX 2080Ti, 10.8GB GPU memory). Excellent performance can also be achieved without retraining(fine-tuning) which is essential in most NAS methods. These advantages can be achieved by using a reinforcement learning-based controller with parameter sharing and focusing on the optimal search space configuration of macro search rather than micro search. Our experiments demonstrate that the proposed NAS method outperforms manually designed networks with state-of-the-art performance in 3D medical image segmentation.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
FBNetV5: Neural Architecture Search for Multiple Tasks in One Run

Bichen Wu,Chaojian Li,Hang Zhang,Xiaoliang Dai,Peizhao Zhang,Matthew Yu,Jialiang Wang,Yingyan Lin,Peter Vajda

DOI: https://doi.org/10.48550/arXiv.2111.10007

2021-11-19

Computer Vision and Pattern Recognition

Abstract:Neural Architecture Search (NAS) has been widely adopted to design accurate and efficient image classification models. However, applying NAS to a new computer vision task still requires a huge amount of effort. This is because 1) previous NAS research has been over-prioritized on image classification while largely ignoring other tasks; 2) many NAS works focus on optimizing task-specific components that cannot be favorably transferred to other tasks; and 3) existing NAS methods are typically designed to be "proxyless" and require significant effort to be integrated with each new task's training pipelines. To tackle these challenges, we propose FBNetV5, a NAS framework that can search for neural architectures for a variety of vision tasks with much reduced computational cost and human effort. Specifically, we design 1) a search space that is simple yet inclusive and transferable; 2) a multitask search process that is disentangled with target tasks' training pipeline; and 3) an algorithm to simultaneously search for architectures for multiple tasks with a computational cost agnostic to the number of tasks. We evaluate the proposed FBNetV5 targeting three fundamental vision tasks -- image classification, object detection, and semantic segmentation. Models searched by FBNetV5 in a single run of search have outperformed the previous stateof-the-art in all the three tasks: image classification (e.g., +1.3% ImageNet top-1 accuracy under the same FLOPs as compared to FBNetV3), semantic segmentation (e.g., +1.8% higher ADE20K val. mIoU than SegFormer with 3.6x fewer FLOPs), and object detection (e.g., +1.1% COCO val. mAP with 1.2x fewer FLOPs as compared to YOLOX).
NG-NAS: Node growth neural architecture search for 3D medical image segmentation

Shixi Qin,Zixun Zhang,Yuncheng Jiang,Shuguang Cui,Shenghui Cheng,Zhen Li

DOI: https://doi.org/10.1016/j.compmedimag.2023.102268

IF: 7.422

2023-06-18

Computerized Medical Imaging and Graphics

Abstract:Neural architecture search (NAS) has been applied to design proper 3D networks for medical image segmentation. In order to reduce the computation cost in NAS, researchers tend to adopt weight sharing mechanism to search architectures in a supernet. However, recent studies state that the searched architecture rankings may not be accurate with weight sharing mechanism because the training situations are inconsistent between the searching and training phases. In addition, some NAS algorithms design inflexible supernets that only search operators in a pre-defined backbone and ignore the importance of network topology, which limits the performance of searched architecture. To avoid weight sharing mechanism which may lead to inaccurate results and to comprehensively search network topology and operators, we propose a novel NAS algorithm called NG-NAS. Following the previous studies, we consider the segmentation network as a U-shape structure composed of a set of nodes. Instead of searching from the supernet with a limited search space, our NG-NAS starts from a simple architecture with only 5 nodes, and greedily grows the best candidate node until meeting the constraint. We design 2 kinds of node generations to form various network topological structures and prepare 4 candidate operators for each node. To efficiently evaluate candidate node generations, we use NAS without training strategies. We evaluate our method on several public 3D medical image segmentation benchmarks and achieve state-of-the-art performance, demonstrating the effectiveness of the searched architecture and our NG-NAS. Concretely, our method achieves an average Dice score of 85.11 on MSD liver, 65.70 on MSD brain, and 87.59 in BTCV, which performs much better than the previous SOTA methods.

engineering, biomedical,radiology, nuclear medicine & medical imaging
Attention-Guided Residual U-Net with SE Connection and ASPP for Watershed-Based Cell Segmentation in Microscopy Images

Jovial Niyogisubizo,Keliang Zhao,Jintao Meng,Yi Pan,Rosiyadi Didi,Yanjie Wei

DOI: https://doi.org/10.1089/cmb.2023.0446

2024-10-18

Abstract:Time-lapse microscopy imaging is a crucial technique in biomedical studies for observing cellular behavior over time, providing essential data on cell numbers, sizes, shapes, and interactions. Manual analysis of hundreds or thousands of cells is impractical, necessitating the development of automated cell segmentation approaches. Traditional image processing methods have made significant progress in this area, but the advent of deep learning methods, particularly those using U-Net-based networks, has further enhanced performance in medical and microscopy image segmentation. However, challenges remain, particularly in accurately segmenting touching cells in images with low signal-to-noise ratios. Existing methods often struggle with effectively integrating features across different levels of abstraction. This can lead to model confusion, particularly when important contextual information is lost or the features are not adequately distinguished. The challenge lies in appropriately combining these features to preserve critical details while ensuring robust and accurate segmentation. To address these issues, we propose a novel framework called RA-SE-ASPP-Net, which incorporates Residual Blocks, Attention Mechanism, Squeeze-and-Excitation connection, and Atrous Spatial Pyramid Pooling to achieve precise and robust cell segmentation. We evaluate our proposed architecture using an induced pluripotent stem cell reprogramming dataset, a challenging dataset that has received limited attention in this field. Additionally, we compare our model with different ablation experiments to demonstrate its robustness. The proposed architecture outperforms the baseline models in all evaluated metrics, providing the most accurate semantic segmentation results. Finally, we applied the watershed method to the semantic segmentation results to obtain precise segmentations with specific information for each cell.
Neural Architecture Search for Deep Face Recognition

Ning Zhu

DOI: https://doi.org/10.48550/arXiv.1904.09523

2019-04-21

Computer Vision and Pattern Recognition

Abstract:By the widespread popularity of electronic devices, the emergence of biometric technology has brought significant convenience to user authentication compared with the traditional password and mode unlocking. Among many biological characteristics, the face is a universal and irreplaceable feature that does not need too much cooperation and can significantly improve the user's experience at the same time. Face recognition is one of the main functions of electronic equipment propaganda. Hence it's virtually worth researching in computer vision. Previous work in this field has focused on two directions: converting loss function to improve recognition accuracy in traditional deep convolution neural networks (Resnet); combining the latest loss function with the lightweight system (MobileNet) to reduce network size at the minimal expense of accuracy. But none of these has changed the network structure. With the development of AutoML, neural architecture search (NAS) has shown excellent performance in the benchmark of image classification. In this paper, we integrate NAS technology into face recognition to customize a more suitable network. We quote the framework of neural architecture search which trains child and controller network alternately. At the same time, we mutate NAS by incorporating evaluation latency into rewards of reinforcement learning and utilize policy gradient algorithm to search the architecture automatically with the most classical cross-entropy loss. The network architectures we searched out have got state-of-the-art accuracy in the large-scale face dataset, which achieves 98.77% top-1 in MS-Celeb-1M and 99.89% in LFW with relatively small network size. To the best of our knowledge, this proposal is the first attempt to use NAS to solve the problem of Deep Face Recognition and achieve the best results in this domain.
Towards Less Constrained Macro-Neural Architecture Search

Vasco Lopes,Luís A. Alexandre

DOI: https://doi.org/10.48550/arXiv.2203.05508

2023-01-07

Abstract:Networks found with Neural Architecture Search (NAS) achieve state-of-the-art performance in a variety of tasks, out-performing human-designed networks. However, most NAS methods heavily rely on human-defined assumptions that constrain the search: architecture's outer-skeletons, number of layers, parameter heuristics and search spaces. Additionally, common search spaces consist of repeatable modules (cells) instead of fully exploring the architecture's search space by designing entire architectures (macro-search). Imposing such constraints requires deep human expertise and restricts the search to pre-defined settings. In this paper, we propose LCMNAS, a method that pushes NAS to less constrained search spaces by performing macro-search without relying on pre-defined heuristics or bounded search spaces. LCMNAS introduces three components for the NAS pipeline: i) a method that leverages information about well-known architectures to autonomously generate complex search spaces based on Weighted Directed Graphs with hidden properties, ii) an evolutionary search strategy that generates complete architectures from scratch, and iii) a mixed-performance estimation approach that combines information about architectures at initialization stage and lower fidelity estimates to infer their trainability and capacity to model complex functions. We present experiments in 13 different data sets showing that LCMNAS is capable of generating both cell and macro-based architectures with minimal GPU computation and state-of-the-art results. More, we conduct extensive studies on the importance of different NAS components in both cell and macro-based settings. Code for reproducibility is public at <a class="link-external link-https" href="https://github.com/VascoLopes/LCMNAS" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Neural and Evolutionary Computing
XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks

Trevor Londt,Xiaoying Gao,Peter Andreae,Yi Mei

DOI: https://doi.org/10.1007/978-981-99-8391-9_33

2023-12-13

Abstract:Convolutional Neural Networks (CNNs) continue to achieve great success in classification tasks as innovative techniques and complex multi-path architecture topologies are introduced. Neural Architecture Search (NAS) aims to automate the design of these complex architectures, reducing the need for costly manual design work by human experts. Cellular Encoding (CE) is an evolutionary computation technique which excels in constructing novel multi-path topologies of varying complexity and has recently been applied with NAS to evolve CNN architectures for various classification tasks. However, existing CE approaches have severe limitations. They are restricted to only one domain, only partially implement the theme of CE, or only focus on the micro-architecture search space. This paper introduces a new CE representation and algorithm capable of evolving novel multi-path CNN architectures of varying depth, width, and complexity for image and text classification tasks. The algorithm explicitly focuses on the macro-architecture search space. Furthermore, by using a surrogate model approach, we show that the algorithm can evolve a performant CNN architecture in less than one GPU day, thereby allowing a sufficient number of experiment runs to be conducted to achieve scientific robustness. Experiment results show that the approach is highly competitive, defeating several state-of-the-art methods, and is generalisable to both the image and text domains.

Neural and Evolutionary Computing
Advances in neural architecture search

Xin Wang,Wenwu Zhu

DOI: https://doi.org/10.1093/nsr/nwae282

2024-08-23

Abstract:Automated machine learning (AutoML) has achieved remarkable success in automating the non-trivial process of designing machine learning models. Among the focal areas of AutoML, neural architecture search (NAS) stands out, aiming to systematically explore the complex architecture space to discover the optimal neural architecture configurations without intensive manual interventions. NAS has demonstrated its capability of dramatic performance improvement across a large number of real-world tasks. The core components in NAS methodologies normally include (i) defining the appropriate search space, (ii) designing the right search strategy and (iii) developing the effective evaluation mechanism. Although early NAS endeavors are characterized via groundbreaking architecture designs, the imposed exorbitant computational demands prompt a shift towards more efficient paradigms such as weight sharing and evaluation estimation, etc. Concurrently, the introduction of specialized benchmarks has paved the way for standardized comparisons of NAS techniques. Notably, the adaptability of NAS is evidenced by its capability of extending to diverse datasets, including graphs, tabular data and videos, etc., each of which requires a tailored configuration. This paper delves into the multifaceted aspects of NAS, elaborating on its recent advances, applications, tools, benchmarks and prospective research directions.

Letter: Chemoimmunotherapy of adult acute leukaemia.

MS-NAS: Multi-scale Neural Architecture Search for Medical Image Segmentation

RT-DNAS: Real-Time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation

Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation

DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation

NAS-Unet: Neural Architecture Search for Medical Image Segmentation

C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation

Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation

V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation

MedNAS: Multi-Scale Training-Free Neural Architecture Search for Medical Image Analysis

DNAS: Decoupling Neural Architecture Search for High-Resolution Remote Sensing Image Semantic Segmentation

Hierarchical Shared Architecture Search for Real-Time Semantic Segmentation of Remote Sensing Images

Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation

FBNetV5: Neural Architecture Search for Multiple Tasks in One Run

NG-NAS: Node growth neural architecture search for 3D medical image segmentation

Attention-Guided Residual U-Net with SE Connection and ASPP for Watershed-Based Cell Segmentation in Microscopy Images

Neural Architecture Search for Deep Face Recognition

Towards Less Constrained Macro-Neural Architecture Search

XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks

Advances in neural architecture search