[Obesity and vascular diseases].

S. Oikawa,A. Sasaki

1995-08-10

Abstract:

What problem does this paper attempt to address?

DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature

Kohulan Rajan,Henning Otto Brinkhaus,Maria Sorokina,Achim Zielesny,Christoph Steinbeck

DOI: https://doi.org/10.1186/s13321-021-00496-1

2021-03-08

Journal of Cheminformatics

Abstract:Abstract Chemistry looks back at many decades of publications on chemical compounds, their structures and properties, in scientific articles. Liberating this knowledge (semi-)automatically and making it available to the world in open-access databases is a current challenge. Apart from mining textual information, Optical Chemical Structure Recognition (OCSR), the translation of an image of a chemical structure into a machine-readable representation, is part of this workflow. As the OCSR process requires an image containing a chemical structure, there is a need for a publicly available tool that automatically recognizes and segments chemical structure depictions from scientific publications. This is especially important for older documents which are only available as scanned pages. Here, we present DECIMER (Deep lEarning for Chemical IMagE Recognition) Segmentation , the first open-source, deep learning-based tool for automated recognition and segmentation of chemical structures from the scientific literature. The workflow is divided into two main stages. During the detection step, a deep learning model recognizes chemical structure depictions and creates masks which define their positions on the input page. Subsequently, potentially incomplete masks are expanded in a post-processing workflow. The performance of DECIMER Segmentation has been manually evaluated on three sets of publications from different publishers. The approach operates on bitmap images of journal pages to be applicable also to older articles before the introduction of vector images in PDFs. By making the source code and the trained model publicly available, we hope to contribute to the development of comprehensive chemical data extraction workflows. In order to facilitate access to DECIMER Segmentation, we also developed a web application. The web application, available at https://decimer.ai , lets the user upload a pdf file and retrieve the segmented structure depictions.

chemistry, multidisciplinary,computer science, interdisciplinary applications, information systems
DECIMER 1.0: deep learning for chemical image recognition using transformers

Kohulan Rajan,Achim Zielesny,Christoph Steinbeck

DOI: https://doi.org/10.1186/s13321-021-00538-8

2021-08-17

Journal of Cheminformatics

Abstract:Abstract The amount of data available on chemical structures and their properties has increased steadily over the past decades. In particular, articles published before the mid-1990 are available only in printed or scanned form. The extraction and storage of data from those articles in a publicly accessible database are desirable, but doing this manually is a slow and error-prone process. In order to extract chemical structure depictions and convert them into a computer-readable format, Optical Chemical Structure Recognition (OCSR) tools were developed where the best performing OCSR tools are mostly rule-based. The DECIMER (Deep lEarning for Chemical ImagE Recognition) project was launched to address the OCSR problem with the latest computational intelligence methods to provide an automated open-source software solution. Various current deep learning approaches were explored to seek a best-fitting solution to the problem. In a preliminary communication, we outlined the prospect of being able to predict SMILES encodings of chemical structure depictions with about 90% accuracy using a dataset of 50–100 million molecules. In this article, the new DECIMER model is presented, a transformer-based network, which can predict SMILES with above 96% accuracy from depictions of chemical structures without stereochemical information and above 89% accuracy for depictions with stereochemical information.

chemistry, multidisciplinary,computer science, interdisciplinary applications, information systems
DECIMER: towards deep learning for chemical image recognition

Kohulan Rajan,Achim Zielesny,Christoph Steinbeck

DOI: https://doi.org/10.1186/s13321-020-00469-w

2020-10-27

Journal of Cheminformatics

Abstract:Abstract The automatic recognition of chemical structure diagrams from the literature is an indispensable component of workflows to re-discover information about chemicals and to make it available in open-access databases. Here we report preliminary findings in our development of Deep lEarning for Chemical ImagE Recognition (DECIMER), a deep learning method based on existing show-and-tell deep neural networks, which makes very few assumptions about the structure of the underlying problem. It translates a bitmap image of a molecule, as found in publications, into a SMILES. The training state reported here does not yet rival the performance of existing traditional approaches, but we present evidence that our method will reach a comparable detection power with sufficient training time. Training success of DECIMER depends on the input data representation: DeepSMILES are superior over SMILES and we have a preliminary indication that the recently reported SELFIES outperform DeepSMILES. An extrapolation of our results towards larger training data sizes suggests that we might be able to achieve near-accurate prediction with 50 to 100 million training structures. This work is entirely based on open-source software and open data and is available to the general public for any purpose.

chemistry, multidisciplinary,computer science, interdisciplinary applications, information systems
Advancements in Hand-Drawn Chemical Structure Recognition through an Enhanced DECIMER Architecture

Kohulan Rajan,Henning Otto Brinkhaus,Achim Zielesny,Christoph Steinbeck

DOI: https://doi.org/10.26434/chemrxiv-2024-7ch9f

2024-03-11

Abstract:Accurate recognition of hand-drawn chemical structures is crucial for digitising hand-written chemical information found in traditional laboratory notebooks or for facilitating stylus-based structure entry on tablets or smartphones. However, the inherent variability in hand-drawn structures poses challenges for existing Optical Chemical Structure Recognition (OCSR) software. To address this, we present an enhanced Deep lEarning for Chemical ImagE Recognition (DECIMER) architecture that leverages a combination of Convolutional Neural Networks (CNNs) and Transformers to improve the recognition of hand-drawn chemical structures. The model incorporates an EfficientNetV2 CNN encoder that extracts features from hand-drawn images, followed by a Transformer decoder that converts the extracted features into Simplified Molecular Input Line Entry System (SMILES) strings. Our models were trained using synthetic hand-drawn images generated by RanDepict, a tool for depicting chemical structures with different style elements. To evaluate the model's performance, a benchmark was performed using a real-world dataset of hand-drawn chemical structures. The results indicate that our improved DECIMER architecture exhibits a significantly enhanced recognition accuracy compared to other approaches.

Chemistry
YoDe-Segmentation: automated noise-free retrieval of molecular structures from scientific publications

Chong Zhou,Wei Liu,Xiyue Song,Mengling Yang,Xiaowang Peng

DOI: https://doi.org/10.1186/s13321-023-00783-z

2023-11-20

Journal of Cheminformatics

Abstract:Abstract In chemistry-related disciplines, a vast repository of molecular structural data has been documented in scientific publications but remains inaccessible to computational analyses owing to its non-machine-readable format. Optical chemical structure recognition (OCSR) addresses this gap by converting images of chemical molecular structures into a format accessible to computers and convenient for storage, paving the way for further analyses and studies on chemical information. A pivotal initial step in OCSR is automating the noise-free extraction of molecular descriptions from literature. Despite efforts utilising rule-based and deep learning approaches for the extraction process, the accuracy achieved to date is unsatisfactory. To address this issue, we introduce a deep learning model named YoDe-Segmentation in this study, engineered for the automated retrieval of molecular structures from scientific documents. This model operates via a three-stage process encompassing detection, mask generation, and calculation. Initially, it identifies and isolates molecular structures during the detection phase. Subsequently, mask maps are created based on these isolated structures in the mask generation stage. In the final calculation stage, refined and separated mask maps are combined with the isolated molecular structure images, resulting in the acquisition of pure molecular structures. Our model underwent rigorous testing using texts from multiple chemistry-centric journals, with the outcomes subjected to manual validation. The results revealed the superior performance of YoDe-Segmentation compared to alternative algorithms, documenting an average extraction efficiency of 97.62%. This outcome not only highlights the robustness and reliability of the model but also suggests its applicability on a broad scale.

chemistry, multidisciplinary,computer science, interdisciplinary applications, information systems
A review of optical chemical structure recognition tools

Kohulan Rajan,Henning Otto Brinkhaus,Achim Zielesny,Christoph Steinbeck

DOI: https://doi.org/10.1186/s13321-020-00465-0

2020-10-07

Abstract:Structural information about chemical compounds is typically conveyed as 2D images of molecular structures in scientific documents. Unfortunately, these depictions are not a machine-readable representation of the molecules. With a backlog of decades of chemical literature in printed form not properly represented in open-access databases, there is a high demand for the translation of graphical molecular depictions into machine-readable formats. This translation process is known as Optical Chemical Structure Recognition (OCSR). Today, we are looking back on nearly three decades of development in this demanding research field. Most OCSR methods follow a rule-based approach where the key step of vectorization of the depiction is followed by the interpretation of vectors and nodes as bonds and atoms. Opposed to that, some of the latest approaches are based on deep neural networks (DNN). This review provides an overview of all methods and tools that have been published in the field of OCSR. Additionally, a small benchmark study was performed with the available open-source OCSR tools in order to examine their performance.
Comparing Software Tools for Optical Chemical Structure Recognition

Lutz Weber,Aleksei Krasnov,Shadrack Barnabas,Timo Böhme,Stephen Boyer

DOI: https://doi.org/10.26434/chemrxiv-2023-d6kmg-v2

2024-02-01

Abstract:The extraction of chemical information from images, also known as Optical Chemical Structure Recognition (OCSR) has recently gained new attention. This new interest is ignited by various machine learning methods introduced over the last years and the new possibilities to train image models for specific tasks such as OCSR. In the present paper, we have compared 8 open access OCSR methods (DECIMER, ReactionDataExtractor, MolScribe, RxnScribe, SwinOCSR, OCMR, MolVec, and OSRA) using an independent test set of images from patents and patent applications as this is an application area of general interest - precision and recall are highly desired by those who are analysing the intellectual property of chemistry patents. As a result, the used methods have shown different strengths when predicting structures from different images containing different modalities and chemistry categories. These existing methodologies for image extraction overall remain unsatisfactory, indicating a need for further advancements in the field. Further, we have created a machine learning image classifier, classifying images into one out of four image categories and applying the best performing OCSR method for each category. This classifier, the image comparator tools, and datasets have been made available to the public as open access tools.

Chemistry
MolMiner: You only look once for chemical structure recognition

Youjun Xu,Jinchuan Xiao,Chia-Han Chou,Jianhang Zhang,Jintao Zhu,Qiwan Hu,Hemin Li,Ningsheng Han,Bingyu Liu,Shuaipeng Zhang,Jinyu Han,Zhen Zhang,Shuhao Zhang,Weilin Zhang,Luhua Lai,Jianfeng Pei

DOI: https://doi.org/10.48550/arXiv.2205.11016

2022-05-23

Abstract:Molecular structures are always depicted as 2D printed form in scientific documents like journal papers and patents. However, these 2D depictions are not machine-readable. Due to a backlog of decades and an increasing amount of these printed literature, there is a high demand for the translation of printed depictions into machine-readable formats, which is known as Optical Chemical Structure Recognition (OCSR). Most OCSR systems developed over the last three decades follow a rule-based approach where the key step of vectorization of the depiction is based on the interpretation of vectors and nodes as bonds and atoms. Here, we present a practical software MolMiner, which is primarily built up using deep neural networks originally developed for semantic segmentation and object detection to recognize atom and bond elements from documents. These recognized elements can be easily connected as a molecular graph with distance-based construction algorithm. We carefully evaluate our software on four benchmark datasets with the state-of-the-art performance. Various real application scenarios are also tested, yielding satisfactory outcomes. The free download links of Mac and Windows versions are available: Mac: <a class="link-external link-https" href="https://molminer-cdn.iipharma.cn/pharma-mind/artifact/latest/mac/PharmaMind-mac-latest-setup.dmg" rel="external noopener nofollow">this https URL</a> and Windows: <a class="link-external link-https" href="https://molminer-cdn.iipharma.cn/pharma-mind/artifact/latest/win/PharmaMind-win-latest-setup.exe" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition,Quantitative Methods
OCMR: A comprehensive framework for optical chemical molecular recognition

Yan Wang,Ruochi Zhang,Shengde Zhang,Liming Guo,Qiong Zhou,Bowen Zhao,Xiaotong Mo,Qian Yang,Yajuan Huang,Kewei Li,Yusi Fan,Lan Huang,Fengfeng Zhou

DOI: https://doi.org/10.1016/j.compbiomed.2023.107187

IF: 7.7

2023-06-30

Computers in Biology and Medicine

Abstract:Artificial intelligence (AI) has achieved significant progress in the field of drug discovery. AI-based tools have been used in all aspects of drug discovery, including chemical structure recognition. We propose a chemical structure recognition framework, Optical Chemical Molecular Recognition (OCMR), to improve the data extraction capability in practical scenarios compared with the rule-based and end-to-end deep learning models. The proposed OCMR framework enhances the recognition performances via the integration of local information in the topology of molecular graphs. OCMR handles complex tasks like non-canonical drawing and atomic group abbreviation and substantially improves the current state-of-the-art results on multiple public benchmark datasets and one internally curated dataset.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology
ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning

Martijn Oldenhof,Adam Arany,Yves Moreau,Jaak Simm

DOI: https://doi.org/10.1021/acs.jcim.0c00459

IF: 6.162

2020-09-14

Journal of Chemical Information and Modeling

Abstract:In drug discovery, knowledge of the graph structure of chemical compounds is essential. Many thousands of scientific articles and patents in chemistry and pharmaceutical sciences have investigated chemical compounds, but in many cases, the details of the structure of these chemical compounds are published only as an image. A tool to analyze these images automatically and convert them into a chemical graph structure would be useful for many applications, such as drug discovery. A few such tools are available and they are mostly derived from optical character recognition. However, our evaluation of the performance of these tools reveals that they often make mistakes in recognizing the correct bond multiplicity and stereochemical information. In addition, errors sometimes even lead to missing atoms in the resulting graph. In our work, we address these issues by developing a compound recognition method based on machine learning. More specifically, we develop a deep neural network model for optical compound recognition. The deep learning solution presented here consists of a segmentation model, followed by three classification models that predict atom locations, bonds, and charges. Furthermore, this model not only predicts the graph structure of the molecule but also provides all information necessary to relate each component of the resulting graph to the source image. This solution is scalable and can rapidly process thousands of images. Finally, we empirically compare the proposed method with the well-established tool OSRA<sup>1</sup> and observe significant error reduction.The Supporting Information is available free of charge at <a class="ext-link" href="/doi/10.1021/acs.jcim.0c00459?goto=supporting-info">https://pubs.acs.org/doi/10.1021/acs.jcim.0c00459</a>.Eight labeled (SMILES) data sets with 1000 images each generated in four different styles, with each style, containing a set with images without stereochemical information encoded and a set with images encoded with stereochemical information (<a class="ext-link" href="/doi/suppl/10.1021/acs.jcim.0c00459/suppl_file/ci0c00459_si_001.zip">ZIP</a>)Data set with 2D images of chemical compounds cut out of journal articles, with the source article provided for every image (<a class="ext-link" href="/doi/suppl/10.1021/acs.jcim.0c00459/suppl_file/ci0c00459_si_002.zip">ZIP</a>)This article has not yet been cited by other publications.

chemistry, multidisciplinary, medicinal,computer science, interdisciplinary applications, information systems
ChemReco: automated recognition of hand-drawn carbon–hydrogen–oxygen structures using deep learning

Hengjie Ouyang,Wei Liu,Jiajun Tao,Yanghong Luo,Wanjia Zhang,Jiayu Zhou,Shuqi Geng,Chengpeng Zhang

DOI: https://doi.org/10.1038/s41598-024-67496-7

IF: 4.6

2024-07-26

Scientific Reports

Abstract:Chemical molecular structures are a direct and convenient means of expressing chemical knowledge, playing a vital role in academic communication. In chemistry, hand drawing is a common task for students and researchers. If we can convert hand-drawn chemical molecular structures into machine-readable formats, like SMILES encoding, computers can efficiently process and analyze these structures, significantly enhancing the efficiency of chemical research. Furthermore, with the progress of educational technology, automated grading is gaining popularity. When machines automatically recognize chemical molecular structures and assess the correctness of the drawings, it offers great convenience to teachers. We created ChemReco, a tool designed to identify chemical molecular structures involving three atoms: C, H, and O, providing convenience for chemical researchers. Currently, there are limited studies on hand-drawn chemical molecular structures. Therefore, the primary focus of this paper is constructing datasets. We propose a synthetic image method to rapidly generate images resembling hand-drawn chemical molecular structures, enhancing dataset acquisition efficiency. Regarding model selection, the hand-drawn chemical molecule structural recognition model developed in this article achieves a final recognition accuracy of 96.90%. This model employs the encoder-decoder architecture of EfficientNet + Transformer, demonstrating superior performance compared to other encoder-decoder combinations.

multidisciplinary sciences
MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild

Xi Fang,Jiankun Wang,Xiaochen Cai,Shangqian Chen,Shuwen Yang,Lin Yao,Linfeng Zhang,Guolin Ke

2024-11-17

Abstract:In recent decades, chemistry publications and patents have increased rapidly. A significant portion of key information is embedded in molecular structure figures, complicating large-scale literature searches and limiting the application of large language models in fields such as biology, chemistry, and pharmaceuticals. The automatic extraction of precise chemical structures is of critical importance. However, the presence of numerous Markush structures in real-world documents, along with variations in molecular image quality, drawing styles, and noise, significantly limits the performance of existing optical chemical structure recognition (OCSR) methods. We present MolParser, a novel end-to-end OCSR method that efficiently and accurately recognizes chemical structures from real-world documents, including difficult Markush structure. We use a extended SMILES encoding rule to annotate our training dataset. Under this rule, we build MolParser-7M, the largest annotated molecular image dataset to our knowledge. While utilizing a large amount of synthetic data, we employed active learning methods to incorporate substantial in-the-wild data, specifically samples cropped from real patents and scientific literature, into the training process. We trained an end-to-end molecular image captioning model, MolParser, using a curriculum learning approach. MolParser significantly outperforms classical and learning-based methods across most scenarios, with potential for broader downstream applications. The dataset is publicly available.

Computer Vision and Pattern Recognition
Automated molecular structure segmentation from documents using ChemSAM

Bowen Tang,Zhangming Niu,Xiaofeng Wang,Junjie Huang,Chao Ma,Jing Peng,Yinghui Jiang,Ruiquan Ge,Hongyu Hu,Luhao Lin,Guang Yang,Luhao Lin and Guang Yang

DOI: https://doi.org/10.1186/s13321-024-00823-2

2024-03-15

Journal of Cheminformatics

Abstract:Chemical structure segmentation constitutes a pivotal task in cheminformatics, involving the extraction and abstraction of structural information of chemical compounds from text-based sources, including patents and scientific articles. This study introduces a deep learning approach to chemical structure segmentation, employing a Vision Transformer (ViT) to discern the structural patterns of chemical compounds from their graphical representations. The Chemistry-Segment Anything Model (ChemSAM) achieves state-of-the-art results on publicly available benchmark datasets and real-world tasks, underscoring its effectiveness in accurately segmenting chemical structures from text-based sources. Moreover, this deep learning-based approach obviates the need for handcrafted features and demonstrates robustness against variations in image quality and style. During the detection phase, a ViT-based encoder-decoder model is used to identify and locate chemical structure depictions on the input page. This model generates masks to ascertain whether each pixel belongs to a chemical structure, thereby offering a pixel-level classification and indicating the presence or absence of chemical structures at each position. Subsequently, the generated masks are clustered based on their connectivity, and each mask cluster is updated to encapsulate a single structure in the post-processing workflow. This two-step process facilitates the effective automatic extraction of chemical structure depictions from documents. By utilizing the deep learning approach described herein, it is demonstrated that effective performance on low-resolution and densely arranged molecular structural layouts in journal articles and patents is achievable.

chemistry, multidisciplinary,computer science, interdisciplinary applications, information systems
IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System

Daniel Campos,Heng Ji

DOI: https://doi.org/10.48550/arXiv.2109.04202

2021-09-03

Quantitative Methods

Abstract:Like many scientific fields, new chemistry literature has grown at a staggering pace, with thousands of papers released every month. A large portion of chemistry literature focuses on new molecules and reactions between molecules. Most vital information is conveyed through 2-D images of molecules, representing the underlying molecules or reactions described. In order to ensure reproducible and machine-readable molecule representations, text-based molecule descriptors like SMILES and SELFIES were created. These text-based molecule representations provide molecule generation but are unfortunately rarely present in published literature. In the absence of molecule descriptors, the generation of molecule descriptors from the 2-D images present in the literature is necessary to understand chemistry literature at scale. Successful methods such as Optical Structure Recognition Application (OSRA), and ChemSchematicResolver are able to extract the locations of molecules structures in chemistry papers and infer molecular descriptions and reactions. While effective, existing systems expect chemists to correct outputs, making them unsuitable for unsupervised large-scale data mining. Leveraging the task formulation of image captioning introduced by DECIMER, we introduce IMG2SMI, a model which leverages Deep Residual Networks for image feature extraction and an encoder-decoder Transformer layers for molecule description generation. Unlike previous Neural Network-based systems, IMG2SMI builds around the task of molecule description generation, which enables IMG2SMI to outperform OSRA-based systems by 163% in molecule similarity prediction as measured by the molecular MACCS Fingerprint Tanimoto Similarity. Additionally, to facilitate further research on this task, we release a new molecule prediction dataset. including 81 million molecules for molecule description generation
Atom-Level Optical Chemical Structure Recognition with Limited Supervision

Martijn Oldenhof,Edward De Brouwer,Adam Arany,Yves Moreau

2024-04-02

Abstract:Identifying the chemical structure from a graphical representation, or image, of a molecule is a challenging pattern recognition task that would greatly benefit drug development. Yet, existing methods for chemical structure recognition do not typically generalize well, and show diminished effectiveness when confronted with domains where data is sparse, or costly to generate, such as hand-drawn molecule images. To address this limitation, we propose a new chemical structure recognition tool that delivers state-of-the-art performance and can adapt to new domains with a limited number of data samples and supervision. Unlike previous approaches, our method provides atom-level localization, and can therefore segment the image into the different atoms and bonds. Our model is the first model to perform OCSR with atom-level entity detection with only SMILES supervision. Through rigorous and extensive benchmarking, we demonstrate the preeminence of our chemical structure recognition approach in terms of data efficiency, accuracy, and atom-level entity prediction.

Computer Vision and Pattern Recognition
MICER: A Pre-trained Encoder-Decoder Architecture for Molecular Image Captioning

Jiacai Yi,Chengkun Wu,Xiaochen Zhang,Xinyi Xiao,Yanlong Qiu,Wentao Zhao,Tingjun Hou,Dongsheng Cao

DOI: https://doi.org/10.1093/bioinformatics/btac545

IF: 5.8

2022-08-06

Bioinformatics

Abstract:Motivation Automatic recognition of chemical structures from molecular images provides an important avenue for the rediscovery of chemicals. Traditional rule-based approaches that rely on expert knowledge and fail to consider all the stylistic variations of molecular images usually suffer from cumbersome recognition processes and low generalization ability. Deep learning-based methods that integrate different image styles and automatically learn valuable features are flexible, but currently under-researched and have limitations, and are therefore not fully exploited. Results MICER, an encoder-decoder-based, reconstructed architecture for molecular image captioning, combines transfer learning, attention mechanisms, and several strategies to strengthen effectiveness and plasticity in different datasets. The effects of stereochemical information, molecular complexity, data volume, and pre-trained encoders on MICER performance were evaluated. Experimental results show that the intrinsic features of the molecular images and the sub-model match have a significant impact on the performance of this task. These findings inspire us to design the training dataset and the encoder for the final validation model, and the experimental results suggest that the MICER model consistently outperforms the state-of-the-art methods on four datasets. MICER was more reliable and scalable due to its interpretability and transfer capacity and provides a practical framework for developing comprehensive and accurate automated molecular structure identification tools to explore unknown chemical space. Availability https://github.com/Jiacai-Yi/MICER Supplementary information Supplementary data are available at Bioinformatics online.

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
ABC-Net: a Divide-and-conquer Based Deep Learning Architecture for SMILES Recognition from Molecular Images.

Xiao-Chen Zhang,Jia-Cai Yi,Guo-Ping Yang,Cheng-Kun Wu,Ting-Jun Hou,Dong-Sheng Cao

DOI: https://doi.org/10.1093/bib/bbac033

IF: 9.5

2022-01-01

Briefings in Bioinformatics

Abstract:Structural information for chemical compounds is often described by pictorial images in most scientific documents, which cannot be easily understood and manipulated by computers. This dilemma makes optical chemical structure recognition (OCSR) an essential tool for automatically mining knowledge from an enormous amount of literature. However, existing OCSR methods fall far short of our expectations for realistic requirements due to their poor recovery accuracy. In this paper, we developed a deep neural network model named ABC-Net (Atom and Bond Center Network) to predict graph structures directly. Based on the divide-and-conquer principle, we propose to model an atom or a bond as a single point in the center. In this way, we can leverage a fully convolutional neural network (CNN) to generate a series of heat-maps to identify these points and predict relevant properties, such as atom types, atom charges, bond types and other properties. Thus, the molecular structure can be recovered by assembling the detected atoms and bonds. Our approach integrates all the detection and property prediction tasks into a single fully CNN, which is scalable and capable of processing molecular images quite efficiently. Experimental results demonstrate that our method could achieve a significant improvement in recognition performance compared with publicly available tools. The proposed method could be considered as a promising solution to OCSR problems and a starting point for the acquisition of molecular information in the literature.
ReactionDataExtractor: A Tool for Automated Extraction of Information from Chemical Reaction Schemes

Damian M. Wilary,Jacqueline M. Cole

DOI: https://doi.org/10.1021/acs.jcim.1c01017

IF: 6.162

2021-09-15

Journal of Chemical Information and Modeling

Abstract:Chemical reaction schemes are commonly used for visual encapsulation of chemical information. Figures of reaction schemes contain chemical transformations, the chemical species involved, as well as reaction conditions. From a data-mining point of view, they constitute rich sources, densely packed with knowledge. Yet, the challenge of automatically extracting data from them has remained largely untackled. This work presents ReactionDataExtractor, a software tool that can be used for the automatic extraction of information from multistep reaction schemes. Its capabilities include segmentation of reaction steps, regions containing reaction conditions, chemical diagrams, as well as optical character and structure recognition. A combination of rules and unsupervised machine-learning approaches is used, with bespoke detection algorithms that identify arrows, structures, labels, and conditions detection algorithms. It can be used as a low-maintenance tool for database generation capable of extracting data from large quantities of images supplied by the user. On assessment using a self-generated evaluation set, the tool achieved precision and recall metrics of between 67% and 91% in the six core areas of data extraction. The ReactionDataExtractor tool is released under the MIT license and is available to download from http://www.reactiondataextractor.org.The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.jcim.1c01017.List of all data and strict definitions of evaluation metrics at each stage (PDF)List of figures used in training set (ZIP)This article has not yet been cited by other publications.

chemistry, multidisciplinary, medicinal,computer science, interdisciplinary applications, information systems
MISATO - Machine Learning Dataset for Structure-Based Drug Discovery

Till Siebenmorgen,Filipe Menezes,Sabrina Benassou,Erinç Merdivan,Stefan Kesselheim,Marie Piraud,Fabian J. Theis,Michael Sattler,Grzegorz M. Popowicz

DOI: https://doi.org/10.5281/zenodo.7711952

2023-01-01

Abstract:Developments in Artificial Intelligence (AI) have had an enormous impact on scientific research in recent years. Yet, relatively few robust methods have been reported in the field of structure-based drug discovery. To train AI models to abstract from structural data, highly curated and precise biomolecule-ligand interaction datasets are urgently needed. We present MISATO, a curated dataset of almost 20000 experimental structures of protein-ligand complexes, associated molecular dynamics traces, and electronic properties. Semi-empirical quantum mechanics was used to systematically refine protonation states of proteins and small molecule ligands. Molecular dynamics traces for protein-ligand complexes were obtained in explicit water. The dataset is made readily available to the scientific community via simple python data-loaders. AI baseline models are provided for dynamical and electronic properties. This highly curated dataset is expected to enable the next-generation of AI models for structure-based drug discovery. Our vision is to make MISATO the first step of a vibrant community project for the development of powerful AI-based drug discovery tools.
Handwritten Chemical Structure Image to Structure-Specific Markup Using Random Conditional Guided Decoder

Jinshui Hu,Hao Wu,Mingjun Chen,Chenyu Liu,Jiajia Wu,Shi Yin,Baocai Yin,Bing Yin,Cong Liu,Jun Du,Lirong Dai

DOI: https://doi.org/10.1145/3581783.3612573

2023-01-01

Abstract:Satisfactory recognition performance has been achieved for simple and controllable printed molecular images. However, recognizing handwritten chemical structure images remains unresolved due to the inherent ambiguities in handwritten atoms and bonds, as well as the signifcant challenge of converting projected 2D molecular layouts into markup strings. Target to address these problems, this paper proposes an end-to-end framework for handwritten chemical structure images recognition, with novel structure-specific markup language (SSML) and random conditional guided decoder (RCGD). SSML alleviates ambiguity and complexity in Chemfig syntax by designing an innovative markup language to accurately depict molecular structures. Besides, we propose RCGD to address the issue of multiple path decoding of molecular structures, which is composed of conditional attention guidance, memory classification and path selection mechanisms. In order to fully confirm the effectiveness of the end-to-end method, a new database containing 50,000 handwritten chemical structure images (EDU-CHEMC) has been established. Experimental results demonstrate that compared to traditional SMILES sequences, our SSML can significantly reduces the semantic gap between chemical images and markup strings. It is worth noting that our method can also recognize invalid or non-existent organic molecular structures, making it highly applicable for tasks related to teaching evaluations in the fields of chemistry and biology education. The EDU-CHEMC will be released soon in https://github.com/iFLYTEK-CV/EDU-CHEMC.

[Obesity and vascular diseases].

DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature

DECIMER 1.0: deep learning for chemical image recognition using transformers

DECIMER: towards deep learning for chemical image recognition

Advancements in Hand-Drawn Chemical Structure Recognition through an Enhanced DECIMER Architecture

YoDe-Segmentation: automated noise-free retrieval of molecular structures from scientific publications

A review of optical chemical structure recognition tools

Comparing Software Tools for Optical Chemical Structure Recognition

MolMiner: You only look once for chemical structure recognition

OCMR: A comprehensive framework for optical chemical molecular recognition

ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning

ChemReco: automated recognition of hand-drawn carbon–hydrogen–oxygen structures using deep learning

MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild

Automated molecular structure segmentation from documents using ChemSAM

IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System

Atom-Level Optical Chemical Structure Recognition with Limited Supervision

MICER: A Pre-trained Encoder-Decoder Architecture for Molecular Image Captioning

ABC-Net: a Divide-and-conquer Based Deep Learning Architecture for SMILES Recognition from Molecular Images.

ReactionDataExtractor: A Tool for Automated Extraction of Information from Chemical Reaction Schemes

MISATO - Machine Learning Dataset for Structure-Based Drug Discovery

Handwritten Chemical Structure Image to Structure-Specific Markup Using Random Conditional Guided Decoder