Natural Language Processing with Improved Deep Learning Neural Networks

YiTao Zhou

DOI: https://doi.org/10.1155/2022/6028693

2022-01-07

Scientific Programming

Abstract:As one of the core tasks in the field of natural language processing, syntactic analysis has always been a hot topic for researchers, including tasks such as Questions and Answer (Q&A), Search String Comprehension, Semantic Analysis, and Knowledge Base Construction. This paper aims to study the application of deep learning and neural network in natural language syntax analysis, which has significant research and application value. This paper first studies a transfer-based dependent syntax analyzer using a feed-forward neural network as a classifier. By analyzing the model, we have made meticulous parameters of the model to improve its performance. This paper proposes a dependent syntactic analysis model based on a long-term memory neural network. This model is based on the feed-forward neural network model described above and will be used as a feature extractor. After the feature extractor is pretrained, we use a long short-term memory neural network as a classifier of the transfer action, and the characteristics extracted by the syntactic analyzer as its input to train a recursive neural network classifier optimized by sentences. The classifier can not only classify the current pattern feature but also multirich information such as analysis of state history. Therefore, the model is modeled in the analysis process of the entire sentence in syntactic analysis, replacing the method of modeling independent analysis. The experimental results show that the model has achieved greater performance improvement than baseline methods.

computer science, software engineering

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use deep - learning and neural - network technologies to improve Dependency Syntax Analysis in natural - language processing. Specifically, the author has studied a neural - network model of Dependency Syntax Analysis based on transfer learning, aiming to use a feed - forward neural network as a classifier and improve the performance of Dependency Syntax Analysis by analyzing the model to adjust its parameters. In addition, the paper also proposes a Dependency Syntax Analysis model based on the Long - Short - Term Memory (LSTM) neural network. This model can not only capture and utilize more historical information, but also model the analysis process of the entire sentence, thereby further improving the performance of Dependency Syntax Analysis.

The main contributions of the paper include:
1. Proposing a feed - forward neural network with one - way parameter propagation.
2. Using a neural - network model as a classifier and the back - propagation algorithm as a learning algorithm.
3. Proposing a well - organized data set to evaluate the proposed framework.

The experimental results show that, compared with the baseline method, this model achieves an unlabeled attachment score (UAS) of 91.9% and a labeled attachment score (LAS) of 90.5% on the Penn Treebank development set, with an increase of about 0.7% respectively; on the test set, the UAS accuracy is 90.7% and the LAS accuracy is 89.0%, which are also improved. This indicates that the LSTM - based Dependency Syntax Analysis model is superior to the traditional greedy feed - forward neural - network model in performance.