A Method for Parsing GCC abstract Syntax Tree

Liu Jian

Abstract:The structure of GCC syntax tree and its functions in compiling are introduced. Lexical rules and syntax rules used for parsing syntax trees are given, and a method for parsing GCC is proposed.

Computer Science

What problem does this paper attempt to address?

Parsing and abstract syntax tree generation in the GIPSY system

C. Ren

Abstract:The GIPSY system is the implementation of a new programming paradigm: Lucid language are the basis of the system. The objective of the thesis is syntax trees for the programs of both languages. We chose JavaCC/JJTree from tens of compiler construction tools. By modifying the original Lucid lexical and syntactical specifications, and customizing the JavaCC/JJTree generated files, we implemented target AST-building-abled SIPL and GIPL Lucid parsers. Lucid SIPL. Finally, we successfully integrated GIPL and Indexical Lucid SIPL parsers, their translator, and the simulating GIPSY stub.

Computer Science
Extraction of Static Information of C Program from GCC Abstract Syntax Tree Text

FENG Zhan-sheng,SU Xiao-hong,MA Pei-jun

2010-01-01

Abstract:In order to correctly analyze the control dependence and data dependence relations of C program,and then carry out the program slicing,code redundancy and code duplication detection,a method to extract the static information of C program from GCC AST text was put forward.Firstly,the GCC AST text was standardized and the nodes unrelated to control dependence and data dependence analysis were eliminated.Secondly,control dependence subgraph was constructed.If data flow analysis was necessary,control flow graph was constructed based on control dependence subgraph and data dependence subgraph was constructed based on control flow graph.At last,by the introduction of inter-process analysis,system dependence graph was improved.It is indicated that the research can correctly analyze the control dependence and data dependence relations.
Research on Eliminating Redundancies of GCC AST Text

LI Xin,WANG Tian-tian,SU Xiao-hong,MA Pei-jun

DOI: https://doi.org/10.3969/j.issn.1002-137X.2008.10.043

2008-01-01

Computer Science

Abstract:There exist a lot of redundancies in AST text produced by GCC complier in the process of syntax analysis to a C program.If AST text is parsed directly,it will lead to a bad result that the efficiency of the parsing process is low and lots of memory is used by AST.In order to solve this problem,a highly efficient algorithm is put forward to eliminate redundancies based on a thorough research on the structure of GCC AST text and the parsing process.The correctness and practicality were proved by experiments and the mathematical definition of AST parsing was proposed.
Implementing of Gaussian Syntax-Analyzer Using ANTLR

Sanxian Liu,Ruisheng Zhang,TongMing Wei,Yongying He

DOI: https://doi.org/10.1109/ISISE.2008.50

2008-01-01

Abstract:The paper develops a Gaussian Syntax-analyzer using ANTLR. The Gaussian Syntax-analyzer is tested by an example, and builds the AST. Then we examine the analysis process of the example through parser tree generated by ANTLRWorks. The Gaussian Syntax-analyzer's design process is helpful for developing analyzer of other chemical software. At the same time, the paper also provides one kind of syntactical parser methods.
Design and Implementation of Lexical and Syntax Analysis Tool CParser for C Language

Shaojun YANG,Xiaohong SU,Tiantian WANG,Peijun MA

DOI: https://doi.org/10.3969/j.issn.2095-2163.2014.05.020

2014-01-01

Abstract:Program analysis techniques contains control flow analysis,data flow analysis,alias analysis,program slicing techniques and program instrumentation,and has important applications in program comprehension,code refactoring,code optimization,automated software debugging and other aspects. Especially,the lexical analysis and syntax analysis technology is the basis for program analysis techniques. This paper designs and implements a new C language syntax analysis tool named CParser,through three steps which are lexical analysis,preprocessing and syntax analysis to achieve the establishment of the abstract syntax tree based on the source code. This tool is easy to use,and can fully support the C99 standard,furtherly can be used to code clone detection and fault localization.
Tree2tree Structural Language Modeling for Compiler Fuzzing

Haoran Xu,Shuhui Fan,Yongjun Wang,Zhijian Huang,Hongzuo Xu,Peidai Xie

DOI: https://doi.org/10.1007/978-3-030-60245-1_38

2020-01-01

Abstract:Compiler fuzzing requires well-formed test cases. Only syntactically correct programs can pass the parsing stage of a compiler. Recently, advanced compiler fuzzers produce test cases by learning a generative language model of regular programs. They treat programs as natural language texts without leveraging any syntactic structure, making them hard to produce syntactically correct programs when programs get long. In this paper, we propose a novel tree-to-tree (tree2tree) model to leverage the structural information for a robust test case generation. We adopt an encoder-decoder architecture to map a partial abstract syntax tree (AST) to a complete AST. We introduce a C compiler fuzzing framework, named TSmith. Specifically, TSmith employs a tree-based encoder to encode the input partial AST to capture the hierarchical structure information. It then adopts a tree decoder to generate a complete AST by expanding the non-terminals in a top-down manner. Finally, the output ASTs are converted into corresponding test programs. Experimental results show that our generation strategies have a maximum parsing pass rate of 83%, which is about 21% higher than sequential models. Besides, TSmith significantly improves the code coverage of the compiler. Benefiting from the high pass rate and broad code coverage, TSmith has found 14 new bugs in GCC.
Parsing TCT with a Coarse-to-fine Approach.

Dongchen Li,Xihong Wu

2012-01-01

Abstract:A key observation is that concept compound constituent labels are detrimental to parsing performance. We use a PCFG parsing algorithm that uses a multilevel coarse-to-fine scheme. Our approach requires a sequence of nested partitions or equivalence classes of the PCFG nonterminals, where the nonterminals of each PCFG are clusters of nonterminals of the finer PCFG. We use the results of parsing at a coarser level to prune the next finer level. The coarse-to-fine method use hierarchical projections for incremental pruning. We present experiments which show that parsing with hierarchical state-splitting is fast and accurate on Tsinghua Chinese Treebank. In addition, we propose a multiple-model method that adds concept compound labels to the output of the simple PCFG model and gains higher bracketing recall from the simple model. This scheme can be implemented by training two models on different labeling styles.
A Chinese Dependency Syntax for Treebanking

Haitao Liu,Wei Huang

2006-01-01

Abstract:This paper presents a Chinese dependency syntax for treebanking. The syntax contains 13 word classes and 34 dependency types. A format of treebank based on the syntax is also proposed for the applications of computational and general linguistic research. Some experiments show that the treebank based on the proposed dependency syntax can be used for training and evaluating the dependency parser and for quantitative analysis of Chinese syntax.
A General Parsing Method for ABNF-coded Protocol Messages

LI Wei,TIAN Ye,ZHAO Baohua,ZHOU Hao

DOI: https://doi.org/10.3969/j.issn.1000-3428.2006.13.051

2006-01-01

Abstract:A new general parsing method based on the syntax parsing techniques is proposed for the ABNF-coded protocol messages.Taking the ABNF rule set of the text-message syntax as input,it can automatically parse the protocol messages.The experimental results presented show that the new way is stricter and provides a more general usage compared with the traditional specific methods.
Research on C/C++ Code Static Detection Based on Syntax Tree

Hairong Zheng

2011-01-01

Abstract:To reduce the redundancy of information in software static analysis and improve the accuracy and efficiency of the information extraction,a syntax trees model based on relational storage mode is proposed.Modeling on dependency relationship between the terminal will effectively reducing the information redundancy and be stored in XML(extensible markup language) format.Using the mature parsing technique on XML,a new static detection method based on XML model is put forward and applied to program norm.Experimental results show that the method not only improves the detection efficiency but also increases the detection accuracy.
Auto Extracting for Lexicalized Tree Adjoining Grammar

许云,樊孝忠,张锋

2005-01-01

Abstract:An algorithm of the extracting Lexicalized Tree Adjoining Grammar(LTAG) from Penn Chinese corpus was presented. Idea of the algorithm is to induce three kinds of trees from lexicalized tree bank. Then the method of Head-driven Phrase Structure Grammar(HPSG) was applied to extract lexicalized tree from corpus. In the end, invalid lexicalized trees were filtered out by linguistic rules. It requires fewer human efforts compared with hand-crafted grammar. It is possible to remedy omission of grammatical syntactic structures in hand-crafted grammar.
A General Method for Plants Modeling

Qingshun Zhao

2003-01-01

Abstract:When graphically modeling plants, the process of L-system changes with different rule definitions. As a general method, a language parser based on L-system rules is presented in this paper. Firstly, this method acquires and constructs the parser of, through induction and abstraction, L-plants which is a language able to define various L-system rewrite rules. Secondly, the start state and the rules of L-system are recognized with the parser and then the rule replacement is executed to produce a string. At last, the shape grammar is used to model a specific plant while it parsing the string. This plant modeling method proves to be highly efficient with experiments and with it users can pay more attention to designing the shape of plants and defining the rules of L-system than to thinking about the details of how to replace in each step. This method also has good expansibility. When users want the parser to recognize other types of rules of L-system, they can simply add some new syntax expressions in syntax definition files.
Application of Object-oriented Technique in the OCC C Compiler

REN Jie,YANG Xin,SHI Lei,CHEN Yu,YANG Wei-kang

DOI: https://doi.org/10.3969/j.issn.1000-3428.2008.09.033

2008-01-01

Abstract:The architecture of the C compiler OCC is introduced.The Abstract Syntax Tree(AST) served as the intermediate representation,o which the 86 different nodes are grouped into 8 categories and the coupling degree of the semantic analysis with the visitor design pattern is reduced The memory leak problem during the dynamic allocation and destruction of abstract syntax tree nodes is solved using a simple garbage collector.A type builder module gives 7 different treatments to 9 parts of the type specification,which ease the maintenance problem of the attribute gramma script.
Incorporating rich syntax information in Grammatical Error Correction

Zuchao Li,Kevin Parnow,Hai Zhao

DOI: https://doi.org/10.1016/j.ipm.2022.102891

2022-05-01

Abstract:Syntax parse trees are a method of representing sentence structure and are often used to provide models with syntax information and enhance downstream task performance. Because grammar and syntax are inherently linked, the incorporation of syntax parse trees in GEC is a natural solution. In this work, we present a method of incorporating syntax parse trees for Grammatical Error Correction (GEC). Building off a strong sequence-to-sequence Transformer baseline, we present a unified parse integration method for GEC that allows for the use of both dependency and constituency parse trees, as well as their combination - a syntactic graph. Specifically, on the sentence encoder, we propose a graph encoder that can encode dependency trees and constituent trees at the same time, yielding two representations for terminal nodes (i.e., the token of the sentence) and non-terminal nodes. We next use two cross-attentions (NT-Cross-Attention and T-Cross-Attention) to aggregate these source syntactic representations to the target side for final corrections prediction. In addition to evaluating our models on the popular CoNLL-2014 Shared Task and JFLEG GEC benchmarks, we affirm the effectiveness of our proposed method by testing both varying levels of parsing quality and exploring the use of both parsing formalisms. With further empirical exploration and analysis to identify the source of improvement, we found that rich syntax information provided clear clues for GEC; a syntactic graph composed of multiple syntactic parse trees can effectively compensate for the limited quality and insufficient error correction capability of a single syntactic parse tree.
A Method for Parsing Functional Language in Digital System Design

GUAN XIAOXIAO,GE NING,RUAN FANG

DOI: https://doi.org/10.3969/j.issn.1008-0570.2006.34.091

2006-01-01

Abstract:Introduce a method for parsing, which is based on functional language and used in digital system design, including the in- troduction of the functional language and the parsing tools, the structure of lexical analysis and syntax analysis, the design of syntax tree, the definition of syntax rules, the structure and the generation of abstract syntax tree, and an example to indicate the output re- sult of this method. This method has been successfully used in digital system design project.
Construction Tools for Compiler Front-End and JLUCC's Implementation

JIN Long-fei,LIU Lei

DOI: https://doi.org/10.3969/j.issn.1671-5896.2005.04.017

2005-01-01

Abstract:Some problems of typical Compiler construction tools were pointed out, such as restrictions on grammars, negative efficiencies of embedded semantic actions, limitation of single target language and difficulties of maintenance. In order to solve these problems, a new compiler front-end construction tool -JLUCC (Jilin University Compiler Compiler) has been designed and implemented. JLUCC used Earley algorithm and some technologies involving reflection, object-oriented and design patterns. JLUCC can generate a lexer, a parser, ASTs and tree walkers automatically. JLUCC can support entire class of context-free grammar, multiple-pass and extension of types of target languages. JLUCC is a powerful tool which is easy to use, maintain and extend. At last, this paper looked forward the development tendency of compiler front-end construction tools in the future.
A generalized parsing framework for Abstract Grammars

Daniel Harasim,Chris Bruno,Eva Portelance,Martin Rohrmeier,Timothy J. O'Donnell

DOI: https://doi.org/10.48550/arXiv.1710.11301

2018-01-20

Abstract:This technical report presents a general framework for parsing a variety of grammar formalisms. We develop a grammar formalism, called an Abstract Grammar, which is general enough to represent grammars at many levels of the hierarchy, including Context Free Grammars, Minimalist Grammars, and Generalized Context-free Grammars. We then develop a single parsing framework which is capable of parsing grammars which are at least up to GCFGs on the hierarchy. Our parsing framework exposes a grammar interface, so that it can parse any particular grammar formalism that can be reduced to an Abstract Grammar.

Computation and Language,Formal Languages and Automata Theory
Parsing Penn Chinese Treebank Based on Lexicalized Model

Hailong Cao,Yujie Zhang,Hitoshi Isahara

2007-01-01

Abstract:Syntactic parsing is one of the most important technologies of natural language processing. The development of Penn Chinese Treebank (CTB) spurred the research of Chinese parsing. This paper describes a lexicalized statistical Chinese parser. First, a lexicalized model based on hidden Markov model is proposed for part of speech tagging. Second, a well-known lexicalized model i.e. the head-driven model is adapted to parse the automatically POS tagged Chinese sentences. The construction of the parser is described, and the effects of details that can make great difference in the parsing performance are analyzed. On sentences of length less than 100 words, the parser performances at 80.08% precision and 78.45% recall on, surpassing the best published results.
Theories and Methods of Chinese Automatic Syntactic Parsing: A Critical Survey

LIU Ting,MA Jin-shan

2009-01-01

Abstract:This paper reviews the theories and methods of Chinese parsing in three closely related areas.First,several important syntactic theories underlying both the parsing technology and Chinese treebank compilation are examined.Second,several popular parsing methods and the difficult problems associated with them are reviewed.Third,the state of the art of Chinese syntactic parsing is presented.The paper concludes by summing up the general progress of Chinese parsing and discussing some persistent open problems.

A Method for Parsing GCC abstract Syntax Tree

Parsing and abstract syntax tree generation in the GIPSY system

Extraction of Static Information of C Program from GCC Abstract Syntax Tree Text

Research on Eliminating Redundancies of GCC AST Text

Implementing of Gaussian Syntax-Analyzer Using ANTLR

Design and Implementation of Lexical and Syntax Analysis Tool CParser for C Language

Tree2tree Structural Language Modeling for Compiler Fuzzing

Parsing TCT with a Coarse-to-fine Approach.

A Chinese Dependency Syntax for Treebanking

A General Parsing Method for ABNF-coded Protocol Messages

Research on C/C++ Code Static Detection Based on Syntax Tree

Auto Extracting for Lexicalized Tree Adjoining Grammar

A General Method for Plants Modeling

Application of Object-oriented Technique in the OCC C Compiler

Incorporating rich syntax information in Grammatical Error Correction

A Method for Parsing Functional Language in Digital System Design

Construction Tools for Compiler Front-End and JLUCC's Implementation

A generalized parsing framework for Abstract Grammars

Parsing Penn Chinese Treebank Based on Lexicalized Model

Theories and Methods of Chinese Automatic Syntactic Parsing: A Critical Survey