Abstract:This paper proposes a parsing algorithm for scene understanding which includes four aspects: computing 3D scene layout, detecting 3D objects (e.g. furniture), detecting 2D faces (windows, doors etc.), and segmenting background. In contrast to previous scene labeling work that applied discriminative classifiers to pixels (or super-pixels), we use a generative Stochastic Scene Grammar (SSG). This grammar represents the compositional structures of visual entities from scene categories, 3D foreground/background, 2D faces, to 1D lines. The grammar includes three types of production rules and two types of contextual relations. Production rules: (i) AND rules represent the decomposition of an entity into sub-parts; (ii) OR rules represent the switching among sub-types of an entity; (iii) SET rules represent an ensemble of visual entities. Contextual relations: (i) Cooperative + relations represent positive links between binding entities, such as hinged faces of a object or aligned boxes; (ii) Competitive - relations represents negative links between competing entities, such as mutually exclusive boxes. We design an efficient MCMC inference algorithm, namely Hierarchical cluster sampling, to search in the large solution space of scene configurations. The algorithm has two stages: (i) Clustering: It forms all possible higher-level structures (clusters) from lower-level entities by production rules and contextual relations. (ii) Sampling: It jumps between alternative structures (clusters) in each layer of the hierarchy to find the most probable configuration (represented by a parse tree). In our experiment, we demonstrate the superiority of our algorithm over existing methods on public dataset. In addition, our approach achieves richer structures in the parse tree.

F ast Computing Grammar-driven Convolution Tree Kernel for Semantic Role Labeling

TGMM: Combining Parse Tree with GPU for Scalable Multilingual and Multi-Granularity Code Clone Detection

GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-edge Collaboration LLM Deployment

An Extended GHKM Algorithm for Inducing Lambda-SCFG.

PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols

FastGAS: Fast Graph-based Annotation Selection for In-Context Learning

Image Parsing Via Stochastic Scene Grammar

An Empirical Study of Compound PCFGs

Faster and Better Grammar-based Text-to-SQL Parsing via Clause-level Parallel Decoding and Alignment Loss

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling

An Extended Grammar System for Learning and Recognizing Complex Visual Events

XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models

GFT: Graph Foundation Model with Transferable Tree Vocabulary

GN-Transformer: Fusing Sequence and Graph Representation for Improved Code Summarization

Complex expressional characterizations learning based on block decomposition for temporal knowledge graph completion

Forest-Based Semantic Role Labeling

GrammarT5: Grammar-Integrated Pretrained Encoder-Decoder Neural Model for Code

KDLGT: A Linear Graph Transformer Framework Via Kernel Decomposition Approach.

GASCOM: Graph-based Attentive Semantic Context Modeling for Online Conversation Understanding

DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference

Efficient Generalized Fused Lasso and Its Applications.