A stochastic grammar of images

ZhuSong-Chun,MumfordDavid

2006-01-01

Foundations and Trends in Computer Graphics and Vision

Abstract:This exploratory paper quests for a stochastic and context sensitive grammar of images. The grammar should achieve the following four objectives and thus serves as a unified framework of representa...

What problem does this paper attempt to address?

A Stochastic Grammar of Images

Song-Chun Zhu,David Mumford

DOI: https://doi.org/10.1561/0600000018

2006-01-01

Abstract:This exploratory paper quests for a stochastic and context sensitive grammar of images. The grammar should achieve the following four objectives and thus serves as a unified framework of representation, learning, and recognition for a large number of object categories. (i) The grammar represents both the hierarchical decompositions from scenes, to objects, parts, primitives and pixels by terminal and non-terminal nodes and the contexts for spatial and functional relations by horizontal links between the nodes. It formulates each object category as the set of all possible valid configurations produced by the grammar. (ii) The grammar is embodied in a simple And-Or graph representation where each Or-node points to alternative sub-configurations and an And-node is decomposed into a number of components. This representation supports recursive top-down/ bottom-up procedures for image parsing under the Bayesian framework and make it convenient to scale up in complexity. Given an input image, the image parsing task constructs a most probable parse graph on-the-fly as the output interpretation and this parse graph is a subgraph of the And-Or graph after making choice on the Or-nodes. (iii) A probabilistic model is defined on this And-Or graph representation to account for the natural occurrence frequency of objects and parts as well as their relations. This model is learned from a relatively small training set per category and then sampled to synthesize a large number of configurations to cover novel object instances in the test set. This generalization capability is mostly missing in discriminative machine learning methods and can largely improve recognition performance in experiments. (iv) To fill the well-known semantic gap between symbols and raw signals, the grammar includes a series of visual dictionaries and organizes them through graph composition. At the bottom-level the dictionary is a set of image primitives each having a number of anchor points with open bonds to link with other primitives. These primitives can be combined to form larger and larger graph structures for parts and objects. The ambiguities in inferring local primitives shall be resolved through top-down computation using larger structures. Finally these primitives forms a primal sketch representation which will generate the input image with every pixels explained. The proposal grammar integrates three prominent representations in the literature: stochastic grammars for composition, Markov (or graphical) models for contexts, and sparse coding with primitives (wavelets). It also combines the structure-based and appearance based methods in the vision literature. Finally the paper presents three case studies to illustrate the proposed grammar.
A Stochastic Image Grammar for Fine-Grained 3D Scene Reconstruction.

Xiaobai Liu,Yadong Mu,Liang Lin

2016-01-01

Abstract:This paper presents a stochastic grammar for fine-grained 3D scene reconstruction from a single image. At the heart of our approach is a small number of grammar rules that can describe the most common geometric structures, e.g., two straights lines being co-linear or orthogonal, or that a line lying on a planar region etc. With these grammar rules, we re-frame single-view 3D reconstruction problem as jointly solving two coupled sub-tasks: i) segmenting of image entities, e.g. planar regions, straight edge segments, and ii) optimizing pixel-wise 3D scene model through the application of grammar rules over image entities. To reconstruct a new image, we design an efficient hybrid Monte Carlo (HMC) algorithm to simulate Markov Chain walking towards a posterior distribution. Our algorithm utilizes two iterative dynamics: i) Hamiltonian Dynamics that makes proposals along the gradient direction to search the continuous pixel-wise 3D scene model; and ii) Cluster Dynamics, that flip the colors of clusters of pixels to form planar region partition. Following the Metropolis-hasting principle, these dynamics not only make distant proposals but also guarantee detail-balance and fast convergence. Results with comparisons on public image dataset show that our method clearly outperforms the alternate state-of-the-art single-view reconstruction methods.
Program Chairs' Introduction to the First International Workshop on Stochastic Image Grammars (Sig-09) in Conjunction with Ieee Cvpr 2009

Sinisa Todorovic,Song-Chun Zhu

DOI: https://doi.org/10.1109/cvprw.2009.5204339

2009-01-01

Abstract:Hierarchical models, semantic contexts, compositionality, taxonomy of visual categories, visual event ontology, stochastic graph matching, and bottom-up/top-down inference are popular research topics in computer vision and pattern recognition. They can be viewed as different aspects of stochastic image grammars. The virtue of image grammars lies in their expressive power to represent an exponentially large number of object and event configurations by using a relatively much smaller vocabulary, and a few compositional rules. In addition to objects and events, various semantic contexts can also be associated with all levels of hierarchical descriptions in grammars, facilitating rich image interpretations.
A Stochastic Grammar for Natural Shapes

Pedro F. Felzenszwalb

DOI: https://doi.org/10.48550/arXiv.1303.2844

2013-03-12

Computer Vision and Pattern Recognition

Abstract:We consider object detection using a generic model for natural shapes. A common approach for object recognition involves matching object models directly to images. Another approach involves building intermediate representations via a generic grouping processes. We argue that these two processes (model-based recognition and grouping) may use similar computational mechanisms. By defining a generic model for shapes we can use model-based techniques to implement a mid-level vision grouping process.
Image Parsing Via Stochastic Scene Grammar

Yibiao Zhao,Song Chun Zhu

2011-01-01

Abstract:This paper proposes a parsing algorithm for scene understanding which includes four aspects: computing 3D scene layout, detecting 3D objects (e.g. furniture), detecting 2D faces (windows, doors etc.), and segmenting background. In contrast to previous scene labeling work that applied discriminative classifiers to pixels (or super-pixels), we use a generative Stochastic Scene Grammar (SSG). This grammar represents the compositional structures of visual entities from scene categories, 3D foreground/background, 2D faces, to 1D lines. The grammar includes three types of production rules and two types of contextual relations. Production rules: (i) AND rules represent the decomposition of an entity into sub-parts; (ii) OR rules represent the switching among sub-types of an entity; (iii) SET rules represent an ensemble of visual entities. Contextual relations: (i) Cooperative + relations represent positive links between binding entities, such as hinged faces of a object or aligned boxes; (ii) Competitive - relations represents negative links between competing entities, such as mutually exclusive boxes. We design an efficient MCMC inference algorithm, namely Hierarchical cluster sampling, to search in the large solution space of scene configurations. The algorithm has two stages: (i) Clustering: It forms all possible higher-level structures (clusters) from lower-level entities by production rules and contextual relations. (ii) Sampling: It jumps between alternative structures (clusters) in each layer of the hierarchy to find the most probable configuration (represented by a parse tree). In our experiment, we demonstrate the superiority of our algorithm over existing methods on public dataset. In addition, our approach achieves richer structures in the parse tree.
Stochastic And-Or Grammars: A Unified Framework and Logic Perspective

Kewei Tu

DOI: https://doi.org/10.48550/arXiv.1506.00858

2016-04-12

Abstract:Stochastic And-Or grammars (AOG) extend traditional stochastic grammars of language to model other types of data such as images and events. In this paper we propose a representation framework of stochastic AOGs that is agnostic to the type of the data being modeled and thus unifies various domain-specific AOGs. Many existing grammar formalisms and probabilistic models in natural language processing, computer vision, and machine learning can be seen as special cases of this framework. We also propose a domain-independent inference algorithm of stochastic context-free AOGs and show its tractability under a reasonable assumption. Furthermore, we provide two interpretations of stochastic context-free AOGs as a subset of probabilistic logic, which connects stochastic AOGs to the field of statistical relational learning and clarifies their relation with a few existing statistical relational models.

Artificial Intelligence
Unsupervised Structure Learning of Stochastic And-Or Grammars

Kewei Tu,Maria Pavlovskaia,Song Chun Zhu

2013-01-01

Abstract:Stochastic And-Or grammars compactly represent both compositionality and re-configurability and have been used to model different types of data such as images and events. We present a unified formalization of stochastic And-Or grammars that is agnostic to the type of the data being modeled, and propose an unsupervised approach to learning the structures as well as the parameters of such grammars. Starting from a trivial initial grammar, our approach iteratively induces compositions and reconfigurations in a unified manner and optimizes the posterior probability of the grammar. In our empirical evaluation, we applied our approach to learning event grammars and image grammars and achieved comparable or better performance than previous approaches.
Human Parsing Using Stochastic And-or Grammars and Rich Appearances

Brandon Rothrock,Song-Chun Zhu

DOI: https://doi.org/10.1109/iccvw.2011.6130303

2011-01-01

Abstract:One of the key challenges to human parsing and pose recovery is handling the variability in geometry and appearance of humans in natural scenes. This variability is due to the large number of distinct articulated configurations, clothing, and self-occlusion, as well as unknown lighting and viewpoint. In this paper, we present a stochastic grammar model that represents the body as an articulated assembly of compositional and reconfigurable parts. The reconfigurable aspect allows a compatible part to be substituted with an alternative part with different attributes, such as for clothing appearance or viewpoint foreshortening. Relations within the grammar enforce consistency between part attributes as well as geometry, allowing a richer set of appearance and geometry constraints over conventional articulated models. Part appearances are modeled by a sparse deformable image template that can still richly describe salient part structures. We describe a dynamic programming parsing algorithm for our model, and show competitive pose recovery results against the state-of-art on a challenging dataset.
Towards Image Semantics and Syntax Sequence Learning

Chun Tao,Timur Ibrayev,Kaushik Roy

2024-01-31

Abstract:Convolutional neural networks and vision transformers have achieved outstanding performance in machine perception, particularly for image classification. Although these image classifiers excel at predicting image-level class labels, they may not discriminate missing or shifted parts within an object. As a result, they may fail to detect corrupted images that involve missing or disarrayed semantic information in the object composition. On the contrary, human perception easily distinguishes such corruptions. To mitigate this gap, we introduce the concept of "image grammar", consisting of "image semantics" and "image syntax", to denote the semantics of parts or patches of an image and the order in which these parts are arranged to create a meaningful object. To learn the image grammar relative to a class of visual objects/scenes, we propose a weakly supervised two-stage approach. In the first stage, we use a deep clustering framework that relies on iterative clustering and feature refinement to produce part-semantic segmentation. In the second stage, we incorporate a recurrent bi-LSTM module to process a sequence of semantic segmentation patches to capture the image syntax. Our framework is trained to reason over patch semantics and detect faulty syntax. We benchmark the performance of several grammar learning models in detecting patch corruptions. Finally, we verify the capabilities of our framework in Celeb and SUNRGBD datasets and demonstrate that it can achieve a grammar validation accuracy of 70 to 90% in a wide variety of semantic and syntactical corruption scenarios.

Computer Vision and Pattern Recognition
Pictorial syntax

Kevin J. Lande

DOI: https://doi.org/10.1111/mila.12497

2024-01-03

Mind & Language

Abstract:It is commonly assumed that images, whether in the world or in the head, do not have a privileged analysis into constituent parts. They are thought to lack the sort of syntactic structure necessary for representing complex contents and entering into sophisticated patterns of inference. I reject this assumption. "Image grammars" are models in computer vision that articulate systematic principles governing the form and content of images. These models are empirically credible and can be construed as literal grammars for images. Images can have rich syntactic structure, though of a markedly different form than sentences in language.

psychology, experimental,linguistics
An Extended Grammar System for Learning and Recognizing Complex Visual Events

Zhang Zhang,Tieniu Tan,Kaiqi Huang

DOI: https://doi.org/10.1109/tpami.2010.60

IF: 23.6

2011-02-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:For a grammar-based approach to the recognition of visual events, there are two major limitations that prevent it from real application. One is that the event rules are predefined by domain experts, which means huge manual cost. The other is that the commonly used grammar can only handle sequential relations between subevents, which is inadequate to recognize more complex events involving parallel subevents. To solve these problems, we propose an extended grammar approach to modeling and recognizing complex visual events. First, motion trajectories as original features are transformed into a set of basic motion patterns of a single moving object, namely, primitives (terminals) in the grammar system. Then, a Minimum Description Length (MDL) based rule induction algorithm is performed to discover the hidden temporal structures in primitive stream, where Stochastic Context-Free Grammar (SCFG) is extended by Allen's temporal logic to model the complex temporal relations between subevents. Finally, a Multithread Parsing (MTP) algorithm is adopted to recognize interesting complex events in a given primitive stream, where a Viterbi-like error recovery strategy is also proposed to handle large-scale errors, e.g., insertion and deletion errors. Extensive experiments, including gymnastic exercises, traffic light events, and multi-agent interactions, have been executed to validate the effectiveness of the proposed approach.

computer science, artificial intelligence,engineering, electrical & electronic
Single-View 3D Scene Reconstruction and Parsing by Attribute Grammar

Xiaobai Liu,Yibiao Zhao,Song-Chun Zhu

DOI: https://doi.org/10.1109/tpami.2017.2689007

IF: 23.6

2017-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this paper, we present an attribute grammar for solving two coupled tasks: i) parsing a 2D image into semantic regions; and ii) recovering the 3D scene structures of all regions. The proposed grammar consists of a set of production rules, each describing a kind of spatial relation between planar surfaces in 3D scenes. These production rules are used to decompose an input image into a hierarchical parse graph representation where each graph node indicates a planar surface or a composite surface. Different from other stochastic image grammars, the proposed grammar augments each graph node with a set of attribute variables to depict scene-level global geometry, e.g., camera focal length, or local geometry, e.g., surface normal, contact lines between surfaces. These geometric attributes impose constraints between a node and its off-springs in the parse graph. Under a probabilistic framework, we develop a Markov Chain Monte Carlo method to construct a parse graph that optimizes the 2D image recognition and 3D scene reconstruction purposes simultaneously. We evaluated our method on both public benchmarks and newly collected datasets. Experiments demonstrate that the proposed method is capable of achieving state-of-the-art scene reconstruction of a single image.
Integrating Function , Geometry , Appearance for Scene Parsing

Yibiao Zhao,Song-Chun Zhu

2014-01-01

Abstract:In this paper, we present a Stochastic Scene Grammar (SSG) for parsing 2D indoor images into 3D scene layouts. Our grammar model integrates object functionality, 3D object geometry, and their 2D image appearance in a Function-Geometry-Appearance (FGA) hierarchy. In contrast to the prevailing approach in the literature which recognizes scenes and detects objects through appearance-based classification using machine learning techniques, our method takes a different perspective to scene understanding and recognizes objects and scenes by reasoning their functionality. Functionality is an essential property which often defines the categories of objects and scenes, and decides the design of geometry and scene layout. For example, a sofa is for people to sit comfortably, and a kitchen is a space for people to prepare food with various objects. Our SSG formulates object functionality and contextual relations between objects and imagined human poses in a joint probability distribution in the FGA hierarchy. The latter includes both functional concepts (the scene category, functional groups, functional objects, functional parts) and geometric entities (3D/2D/1D shape primitives). The decomposition of the grammar is terminated on the bottom-up detected lines and regions. We use a Markov chain Monte Carlo (MCMC) algorithm to optimize the Bayesian a posteriori probability and the output parse tree includes a 3D description of the 2D image in the FGA hierarchy. Experimental results on two Yibiao Zhao University of California, Los Angeles (UCLA), USA E-mail: ybzhao@ucla.edu www.yibiaozhao.com Song-Chun Zhu University of California, Los Angeles (UCLA), USA E-mail: sczhu@stat.ucla.edu http://www.stat.ucla.edu/~sczhu challenging indoor datasets demonstrate that the proposed approach not only significantly widens the scope of indoor scene parsing from traditional scene segmentation, labeling, and 3D reconstruction to functional object recognition, but also yields improved overall performance.
A Mathematical Theory of Textons and Primal Sketch: Integrating Generative and Descriptive Methods

Cheng-En Guo,Song-Chun Zhu

2005-01-01

Abstract:Textons, coined by Julesz in the 1960s, refer to the atomic structures in natural images and are considered the basic elements in early (pre-attentive) visual perception. The primal sketches, conjectured by Marr in the 1970s, refer to parsimonious token representation of generic images. Both concepts are fundamental to computer and human vision studies. However they have been very elusive due to the lack of rigorous mathematical definitions and models. The objective of this dissertation is to study a mathematical framework for the two concepts and thus to build a theoretical foundation for the early vision theory. Our study leads to generative image models for generic natural images and computing algorithms for learning, simulation, and inference. The framework integrates two main statistical modeling paradigms in the literature. (i) Hierarchic generative methods, such as transformed component analysis, wavelet coding, and sparse coding; and (ii) Descriptive methods, such as Markov random fields, graphical models, and minimax entropy learning. Generally speaking, the former decompose images into image elements, and the latter represent spatial relationships between the components. The dissertation makes the following contributions. (1) We define textons and image primitives as elements in the dictionaries of generative models which are learned from natural images through maximum likelihood estimation. (2) We study a Gestalt field for modeling the spatial relations among textons with reconfigurable neighborhoods. (3) We present a primal sketch model by dividing an image lattice into structural parts as textural parts which are modeled by the generative models and descriptive models respectively. (4) We study the information scaling phenomena and the transitions between various regimes of image representations, for example, the transition between textons and textures.
VLGrammar: Grounded Grammar Induction of Vision and Language

Yining Hong,Qing Li,Song-Chun Zhu,Siyuan Huang

DOI: https://doi.org/10.1109/ICCV48922.2021.00169

2021-01-01

Abstract:Cognitive grammar suggests that the acquisition of language grammar is grounded within visual structures. While grammar is an essential representation of natural language, it also exists ubiquitously in vision to represent the hierarchical part-whole structure. In this work, we study grounded grammar induction of vision and language in a joint learning framework. Specifically, we present VLGrammar, a method that uses compound probabilistic context-free grammars (compound PCFGs) to induce the language grammar and the image grammar simultaneously. We propose a novel contrastive learning framework to guide the joint learning of both modules. To provide a benchmark for the grounded grammar induction task, we collect a large-scale dataset, PARTIT, which contains human-written sentences that describe part-level semantics for 3D objects. Experiments on the PARTIT dataset show that VLGrammar outperforms all baselines in image grammar induction and language grammar induction. The learned VLGrammar naturally benefits related downstream tasks. Specifically, it improves the image unsupervised clustering accuracy by 30%, and performs well in image retrieval and text retrieval. Notably, the induced grammar shows superior generalizability by easily generalizing to unseen categories. Code and pre-trained models are released at https://github.com/evelinehong/VLGrammar.
Learning Object Consistency and Interaction in Image Generation from Scene Graphs

Yangkang Zhang,Chenye Meng,Zejian Li,Pei Chen,Guang Yang,Changyuan Yang,Lingyun Sun

DOI: https://doi.org/10.24963/ijcai.2023/192

2023-01-01

Abstract:This paper is concerned with synthesizing images conditioned on a scene graph (SG), a set of object nodes and their edges of interactive relations. We divide existing works into image-oriented and code-oriented methods. In our analysis, the image-oriented methods do not consider object interaction in spatial hidden feature. On the other hand, in empirical study, the code-oriented methods lose object consistency as their generated images omit certain objects in the input scene graph. To alleviate these two issues, we propose Learning Object Consistency and Interaction (LOCI). To preserve object consistency, we design a consistency module with a weighted augmentation strategy for objects easy to be ignored and a matching loss between scene graphs and image codes. To learn object interaction, we design an interaction module consisting of three kinds of message propagation between the input scene graph and the learned image code. Experiments on COCO-stuff and Visual Genome datasets show our proposed method alleviates the ignorance of objects and outperforms the state-of-the-art on visual fidelity of generated images and objects.
Holistic 3 D Indoor Scene Parsing and Reconstruction from a Single RGB Image

Siyuan Huang,Siyuan Qi,Yixin Zhu,Yinxue Xiao,Yuanlu Xu,Song-Chun Zhu

2018-01-01

Abstract:We propose a computational framework to parse and reconstruct the 3D configuration of an indoor scene from a single RGB image using a stochastic grammar model. We introduce a Holistic Scene Grammar (HSG) to represent the 3D scene structure, which characterizes a joint distribution over the functional and geometric space of indoor scenes. The proposed HSG captures three essential but often latent dimensions of the indoor scenes: i) latent human context, describing the affordance and the functionality of a room arrangement, ii) geometric constraints over the scene configurations, and iii) physical constraints that guarantee physically plausible parsing and reconstruction. We solve this parsing and reconstruction problem in an analysis-bysynthesis fashion, seeking to minimize the differences between the input image and the rendered image generated by our 3D representation, over the space of depth, surface normal, and object segmentation map. The optimal configuration (i.e., parse graph) is inferred using Markov chain Monte Carlo (MCMC), which efficiently traverses through the non-differentiable solution space, jointly optimizing object localization, 3D layout, and hidden human context.
Predicting Human Activities Using Stochastic Grammar.

Siyuan Qi,Siyuan Huang,Ping Wei,Song-Chun Zhu

DOI: https://doi.org/10.1109/iccv.2017.132

2017-01-01

Abstract:This paper presents a novel method to predict future human activities from partially observed RGB-D videos. Human activity prediction is generally difficult due to its non-Markovian property and the rich context between human and environments. We use a stochastic grammar model to capture the compositional structure of events, integrating human actions, objects, and their affordances. We represent the event by a spatial-temporal And-Or graph (ST-AOG). The ST-AOG is composed of a temporal stochastic grammar defined on sub-activities, and spatial graphs representing sub-activities that consist of human actions, objects, and their affordances. Future sub-activities are predicted using the temporal grammar and Earley parsing algorithm. The corresponding action, object, and affordance labels are then inferred accordingly. Extensive experiments are conducted to show the effectiveness of our model on both semantic event parsing and future activity prediction.
Configurable 3D Scene Synthesis and 2D Image Rendering with Per-pixel Ground Truth Using Stochastic Grammars

Chenfanfu Jiang,Siyuan Qi,Yixin Zhu,Siyuan Huang,Jenny Lin,Lap-Fai Yu,Demetri Terzopoulos,Song-Chun Zhu

DOI: https://doi.org/10.1007/s11263-018-1103-5

IF: 13.369

2018-01-01

International Journal of Computer Vision

Abstract:We propose a systematic learning-based approach to the generation of massive quantities of synthetic 3D scenes and arbitrary numbers of photorealistic 2D images thereof, with associated ground truth information, for the purposes of training, benchmarking, and diagnosing learning-based computer vision and robotics algorithms. In particular, we devise a learning-based pipeline of algorithms capable of automatically generating and rendering a potentially infinite variety of indoor scenes by using a stochastic grammar, represented as an attributed Spatial And-Or Graph, in conjunction with state-of-the-art physics-based rendering. Our pipeline is capable of synthesizing scene layouts with high diversity, and it is configurable inasmuch as it enables the precise customization and control of important attributes of the generated scenes. It renders photorealistic RGB images of the generated scenes while automatically synthesizing detailed, per-pixel ground truth data, including visible surface depth and normal, object identity, and material information (detailed to object parts), as well as environments (e.g., illuminations and camera viewpoints). We demonstrate the value of our synthesized dataset, by improving performance in certain machine-learning-based scene understanding tasks--depth and surface normal prediction, semantic segmentation, reconstruction, etc.--and by providing benchmarks for and diagnostics of trained models by modifying object attributes and scene properties in a controllable manner.
Learning of Structurally Unambiguous Probabilistic Grammars

Michal Ziv-Ukelson,Dolav Nitay,Dana Fisman

DOI: https://doi.org/10.46298/lmcs-19(1:10)2023

2023-02-08

Logical Methods in Computer Science

Abstract:The problem of identifying a probabilistic context free grammar has two aspects: the first is determining the grammar's topology (the rules of the grammar) and the second is estimating probabilistic weights for each rule. Given the hardness results for learning context-free grammars in general, and probabilistic grammars in particular, most of the literature has concentrated on the second problem. In this work we address the first problem. We restrict attention to structurally unambiguous weighted context-free grammars (SUWCFG) and provide a query learning algorithm for \structurally unambiguous probabilistic context-free grammars (SUPCFG). We show that SUWCFG can be represented using \emph{co-linear multiplicity tree automata} (CMTA), and provide a polynomial learning algorithm that learns CMTAs. We show that the learned CMTA can be converted into a probabilistic grammar, thus providing a complete algorithm for learning a structurally unambiguous probabilistic context free grammar (both the grammar topology and the probabilistic weights) using structured membership queries and structured equivalence queries. A summarized version of this work was published at AAAI 21.

computer science, theory & methods,logic

A stochastic grammar of images

A Stochastic Grammar of Images

A Stochastic Image Grammar for Fine-Grained 3D Scene Reconstruction.

Program Chairs' Introduction to the First International Workshop on Stochastic Image Grammars (Sig-09) in Conjunction with Ieee Cvpr 2009

A Stochastic Grammar for Natural Shapes

Image Parsing Via Stochastic Scene Grammar

Stochastic And-Or Grammars: A Unified Framework and Logic Perspective

Unsupervised Structure Learning of Stochastic And-Or Grammars

Human Parsing Using Stochastic And-or Grammars and Rich Appearances

Towards Image Semantics and Syntax Sequence Learning

Pictorial syntax

An Extended Grammar System for Learning and Recognizing Complex Visual Events

Single-View 3D Scene Reconstruction and Parsing by Attribute Grammar

Integrating Function , Geometry , Appearance for Scene Parsing

A Mathematical Theory of Textons and Primal Sketch: Integrating Generative and Descriptive Methods

VLGrammar: Grounded Grammar Induction of Vision and Language

Learning Object Consistency and Interaction in Image Generation from Scene Graphs

Holistic 3 D Indoor Scene Parsing and Reconstruction from a Single RGB Image

Predicting Human Activities Using Stochastic Grammar.

Configurable 3D Scene Synthesis and 2D Image Rendering with Per-pixel Ground Truth Using Stochastic Grammars

Learning of Structurally Unambiguous Probabilistic Grammars