Abstract:Learning visual features from unlabeled image data is an important yet challenging task, which is often achieved by training a model on some annotation-free information. We consider spatial contexts, for which we solve so-called jigsaw puzzles, i.e., each image is cut into grids and then disordered, and the goal is to recover the correct configuration. Existing approaches formulated it as a classification task by defining a fixed mapping from a small subset of configurations to a class set, but these approaches ignore the underlying relationship between different configurations and also limit their applications to more complex scenarios. This paper presents a novel approach which applies to jigsaw puzzles with an arbitrary grid size and dimensionality. We provide a fundamental and generalized principle, that weaker cues are easier to be learned in an unsupervised manner and also transfer better. In the context of puzzle recognition, we use an iterative manner which, instead of solving the puzzle all at once, adjusts the order of the patches in each step until convergence. In each step, we combine both unary and binary features of each patch into a cost function judging the correctness of the current configuration. Our approach, by taking similarity between puzzles into consideration, enjoys a more efficient way of learning visual knowledge. We verify the effectiveness of our approach from two aspects. First, it solves arbitrarily complex puzzles, including high-dimensional puzzles, that prior methods are difficult to handle. Second, it serves as a reliable way of network initialization, which leads to better transfer performance in visual recognition tasks including classification, detection and segmentation.

Solve JIGSAW Puzzle Problems with Co-Normalization.

A Non-Local Method For Robust Noisy Image Completion

JigsawNet: Shredded Image Reassembly Using Convolutional Neural Network and Loop-Based Composition

CNLPA-MVS: Coarse-Hypotheses Guided Non-Local PatchMatch Multi-View Stereo

Convergence Analysis of Graph Regularized Non-Negative Matrix Factorization

Iterative Reorganization with Weak Spatial Constraints: Solving Arbitrary Jigsaw Puzzles for Unsupervised Representation Learning

Motif-induced Graph Normalization

Jigsaw Puzzle Solving Using Local Feature Co-Occurrences in Deep Neural Networks

InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction

Normalized Filter Pool for Prior Modeling of Nature Images

JigsawGAN: Auxiliary Learning for Solving Jigsaw Puzzles with Generative Adversarial Networks

Manifold constrained joint sparse learning via non-convex regularization

Multi-channel Nuclear Norm Minus Frobenius Norm Minimization for Color Image Denoising

Local Image Tagging Via Graph Regularized Joint Group Sparsity

A Markov Random Field Model-Based Approach to Natural Image Matting

Solving Jigsaw Puzzles Via Nonconvex Quadratic Programming With The Projected Power Method

Parsing Images into Regions, Curves, and Curve Groups

Image reconstructions using sparse dictionary representations and implicit, non-negative mappings

Nuclear Norm Regularized Coding with Local Position-Patch and Nonlocal Similarity for Face Hallucination.

Image Inpainting Using Non-Convex Low Rank Decomposition and Multidirectional Search

Sparse Norm-Regularized Reconstructive Coefficients Learning