Abstract:This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

Differentiable Convex Polyhedra Optimization from Multi-view Images

Determining Convexo-Concave Vertices of Polygonby Dynamic Distinct Partition

Generating Large Convex Polytopes Directly on Point Clouds

Multiview Textured Mesh Recovery by Differentiable Rendering

Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable Rendering Method

Differentiable Rendering of Parametric Geometry

A General Differentiable Mesh Renderer for Image-Based 3D Reasoning

Differentiable Rendering Approach to Mesh Optimization for Digital Human Reconstruction.

Efficient View Manipulation for Cuboid-Structured Images.

Curved Image Triangulation Based on Differentiable Rendering

Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering

Equidistant Smoothing of Polyhedra with Arbitrary Topologies

Differentiable Rendering Using RGBXY Derivatives and Optimal Transport.

Analytical Derivatives for Differentiable Renderer: 3D Pose Estimation by Silhouette Consistency

Multi-View Reconstruction using Signed Ray Distance Functions (SRDF)

Differentiable Rendering for Pose Estimation in Proximity Operations

A Simple Approach to Differentiable Rendering of SDFs

Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer

Pixel2Mesh++: Multi-View 3D Mesh Generation Via Deformation

FlexiDreamer: Single Image-to-3D Generation with FlexiCubes

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization