update

README.md

@@ -1,11 +1,12 @@
-# Surf-D: High-Quality Surface Generation for Arbitrary Topologies using Diffusion Models
+# Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models
 
 <img src="./assets/fig_teaser.png">
 
 **[Project Page](https://yzmblog.github.io/projects/SurfD/)**
 | **[Paper](https://arxiv.org/abs/2311.17050)**
 
->we present **Surf-D**, a novel method for generating high-quality 3D shapes as **Surfaces** with arbitrary topologies using **Diffusion** models. Specifically, we adopt Unsigned Distance Field (UDF) as the surface representation, as it excels in handling arbitrary topologies, enabling the generation of complex shapes. While the prior methods explored shape generation with different representations, they suffer from limited topologies and geometry details. Moreover, it's non-trivial to directly extend prior diffusion models to UDF because they lack spatial continuity due to the discrete volume structure. However, UDF requires accurate gradients for mesh extraction and learning. To tackle the issues, we first leverage a point-based auto-encoder to learn a compact latent space, which supports gradient querying for any input point through differentiation to effectively capture intricate geometry at a high resolution. Since the learning difficulty for various shapes can differ, a curriculum learning strategy is employed to efficiently embed various surfaces, enhancing the whole embedding process. With pretrained shape latent space, we employ a latent diffusion model to acquire the distribution of various shapes. Our approach demonstrates superior performance in shape generation across multiple modalities and conducts extensive experiments in unconditional generation, category conditional generation, 3D reconstruction from images, and text-to-shape tasks.
+>We present **Surf-D**, a novel method for generating high-quality 3D shapes as **Surfaces** with arbitrary topologies using **Diffusion** models. Previous methods explored shape generation with different representations and they suffer from limited topologies and poor geometry details. To generate high-quality surfaces of arbitrary topologies, we use the Unsigned Distance Field (UDF) as our surface representation to accommodate arbitrary topologies. Furthermore, we propose a new pipeline that employs a point-based AutoEncoder to learn a compact and continuous latent space for accurately encoding UDF and support high-resolution mesh extraction. We further show that our new pipeline significantly outperforms the prior approaches to learning the distance fields, such as the grid-based AutoEncoder, which is not scalable and incapable of learning accurate UDF. In addition, we adopt a curriculum learning strategy to efficiently embed various surfaces. With the pretrained shape latent space, we employ a latent diffusion model to acquire the distribution of various shapes. Extensive experiments are presented on using Surf-D for unconditional generation, category conditional generation, image conditional generation, and text-to-shape tasks. The experiments demonstrate the superior performance of Surf-D in shape generation across multiple modalities as conditions.
+
 
 ## Unconditional Generation
 <img src="./assets/fig_unconditional.gif">
@@ -20,10 +21,10 @@ Given the category condition, Surf-D generates different categories of detailed
 We explore more applications that Surf-D can be applied to. As shown in the video, the clothes generated by Surf-D can be used for virtual try-on with high quality and fidelity. Imagine that you can just use sketches to generate whatever clothes you want, then put on your own avatar to try-on. Although it may sound crazy, this can be achieved with our proposed Surf-D!
 
 ## Single-view 3D Reconstruction
-<img src="./assets/fig_pix3d.png">
+<img src="./assets/fig_pix3d.gif">
 Given single-view images of objects, Surf-D can produce high-quality results faithfully aligned with input images.
 
 ## Text2Shape
-<img src="./assets/fig_text2shape.png">
+<img src="./assets/fig_text2shape.gif">
 Give the text description of objects, Surf-D produces high-quality results aligned with input texts.