Ongoing projects
World Model for Controllable Generation
We propose a controllable world model that can generate future frame while controlling ego motion, scene object motion and human motion.
Accelerating Diffusion Model Training
We propose LayerSync, a domain-agnostic approach for improving the generation quality and the training efficiency of diffusion models.
Accelerating Diffusion Model Inference
We propose SenCache, a grounded caching criterion based on network sensitivity for accelerating diffusion model inference.
Semantically-aware Discriminators
We build on successful cGAN models to propose a new semantically-aware discriminator that better guides the generator. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning.
Collaborative Sampling in GANs
We propose a collaborative sampling scheme between the generator and discriminator for improved data generation. Guided by the discriminator, our approach refines generated samples through gradient-based optimization in the data (or feature / latent) space, shifting the generator distribution closer to the real data distribution.