|
Muyao Niu Hi, I'm Muyao Niu. I am a 2nd year master student in the Department of Mechano-Informatics, the University of Tokyo (UTokyo). My supervisor is Prof. Yinqiang Zheng . I received my B.E. degree from Dalian University of Technology in 2022. My research interests include Computational Photography, AIGC, and 3D Vision. |
|
Research |
|
MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model
Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, Yinqiang Zheng ECCV, 2024 project / paper / code We introduce MOFA-Video to adapt motions from different domains to the frozen Video Diffusion Model. MOFA-Video can effectively animate a single image using various types of control signals, including trajectories, keypoint sequences, and their combinations. |
|
CV-VAE: A Compatible Video VAE for Latent Generative Video Models
Sijie Zhao, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Muyao Niu, Xiaoyu Li, Wenbo Hu, Ying Shan arXiv, 2024 project / paper / code We propose CV-VAE that is compatible with existing image and video models trained with SD image VAE. Our video VAE provides a truly spatio-temporally compressed latent space for latent generative video models, as opposed to uniform frame sampling. |
|
Physics-Based Adversarial Attack on Near-Infrared Human Detector for Nighttime Surveillance Camera Systems
Muyao Niu, Zhuoxiao Li Yifan Zhan, Huy H. Nguyen, Isao Echizen, Yinqiang Zheng ACM MM, 2023 paper / code We introduced an innovative approach that passively manipulates the intensity distribution of NIR images and developed a 3D-aware, black-box attack algorithm to target deep learning-based NIR-powered human detection systems. |
|
NIR-assisted Video Enhancement via Unpaired 24-hour Data
Muyao Niu, Zhihang Zhong, Yinqiang Zheng ICCV, 2023 paper / code We addressed the issue of collecting data for utilizing NIR images to improve low-light VIS videos. Physiscs-inspired algorithms are designed to simulate pseudo paired data of NIR and VIS images, simulating day-to-night situations. We then trained an enhancement network using the generated pseudo data. |
|
Visibility Constrained Wide-band Illumination Spectrum Design for Seeing-in-the-Dark
Muyao Niu, Zhuoxiao Li, Zhihang Zhong, Yinqiang Zheng CVPR, 2023 paper / code We designed an optimal illumination spectrum in the VIS-NIR range by considering human vision constraints, which significantly improves translation performance. A fully differentiable model was proposed, which includes the imaging process, human visual perception, and the enhancement network. |
|
Region Assisted Sketch Colorization
Ning Wang*, Muyao Niu*, Zhihui Wang, Kun Hu, Bin Liu, Zhiyong Wang, Haojie Li TIP, 2023 paper we proposed the Region-Assisted Sketch Colorization (RASC) method, which uses a 'Region Map' to better utilize regional information within the sketch, enhancing the perception of region-wise features. |
Interns
2023.12 - Now: Computer Vision Research Intern at Computer Vision Center, Tencent AI Lab, mentored by Xiaodong Cun.
|
Awards/Scholarships
WING-CFS Special Research Assistant Scholarship, The University of Tokyo, 2023.04 - 2027.09 |
|
The template comes from the personal website of Jon Barron. |