Lukas Höllein

I'm a PhD student at the Visual Computing & Artificial Intelligence Lab at the Technical University of Munich, supervised by Prof. Dr. Matthias Nießner.

During summer 2023, I completed a research scientist internship at Meta Reality Labs in Zurich under the guidance of Aljaž Božič.

I received a Master's Degree in Computer Science from the Technical University of Munich and a Bachelor's Degree in Computer Science from the Coburg University.

My research lies at the intersection of computer vision/graphics and machine learning, concerning mostly 3D reconstruction and generation. I'm especially interested in the creation of fully navigable 3D worlds with the help of generative AI.

Email  /  Google Scholar  /  Twitter  /  Github  /  CV

Research

Our method reconstructs 3D worlds from video diffusion models using non-rigid alignment to resolve inherent 3D inconsistencies in the generated sequences.

Intrinsic Image Fusion reconstructs high-quality physically based materials from multi-view images.

WorldExplorer generates 3D scenes from a given text prompt using camera-guided video diffusion models.

IntrinsiX generates high-quality intrinsic images from text descriptions. The decomposed image prior can be used for various applications, such as re-lighting, material editing, and texture generation.

3DGS-LM accelerates Gaussian-Splatting optimization by replacing the ADAM optimizer with Levenberg-Marquardt.

QuickSplat learns data-driven priors to generate dense initializations for 2D gaussian splatting optimization of large-scale indoor scenes. We further learn to jointly estimate the densification and update of the scene parameters during each iteration.

ViewDiff generates high-quality, multi-view consistent images of a real-world 3D object in authentic surroundings.

ControlRoom3D creates diverse and plausible 3D room meshes aligning well with user-defined room layouts and textual descriptions of the room style.

Text2Room generates textured 3D meshes from a given text prompt using 2D text-to-image models. The core idea of our approach is a tailored viewpoint selection such that the content of each image can be fused into a seamless, textured 3D mesh. More specifically, we propose a continuous alignment strategy that iteratively fuses scene frames with the existing geometry.

We apply style transfer on mesh reconstructions of indoor scenes. We optimize an explicit texture for the reconstructed mesh of a scene and stylize it jointly from all available input images. Our depth- and angle-aware optimization leverages surface normal and depth data of the underlying mesh to create a uniform and consistent stylization for the whole scene.

Miscellanea

• 15.01.2026 — Third Dimension: Towards Generating Fully Navigable 3D Scenes, Online
• 24.10.2024 — TUMVision: Indoor Scene Generation From Diffusion Models, Munich
• 14.10.2024 — Google: Indoor Scene Generation From Diffusion Models, Munich
• 06.06.2024 — Voxel51: 3D-Consistent Image Generation with Text-to-Image Models, Online

Thank you Jon Barron for providing the source code of this website.