EE Seminar: PractiLight: Practical Light Control Using Foundational Diffusion Models
הרישום לסמינר יבוצע באמצעות סריקת הברקוד למודל (יש להיכנס לפני כן למודל, לא באמצעות האפליקציה)- הרישום מסתיים ב- 16:10
Registration to the seminar will be done by scanning the barcode for the Moodle (Please enter ahead to the Moodle, NOT by application)- Registration ends at 16:10
Electrical Engineering Systems Seminar
Speaker: Yotam Erel
Ph.D. student under the supervision of Prof. Amit H. Bermano
Sunday, 22nd June 2025, at 16:00
Room 011, Kitot Building, Faculty of Engineering
PractiLight: Practical Light Control Using Foundational Diffusion Models
Abstract
Light control in generated images is a difficult task, posing specific challenges, spanning over the entire image and frequency spectrum. Most approaches tackle this problem by training on extensive yet domain-specific datasets, limiting the inherent generalization and applicability of the foundational backbones used. Instead, PractiLight is a practical approach, effectively leveraging foundational understanding of recent generative models for the task. Our key insight is that lighting relationships in an image are similar in nature to token interaction in self-attention layers, and hence are best represented there. Based on this and other analyses regarding the importance of early diffusion iterations, PractiLight trains a lightweight LoRA regressor to produce the direct light map for a given image, using a small set of training images. We then employ this regressor to incorporate the desired lighting into the generation process of another image using Classifier Guidance. This careful design generalizes well to diverse conditions and image domains. We demonstrate state-of-the-art performance in terms of quality and control with proven parameter and data efficiency compared to leading works over a wide variety of scene types. We hope this work affirms that image lighting can feasibly be controlled by tapping into foundational knowledge, enabling practical and general relighting.
