EE ZOOM Seminar: On the Role of Model Robustness in CLIP-Based Generation
https://technion.zoom.us/j/92332436495
Electrical Engineering Systems ZOOM Seminar
Speaker: Mika Yagoda
M.Sc. student under the supervision of Prof. Raja Giryes
Wednesday, 30th July 2025, at 15:00
On the Role of Model Robustness in CLIP-Based Generation
Abstract
This seminar presents a study on the impact of model robustness in text-guided image generation using CLIP and Deep Image Prior. In this framework, random noise is fed into a Deep Image Prior network, which is optimized to produce an image aligned with a target text prompt in the CLIP embedding space. The research shows that robust CLIP models, such as Interpretability and TeCoA, generate significantly sharper and more semantically coherent images compared to standard CLIP models, which often produce blurry and weakly aligned outputs.
To further enhance generation quality, the seminar explores enhancement techniques including cropping, input transformations, and combining multiple robust models. These methods improve results for robust models, especially when used with a gradual optimization strategy. Prompt formulation is also shown to play a critical role in the quality of generated images. The findings underscore the value of robustness and augmentations in CLIP-guided pipelines.
השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום בצ'ט של שם מלא + מספר ת.ז.
