Mr. ZHANG, Tianyu（1st-year doctoral student in Xie Lab, Creative Society Design Research Area）received Best Presentation Award in 2024 2nd International Conference on Computer Graphics and Image Processing (CGIP2024).

CGIP2024 was sponsored by Tokai University and held at Kyoto University from January 12 to 15, 2024. CGIP conference series is a forum for presenting new advances and research results in the fields of Computer Graphics and Image Processing, bringing together leading researchers, engineers and scientists in the domain of interest from around the world. The scope of the conference covers all theoretical and practical aspects of the Computer Graphics and Image Processing.

* Reference：CGIP2024

■Date Awarded
January 14,2024

■Title
Sketch-Guided Text-to-Image Generation with Spatial Control

■Authors
Tianyu Zhang, Haoran Xie

■Abstract
Recent text-to-image generation models can produce high-quality images from textual prompts. However, it isn't easy to correctly interpret instructions specifying the complex images with multiple objects using only texts. To solve this issue, we propose a sketch-guided spatial control for text-to-image diffusion models. In the feature extraction stage of the proposed framework, sketch inputs are segmented into individual objects using the image segmentation approach. The obtained bounding boxes and labels are used as spatial-guided inputs into the attention layers of the diffusion model. The proposed model utilizes a pretrained text-to-image diffusion model as the image generator for the image generation stage. We assess the proposed method through quantitative and qualitative evaluations, demonstrating its versatility in spatial control based on user sketches.

■Comment
I am deeply honored to receive this Best Presentation Award. I would like to express my sincere gratitude to Associate Professor Xie Haoran for his invaluable guidance and to all the members of our lab for their constant encouragement. This award inspires me to take on higher challenges in future research. I will cherish this honor and strive relentlessly to contribute more to research.

February 2, 2024