Urban Landscape Game Scene Sketch Generation Framework with Stable Diffusion

서지정보

· 발행기관 : (사)한국컴퓨터게임학회
· 수록지 정보 : 한국컴퓨터게임학회논문지 / 36권 / 4호 / 99 ~ 103페이지
· 저자명 : 김준호, 양희경, 민경하

초록

Games are a vital part of our culture, leading the generation of tools such as Adobe Photoshop and Unity for game developers. Although sketches are a fundamental form that can be stylized in various ways, there is a scarcity of tools capable of generating images into sketch. To bridge this gap in the artistic sketch domain of gaming and the field of deep generative models, we propose a multimodal sketch generation framework with Stable Diffusion, focusing especially on urban scenes.
Stable Diffusion, a model within the diffusion-based category, has gained notable attention in the open-source community and is user-friendly. Thus, we have chosen to utilize Stable Diffusion in our approach. This model processes input prompts and images through the CLIP encoder and effectively generates images. However, Stable Diffusion faces challenges such as a losing locality of input image and difficulties in fine-tuning. To overcome these issues, we incorporate ControlNet and DreamBooth into our framework.
We conclude with a demonstration of promising results, urban landscape sketch, in both text-to-sketch and image-to-sketch generation.

참고자료

· 없음