تقرير
DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models
| العنوان: | DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models |
|---|---|
| المؤلفون: | Xing, Ximing, Wang, Chuang, Zhou, Haitao, Zhang, Jing, Yu, Qian, Xu, Dong |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence |
| الوصف: | Even though trained mainly on images, we discover that pretrained diffusion models show impressive power in guiding sketch synthesis. In this paper, we present DiffSketcher, an innovative algorithm that creates \textit{vectorized} free-hand sketches using natural language input. DiffSketcher is developed based on a pre-trained text-to-image diffusion model. It performs the task by directly optimizing a set of B\'ezier curves with an extended version of the score distillation sampling (SDS) loss, which allows us to use a raster-level diffusion model as a prior for optimizing a parametric vectorized sketch generator. Furthermore, we explore attention maps embedded in the diffusion model for effective stroke initialization to speed up the generation process. The generated sketches demonstrate multiple levels of abstraction while maintaining recognizability, underlying structure, and essential visual details of the subject drawn. Our experiments show that DiffSketcher achieves greater quality than prior work. The code and demo of DiffSketcher can be found at https://ximinng.github.io/DiffSketcher-project/. Comment: Accepted by NIPS 2023. Project page: https://ximinng.github.io/DiffSketcher-project/ |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2306.14685 |
| رقم الأكسشن: | edsarx.2306.14685 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |