https://github.com/MarkMoHR/Awesome-Sketch-Synthesis

:books: A collection of papers about Sketch Synthesis (Generation).
https://github.com/MarkMoHR/Awesome-Sketch-Synthesis

List: Awesome-Sketch-Synthesis

sketch sketch-dataset sketch-generation sketch-generation-paper sketch-generation-papers sketch-synthesis sketch-synthesis-paper sketch-synthesis-papers stroke-level vector-sketch

Last synced: about 2 months ago
JSON representation

:books: A collection of papers about Sketch Synthesis (Generation).

• Host: GitHub
• URL: https://github.com/MarkMoHR/Awesome-Sketch-Synthesis
• Owner: MarkMoHR
• Created: 2019-03-13T05:25:13.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2024-05-22T11:46:05.000Z (7 months ago)
• Last Synced: 2024-05-22T12:49:55.978Z (7 months ago)
• Topics: sketch, sketch-dataset, sketch-generation, sketch-generation-paper, sketch-generation-papers, sketch-synthesis, sketch-synthesis-paper, sketch-synthesis-papers, stroke-level, vector-sketch
• Homepage:
• Size: 285 KB
• Stars: 414
• Watchers: 33
• Forks: 37
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• Awesome-Computer-Vision - 5
• ultimate-awesome - Awesome-Sketch-Synthesis - :books: A collection of papers about Sketch Synthesis (Generation). (Other Lists / Monkey C Lists)

README

        
# Awesome-Sketch-Synthesis

[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)

A collection of papers about Sketch Synthesis (Generation). Mainly focus on stroke-level vector sketch synthesis.

> Feel free to create a PR or an issue.

![examples](https://magenta.tensorflow.org/assets/sketch_rnn_demo/img/sketch_garden.gif)

**Outlines**

- [0. Survey](#0-survey)

- [1. Datasets](#1-datasets)

- [2. Sketch-Synthesis Approaches](#2-sketch-synthesis-approaches)

  - [1) Semantic Concept-to-sketch](#1-semantic-concept-to-sketch)

  - [2) Photo-to-sketch](#2-photo-to-sketch)

  - [3) Text/Attribute-to-sketch](#3-textattribute-to-sketch)

  - [4) 3D shape-to-sketch](#4-3d-shape-to-sketch)

  - [5) Art-to-sketch](#5-art-to-sketch)

- [3. Vector Graphics Generation (2D)](#3-vector-graphics-generation-2d)

- [4. Vector Graphics Generation (3D)](#4-vector-graphics-generation-3d)

---

## 0. Survey

| Paper | Source | Code/Project Link  |

| --- | --- | --- |

| [Deep Learning for Free-Hand Sketch: A Survey](https://ieeexplore.ieee.org/abstract/document/9706366) | TPAMI 2022 | [[code]](https://github.com/PengBoXiangShang/torchsketch) |

## 1. Datasets

Here `Vector strokes` means having *svg* data. `With photos` means having the photo-sketch paired data.

  

    Level

    Dataset

    Source

    Vector strokes

    With photos

    Notes

  

  

    Characters

     Omniglot  

      

     :heavy_check_mark:  

     :x:  

     Alphabets characters 

  

  

     KanjiVG  

      

     :heavy_check_mark:  

     :x:  

     Chinese characters 

  

  

     Kuzushiji  

      

     :x:  

     :x:  

     Japanese characters 

  

  

    Icon

     FIGR-8-SVG  

      

     :heavy_check_mark:  

     :x:  

     Icons with text descriptions 

  

  

    Systematic Symbol

     SketchIME  

     ACM MM 2023 

     :heavy_check_mark:  

     :x:  

     Systematic sketches with semantic annotations 

  

  

    Instance-level

     TU-Berlin  

     SIGGRAPH 2012  

     :heavy_check_mark:  

     :x:  

     Multi-category hand sketches 

  

  

     Sketchy  

     SIGGRAPH 2016  

     :heavy_check_mark:  

     :heavy_check_mark:  

     Multi-category photo-sketch paired 

  

  

     QuickDraw  

     ICLR 2018  

     :heavy_check_mark:  

     :x:  

     Multi-category hand sketches 

  

  

     QMUL-Shoe-Chair-V2  

     CVPR 2016  

     :heavy_check_mark:  

     :heavy_check_mark:  

     Only two categories 

  

  

     Sketch Perceptual Grouping (SPG)  

     ECCV 2018  

     :heavy_check_mark:  

     :x:  

     With part-level semantic segmentation information 

  

  

     FaceX  

     AAAI 2019  

     :heavy_check_mark:  

     :x:  

     Labeled facial sketches   

  

  

     Creative Sketch  

     ICLR 2021  

     :heavy_check_mark:  

     :x:  

     With annotated part segmentation   

  

  

     ImageNet-Sketch  

     NeurIPS 2019  

     :x:  

     :x:  

     50 images for each of the 1000 ImageNet classes   

  

  

     SEVA  

     NeurIPS 2023  

     :heavy_check_mark:  

     :heavy_check_mark:  

     90K human-generated sketches that vary in detail   

  

  

    Scene-level

     SketchyScene  

     ECCV 2018  

     :x:  

     :heavy_check_mark:  

     With semantic/instance segmentation information   

  

  

     CMPlaces  

     TPAMI 2018  

     :x:  

     :heavy_check_mark:  

     Cross-modal scene dataset   

  

  

     Context-Skecth  

     Expressive 2018  

     :x:  

     :heavy_check_mark:  

     Context-based scene sketches for co-classification   

  

  

     SketchyCOCO  

     CVPR 2020  

     :x:  

     :heavy_check_mark:  

     Scene sketch, segmentation and normal images   

  

  

     FS-COCO  

     ECCV 2022  

     :heavy_check_mark:  

     :heavy_check_mark:  

     Scene sketches with text description   

  

  

     SFSD  

     VC 2022  

     :heavy_check_mark:  

     :heavy_check_mark:  

     Completely hand-drawn scene sketches with label annotation   

  

  

    Drawing from photos

     Photo-Sketching  

     WACV 2019  

     :heavy_check_mark:  

     :heavy_check_mark:  

     ScenePhoto-sketch paired 

  

  

     Tracing-vs-Freehand  

     SIGGRAPH 2021  

     :heavy_check_mark:  

     :heavy_check_mark:  

     Tracings and freehand drawings of images   

  

  

    Drawing from 3D models

     DifferSketching  

     SIGGRAPH Asia 2022  

     :heavy_check_mark:  

     :x:  

     3D model-sketch paired, with novice and professional ones 

  

  

    Portrait

     CUFS  

     TPAMI 2009  

     :x:  

     :heavy_check_mark:  

     Face-sketch pairs   

  

  

     APDrawing  

     CVPR 2019  

     :x:  

     :heavy_check_mark:  

     Portrait-sketch paired 

  

  

     SKSF-A  

     EG 2024  

     :x:  

     :heavy_check_mark:  

     Face-sketch pairs of seven styles   

  

  

    Children's Drawing

     Amateur Drawings  

     TOG 2023  

     :x:  

     :heavy_check_mark:  

     With character bounding boxes, segmentation masks, and joint location annotations 

  

  

    Rough sketch

     Da Vinci  

     CGI 2018  

     :x:  

     :heavy_check_mark:  

     Line drawing restoration dataset 

  

  

     Rough Sketch Benchmark  

     SIGGRAPH Asia 2020  

     :heavy_check_mark:  

     :heavy_check_mark:  

     Rough and clean sketch pairs (only for evaluation)   

  

  

    CAD

     ld3d  

     SIGGRAPH 2008  

     :x:  

     :x:  

     Line Drawings of 3D Shapes 

  

  

     OpenSketch  

     SIGGRAPH Asia 2019  

     :heavy_check_mark:  

     :x:  

     Product Design Sketches 

  

  

     SketchGraphs  

     ICML 2020 Workshop  

     :heavy_check_mark:  

     :x:  

     Sketches extracted from real-world CAD models   

  

  

     Fusion 360 Gallery  

     SIGGRAPH 2021  

     :heavy_check_mark:  

     :x:  

     For 'sketch and extrude' designs   

  

  

     FloorPlanCAD  

     ICCV 2021  

     :heavy_check_mark:  

     :x:  

     With instance and semantic annotations   

  

  

    Anime

     Danbooru2021  

     /  

     :x:  

     :heavy_check_mark:  

     Anime images annotated with tags 

  

  

     DanbooRegion  

     ECCV 2020  

     :x:  

     :x:  

     Anime images with region annotations 

  

  

     Danbooru-Parsing  

     TOG 2023  

     :x:  

     :heavy_check_mark:  

     For anime portrait parsing and anime translation 

  

  

     CreativeFlow+  

     CVPR 2019  

     :x:  

     :heavy_check_mark:  

     Large densely annotated artistic video dataset 

  

  

     ATD-12K  

     CVPR 2021  

     :x:  

     :heavy_check_mark:  

     Animation frames with flow annotations 

  

  

     AnimeRun  

     NeurIPS 2022  

     :x:  

     :heavy_check_mark:  

     Correspondence dataset for 2D-styled cartoons 

  

  

     AnimeCeleb  

     ECCV 2022  

     :x:  

     :x:  

     Animation head images with pose annotations 

  

  

     PaintBucket-Character  

     CVPR 2024  

     :x:  

     :heavy_check_mark:  

     Animation frames with region annotations 

  

  

     Sakuga-42M  

     arxiv 24.05  

     :x:  

     :heavy_check_mark:  

     Cartoon videos with text descriptions and tags 

  

  

     Anita  

     online 2024  

     :x:  

     :heavy_check_mark:  

     Professional hand-drawn cartoon keyframes, with 1080P sketch and color images 

  

  

## 2. Sketch-Synthesis Approaches

### 1) Semantic Concept-to-sketch

  

    Level

    Paper

    Source

    Code/Project Link

  

  

    Instance-level

     A Neural Representation of Sketch Drawings (sketch-rnn)  

     ICLR 2018  

     

      [Code] 

      [Project] 

      [Demo] 

    

  

  

     Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories  

       

     

      [Code] 

    

  

  

     AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches  

     AAAI 2019  

     [Project] 

  

  

     Stroke-based sketched symbol reconstruction and segmentation (stroke-rnn)  

     CGA 2019  

     

  

  

     BézierSketch: A generative model for scalable vector sketches  

     ECCV 2020  

     

      [Code]  

    

  

  

     Pixelor: A Competitive Sketching AI Agent. So you think you can beat me?  

     SIGGRAPH Asia 2020  

     

      [Project] 

      [Code] 

    

  

  

     Creative Sketch Generation  

     ICLR 2021  

     

      [Project] 

      [Code] 

    

  

  

     Computer-Aided Design as Language  

     arxiv 2105  

     

    

  

  

     DoodleFormer: Creative Sketch Drawing with Transformers  

     ECCV 2022  

     

      [Project] 

      [Code] 

    

  

  

     SketchKnitter: Vectorized Sketch Generation with Diffusion Models  

     ICLR 2023  

     

      [Code] 

    

  

  

     Self-Organizing a Latent Hierarchy of Sketch Patterns for Controllable Sketch Synthesis  

     TNNLS 2023  

     

      [Code] 

    

  

  

     Scale-Adaptive Diffusion Model for Complex Sketch Synthesis  

     ICLR 2024  

     

    

  

  

---

### 2) Photo-to-sketch

- vector image generation

  

    Data type

    Paper

    Source

    Code/Project Link

  

  

    Facial

     Style and abstraction in portrait sketching  

     TOG 2013  

    

    

  

  

    Instance-level

     Free-Hand Sketch Synthesis with Deformable Stroke Models  

     IJCV 2017  

    

      [Project] 

      [code] 

    

  

  

     Learning to Sketch with Shortcut Cycle Consistency  

     CVPR 2018  

     [Code1] [Code2] 

  

  

     Learning Deep Sketch Abstraction  

     CVPR 2018  

      

  

  

     CLIPasso: Semantically-Aware Object Sketching  

     SIGGRAPH 2022  

     

      [Project] 

      [Code]

    

  

  

    Scene-level

     Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale  

     AAAI 2021  

    

      [Code] 

    

  

  

     CLIPascene: Scene Sketching with Different Types and Levels of Abstraction  

     ICCV 2023  

     

      [Project] 

      [Code] 

    

  

  

     Learning Realistic Sketching: A Dual-agent Reinforcement Learning Approach  

     ACM MM 2024  

     

    

  

  

    Technical Drawings

     Deep Vectorization of Technical Drawings  

     ECCV 2020  

    

      [Project] 

      [code] 

    

  

- raster image generation

  

    Type

    Paper

    Source

    Code/Project Link

  

  

    Facial

     ArtLine  

     Online demo  

     

      [Code] 

    

  

  

     APDrawingGAN: Generating Artistic Portrait Drawings from Face Photos with Hierarchical GANs  

     CVPR 2019  

     

      [Code] 

      [Demo] 

    

  

  

     Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping  

     CVPR 2020  

     

      [Code] 

    

  

  

     Line Drawings for Face Portraits From Photos Using Global and Local Structure Based GANs  

     TPAMI 2020  

     

      [Code] 

    

  

  

     Quality Metric Guided Portrait Line Drawing Generation from Unpaired Training Data  

     TPAMI 2022  

     

      [Code] 

    

  

  

     Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation  

     ICCV 2023  

     

      [Code] 

    

  

  

     Stylized Face Sketch Extraction via Generative Prior with Limited Data  

     EG 2024  

     

      [Code] 

      [Project] 

    

  

  

    Instance-level

     Deep Factorised Inverse-Sketching  

     ECCV 2018  

     

  

  

     Synthesizing human-like sketches from natural images using a conditional convolutional decoder  

     WACV 2020  

     

      [Code] 

    

  

  

    Anime

     sketchKeras  

     online demo  

     

      [Code] 

    

  

  

     LineDistiller  

     online demo  

    

      [Code] 

    

  

  

     Anime2Sketch  

     online demo  

    

      [Code] 

    

  

  

     Reference Based Sketch Extraction via Attention Mechanism  

     SIGGRAPH Asia 2022  

    

      [Code] 

    

  

  

    Scene-level

     Photo-Sketching: Inferring Contour Drawings from Images  

     WACV 2019  

    

      [Code] 

      [Project] 

    

  

  

     Learning to generate line drawings that convey geometry and semantics  

     CVPR 2022  

     

      [Code] 

      [Project] 

    

  

  

    Arbitrary

     Semi-supervised reference-based sketch extraction using a contrastive learning  

     SIGGRAPH 2023  

    

      [Code] 

      [Project] 

    

  

---

### 3) Text/Attribute-to-sketch

- raster image generation

| Type | Paper | Source | Code/Project Link |

| --- | --- | --- | --- |

| **Facial** | [Text2Sketch: Learning Face Sketch from Facial Attribute Text](https://ieeexplore.ieee.org/abstract/document/8451236) | ICIP 2018 |  |

| **Scene-level** | [Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications](https://arxiv.org/pdf/1904.04399.pdf) | UIST 2019 |  |

| **Scene-level** | [Scones: Towards Conversational Authoring of Sketches](http://people.eecs.berkeley.edu/~eschoop/docs/scones.pdf) | IUI 2020 |  |

- vector image generation

| Type | Paper | Source | Code/Project Link |

| --- | --- | --- | --- |

| **Arbitrary** | [Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts](https://arxiv.org/abs/2109.08857) | arxiv 21.09 | [[code]](https://github.com/google/brain-tokyo-workshop) [[project]](https://es-clip.github.io/) |

| **Arbitrary** | [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://arxiv.org/abs/2106.14843) | NeurIPS 2022 | [[code]](https://colab.research.google.com/github/kvfrans/clipdraw/blob/main/clipdraw.ipynb) |

| **Arbitrary** | [StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation](https://arxiv.org/abs/2202.12362) | IJCAI 2022 | [[code]](https://github.com/pschaldenbrand/StyleCLIPDraw) |

| **SVG** | [VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models](https://arxiv.org/abs/2211.11319) | CVPR 2023 |  [[project]](https://ajayj.com/vectorfusion) |

| **Arbitrary** | [SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation](https://arxiv.org/abs/2308.14191) | BMVC 2023 |  [[code]](https://github.com/WinKawaks/SketchDreamer) |

| **Arbitrary** | [DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models](https://arxiv.org/abs/2306.14685) | NeurIPS 2023 |  [[project]](https://ximinng.github.io/DiffSketcher-project/) [[code]](https://github.com/ximinng/DiffSketcher) |

| **Icon** | [IconShop: Text-Based Vector Icon Synthesis with Autoregressive Transformers](https://arxiv.org/abs/2304.14400) | SIGGRAPH Asia 2023 |  [[project]](https://icon-shop.github.io/) |

| **SVG** | [Text-Guided Vector Graphics Customization](https://arxiv.org/abs/2309.12302) | SIGGRAPH Asia 2023 |  [[project]](https://intchous.github.io/SVGCustomization/)  |

| **Arbitrary** | [Text-based Vector Sketch Editing with Image Editing Diffusion Prior](https://www.sysu-imsl.com/files/ICME2024/ICME2024_sketch_editing_final.pdf) | ICME 2024 |  [[code]](https://github.com/MarkMoHR/DiffSketchEdit)  |

| **SVG** | [SVGDreamer: Text Guided SVG Generation with Diffusion Model](https://arxiv.org/abs/2312.16476) | CVPR 2024 |  [[project]](https://ximinng.github.io/SVGDreamer-project/) [[code]](https://github.com/ximinng/SVGDreamer) |

| **SVG** | [NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generation](https://arxiv.org/abs/2405.15217) | CVPR 2024 |  [[project]](https://vikastmz.github.io/NIVeL/)  |

| **SVG** | [Text-to-Vector Generation with Neural Path Representation](https://arxiv.org/abs/2405.10317) | SIGGRAPH 2024 |  [[project]](https://intchous.github.io/T2V-NPR/)  |

| **Arbitrary** | [SVGCraft: Beyond Single Object Text-to-SVG Synthesis with Comprehensive Canvas Layout](https://arxiv.org/abs/2404.00412v1) | arxiv 24.04 | [[code]](https://github.com/ayanban011/SVGCraft) |

| **SVG** | [VectorPainter: A Novel Approach to Stylized Vector Graphics Synthesis with Vectorized Strokes](https://arxiv.org/abs/2405.02962) | arxiv 24.05 |  |

---

### 4) 3D shape-to-sketch

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [DeepShapeSketch : Generating hand drawing sketches from 3D objects](https://shizhezhou.github.io/projects/DeepFreeHandSke2019/deepFreehandSke2019.pdf) | IJCNN 2019 |  |

| [Neural Contours: Learning to Draw Lines from 3D Shapes](https://people.cs.umass.edu/~dliu/papers/NeuralContours.pdf) | CVPR 2020 | [[project]](https://people.cs.umass.edu/~dliu/projects/NeuralContours/) [[code]](https://github.com/DifanLiu/NeuralContours) |

| [Cloud2Curve: Generation and Vectorization of Parametric Sketches](https://arxiv.org/abs/2103.15536) | CVPR 2021 | [[project]](https://ayandas.me/pubs/2021/03/01/pub-9.html) |

| [Neural Strokes: Stylized Line Drawing of 3D Shapes](https://openaccess.thecvf.com/content/ICCV2021/papers/Liu_Neural_Strokes_Stylized_Line_Drawing_of_3D_Shapes_ICCV_2021_paper.pdf) | ICCV 2021 | [[code]](https://github.com/DifanLiu/NeuralStrokes) |

| [Learning a Style Space for Interactive Line Drawing Synthesis from Animated 3D Models](https://graphics.cs.yale.edu/sites/default/files/linedrawinganimation_pg2022.pdf) | PG 2022 |  |

| [CAD2Sketch: Generating Concept Sketches from CAD Sequences](https://repo-sam.inria.fr/d3/cad2sketch/cad2sketch_paper.pdf) | SIGGRAPH Asia 2022 | [[project]](https://ns.inria.fr/d3/cad2sketch/) |

---

### 5) Art-to-sketch

Here we list sketch synthesis based on other image types, like Manga, line art, rough sketch, etc.

a) Line art

- Vector-to-Vector

| Paper | Source | Code/Project Link | Deep learning? |

| --- | --- | --- | --- |

| [Closure-aware Sketch Simplification](http://www.cse.cuhk.edu.hk/~ttwong/papers/sketch/sketch.pdf) | SIGGRAPH Asia 2015 | [[Project]](https://www.cse.cuhk.edu.hk/~ttwong/papers/sketch/sketch.html) | No |

| [StrokeAggregator: Consolidating Raw Sketches into Artist-Intended Curve Drawings](https://www.cs.ubc.ca/labs/imager/tr/2018/StrokeAggregator/StrokeAggregator_authorversion.pdf) | SIGGRAPH 2018 | [[Project]](https://www.cs.ubc.ca/labs/imager/tr/2018/StrokeAggregator/) | No |

| [StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters](https://www.davepagurek.com/programming/strokestrip/) | SIGGRAPH 2021 | [[Project]](https://www.davepagurek.com/programming/strokestrip/) [[code]](https://github.com/davepagurek/StrokeStrip) | No |

| [StripMaker: Perception-driven Learned Vector Sketch Consolidation](https://dl.acm.org/doi/abs/10.1145/3592130) | SIGGRAPH 2023 |  | No |

| [Region-Aware Simplification and Stylization of 3D Line Drawings](https://onlinelibrary.wiley.com/doi/10.1111/cgf.15042) | EG 2024 |  | No |

- Raster-to-Vector (a.k.a. Vectorization)

| Paper | Source | Code/Project Link | Deep learning? |

| --- | --- | --- | --- |

| [Topology-Driven Vectorization of Clean Line Drawings](https://cgl.ethz.ch/Downloads/Publications/Papers/2013/Nor13/Nor13.pdf) | TOG 2013 | | No |

| [Fidelity vs. Simplicity: a Global Approach to Line Drawing Vectorization](https://www-sop.inria.fr/reves/Basilic/2016/FLB16/fidelity_simplicity.pdf) | SIGGRAPH 2016 | [[Project]](https://www-sop.inria.fr/reves/Basilic/2016/FLB16/) | No |

| [A Delaunay triangulation based approach for cleaning rough sketches](https://ed.iitm.ac.in/~raman/agcl/sketchSMI.pdf) | C&G 2018 | [[Code]](https://github.com/amaldevp/RoughSketchSimplification) | No |

| [Semantic Segmentation for Line Drawing Vectorization Using Neural Networks](http://www.byungsoo.me/project/vectornet/paper.pdf) | EG 2018 | [[project]](http://www.byungsoo.me/project/vectornet) [[code]](https://github.com/byungsook/vectornet) | Yes |

| [Deep Line Drawing Vectorization via Line Subdivision and Topology Reconstruction](https://www.researchgate.net/profile/Chu_Han8/publication/337249870_Deep_Line_Drawing_Vectorization_via_Line_Subdivision_and_Topology_Reconstruction/links/5f0fec43299bf1e548ba370f/Deep-Line-Drawing-Vectorization-via-Line-Subdivision-and-Topology-Reconstruction.pdf) | PG 2019 | | Yes |

| [Inertia-based Fast Vectorization of Line Drawings](https://www.researchgate.net/profile/Rafal_Scherer/publication/336936860_Inertia-based_Fast_Vectorization_of_Line_Drawings/links/5dbbc8ad299bf1a47b0721d1/Inertia-based-Fast-Vectorization-of-Line-Drawings.pdf) | PG 2019 | | No |

| [Vectorization of Line Drawings via Polyvector Fields](https://dl.acm.org/doi/10.1145/3202661) | TOG 2019 | [[Code]](https://github.com/bmpix/PolyVectorization) | No |

| [Integer-Grid Sketch Simplification and Vectorization](http://www-sop.inria.fr/reves/Basilic/2020/SBBB20/paper.pdf) | SGP 2020 | [[Project]](https://repo-sam.inria.fr/d3/grid-vectorization/) [[Code]](https://gitlab.inria.fr/D3/grid-vectorization/) | No |

| [Deep Vectorization of Technical Drawings](https://arxiv.org/abs/2003.05471) | ECCV 2020 | [[project]](http://adase.group/3ddl/projects/vectorization/) [[code]](https://github.com/Vahe1994/Deep-Vectorization-of-Technical-Drawings) | Yes |

| [General Virtual Sketching Framework for Vector Line Art](https://esslab.jp/publications/HaoranSIGGRAPH2021.pdf) | SIGGRAPH 2021 | [[project]](https://markmohr.github.io/virtual_sketching/) [[code]](https://github.com/MarkMoHR/virtual_sketching) | Yes |

| [Keypoint-Driven Line Drawing Vectorization via PolyVector Flow](http://www-labs.iro.umontreal.ca/~bmpix/pdf/polyvector_flow.pdf) | SIGGRAPH Asia 2021 | [[project]](https://puhachov.xyz/publications/keypoint-driven-polyvector-flow/) | Hybrid |

| [End-to-end Line Drawing Vectorization](https://www.aaai.org/AAAI22Papers/AAAI-52.LiuH.pdf) | AAAI 2022 |  | Yes |

| [Vectorizing Line Drawings of Arbitrary Thickness via Boundary-based Topology Reconstruction](https://onlinelibrary.wiley.com/doi/full/10.1111/cgf.14485) | CGF 2022 |  | No |

| [Singularity-Free Frame Fields for Line Drawing Vectorization](https://diglib.eg.org/handle/10.1111/cgf14901) | SGP 2023 | [[code]](https://github.com/SingularityFreeFrameFields/Code) | No |

| [Deep Sketch Vectorization via Implicit Surface Extraction](https://cragl.cs.gmu.edu/sketchvector/Deep%20Sketch%20Vectorization%20via%20Implicit%20Surface%20Extraction%20(Chuan%20Yan,%20Yong%20Li,%20Deepali%20Aneja,%20Matthew%20Fisher,%20Edgar%20Simo-Serra,%20Yotam%20Gingold%202024%20SIGGRAPH).pdf) | SIGGRAPH 2024 | [[project]](https://cragl.cs.gmu.edu/sketchvector/) [[code]](https://github.com/Nauhcnay/Deep-Sketch-Vectorization) | Hybrid |

b) Rough sketch simplification / cleanup

- Datasets and benchmark

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [A Benchmark for Rough Sketch Cleanup](https://hal.archives-ouvertes.fr/hal-02939477/document) | SIGGRAPH Asia 2020 | [[Project]](https://cragl.cs.gmu.edu/sketchbench/)  [[Code]](https://github.com/Nauhcnay/A-Benchmark-for-Rough-Sketch-Cleanup) |

- Methods

| Paper | Source | Code/Project Link |

| --- | --- | --- | 

| [Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup](https://esslab.jp/publications/SimoSerraSIGGRAPH2016.pdf) | SIGGRAPH 2016 | [[Code]](https://github.com/bobbens/sketch_simplification) [[Project]](https://esslab.jp/~ess/en/research/sketch/) |

| [Mastering Sketching: Adversarial Augmentation for Structured Prediction](https://esslab.jp/~ess/publications/SimoSerraTOG2018.pdf) | SIGGRAPH 2018 | [[Code]](https://github.com/bobbens/sketch_simplification)  [[Project]](https://esslab.jp/~ess/en/research/sketch_master/) |

| [Real-Time Data-Driven Interactive Rough Sketch Inking](https://esslab.jp/~ess/publications/SimoSerraSIGGRAPH2018.pdf) | SIGGRAPH 2018 | [[Code]](https://github.com/bobbens/line_thinning) [[Project]](https://esslab.jp/~ess/en/research/inking/) |

| [Perceptual-aware Sketch Simplification Based on Integrated VGG Layers](https://ieeexplore.ieee.org/abstract/document/8771128/) | TVCG 2019 |  |

c) Manga (Comics)

| Paper | Source | Code/Project Link |

| --- | --- | --- | 

| [Deep extraction of manga structural lines](https://dl.acm.org/citation.cfm?id=3073675) | SIGGRAPH 2017 | [[Code]](https://github.com/ljsabc/MangaLineExtraction) |

| [Manga Filling Style Conversion with Screentone Variational Autoencoder](https://www.cse.cuhk.edu.hk/~ttwong/papers/screenstyle/screenstyle.html) | SIGGRAPH Asia 2020 | [[Project]](https://www.cse.cuhk.edu.hk/~ttwong/papers/screenstyle/screenstyle.html) [[Code]](https://github.com/msxie92/ScreenStyle) |

| [Generating Manga from Illustrations via Mimicking Manga Workflow](https://openaccess.thecvf.com/content/CVPR2021/papers/Zhang_Generating_Manga_From_Illustrations_via_Mimicking_Manga_Creation_Workflow_CVPR_2021_paper.pdf) | CVPR 2021 | [[Project]](https://lllyasviel.github.io/MangaFilter/) [[Code]](https://github.com/lllyasviel/MangaFilter) |

| [MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing](https://arxiv.org/abs/2004.10634) | AAAI 2021 | |

| [MARVEL: Raster Gray-level Manga Vectorization via Primitive-wise Deep Reinforcement Learning](https://ieeexplore.ieee.org/abstract/document/10233891) | TCSVT 2023 | |

## 3. Vector Graphics Generation (2D)

Here we focus on learning-based vector graphics generation **without** depending on vector training data, and traditional vectorization algorithms.

- Learning with external black-box (non-differentiable) rendering simulator

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [Synthesizing Programs for Images using Reinforced Adversarial Learning](http://proceedings.mlr.press/v80/ganin18a/ganin18a.pdf) | ICML 2018 | [[Code]](https://github.com/deepmind/spiral) |

| [Unsupervised Doodling and Painting with Improved SPIRAL](https://arxiv.org/abs/1910.01007) | arxiv 1910 | [[Project]](https://learning-to-paint.github.io/) |

- Learning with built-in differentiable rendering module

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [Layered Image Vectorization via Semantic Simplification](https://arxiv.org/abs/2406.05404) | arxiv 24.06 | [[webpage]](https://szuviz.github.io/layered_vectorization/) [[code]](https://github.com/SZUVIZ/layered_vectorization/) |

| [Segmentation-guided Layer-wise Image Vectorization with Gradient Fills](https://arxiv.org/abs/2408.15741) | ECCV 2024 | |

| [Towards High-fidelity Artistic Image Vectorization via Texture-Encapsulated Shape Parameterization](https://openaccess.thecvf.com/content/CVPR2024/papers/Chen_Towards_High-fidelity_Artistic_Image_Vectorization_via_Texture-Encapsulated_Shape_Parameterization_CVPR_2024_paper.pdf) | CVPR 2024 |  |

| [SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis](https://openaccess.thecvf.com/content/CVPR2024/papers/Hu_SuperSVG_Superpixel-based_Scalable_Vector_Graphics_Synthesis_CVPR_2024_paper.pdf) | CVPR 2024 | [[code]](https://github.com/sjtuplayer/SuperSVG) |

| [Vector Graphics Generation via Mutually Impulsed Dual-domain Diffusion](https://openaccess.thecvf.com/content/CVPR2024/papers/Zhao_Vector_Graphics_Generation_via_Mutually_Impulsed_Dual-domain_Diffusion_CVPR_2024_paper.pdf) | CVPR 2024 |  |

| [Optimize and Reduce: A Top-Down Approach for Image Vectorization](https://arxiv.org/abs/2312.11334) | AAAI 2024 | [[code]](https://github.com/ajevnisek/optimize-and-reduce) |

| [Segmentation-Based Parametric Painting](https://arxiv.org/abs/2311.14271) | arxiv 23.11 | [[code]](https://github.com/manuelladron/semantic_based_painting) [[project]](https://manuelladron.github.io/semantic_based_painting/) |

| [Editable Image Geometric Abstraction via Neural Primitive Assembly](https://openaccess.thecvf.com/content/ICCV2023/papers/Chen_Editable_Image_Geometric_Abstraction_via_Neural_Primitive_Assembly_ICCV_2023_paper.pdf) | ICCV 2023 |  |

| [Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region](https://arxiv.org/abs/2309.03504) | ACM MM 2023 | [[code]](https://github.com/sjtuplayer/Compositional_Neural_Painter) |

| [Intelli-Paint: Towards Developing More Human-Intelligible Painting Agents](https://arxiv.org/abs/2112.08930) | ECCV 2022 | [[project]](https://1jsingh.github.io/intelli-paint) |

| [Towards Layer-wise Image Vectorization](https://ma-xu.github.io/LIVE/index_files/CVPR22_LIVE_main.pdf) | CVPR 2022 | [[code]](https://github.com/ma-xu/LIVE) [[project]](https://ma-xu.github.io/LIVE/) |

| [Paint Transformer: Feed Forward Neural Painting with Stroke Prediction](https://arxiv.org/abs/2108.03798) | ICCV 2021 | [[code]](https://github.com/Huage001/PaintTransformer) |

| [Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings](https://openaccess.thecvf.com/content/CVPR2021/papers/Singh_Combining_Semantic_Guidance_and_Deep_Reinforcement_Learning_for_Generating_Human_CVPR_2021_paper.pdf) | CVPR 2021 | [[code]](https://github.com/1jsingh/semantic-guidance) |

| [Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes](https://arxiv.org/abs/2103.17185) | CVPR 2021 | [[code]](https://github.com/CompVis/brushstroke-parameterized-style-transfer) |

| [Im2Vec: Synthesizing Vector Graphics without Vector Supervision](https://arxiv.org/abs/2102.02798) | CVPR 2021 | [[Project]](http://geometry.cs.ucl.ac.uk/projects/2021/im2vec/) [[code]](https://github.com/preddy5/Im2Vec) |

| [Stylized Neural Painting](https://arxiv.org/abs/2011.08114) | CVPR 2021 | [[Code]](https://github.com/jiupinjia/stylized-neural-painting) [[project]](https://jiupinjia.github.io/neuralpainter/) |

| [Learning to Paint With Model-based Deep Reinforcement Learning](http://openaccess.thecvf.com/content_ICCV_2019/papers/Huang_Learning_to_Paint_With_Model-Based_Deep_Reinforcement_Learning_ICCV_2019_paper.pdf) | ICCV 2019 | [[code]](https://github.com/megvii-research/ICCV2019-LearningToPaint) |

| [Strokenet: A neural painting environment](https://openreview.net/forum?id=HJxwDiActX) | ICLR 2019 | [[Code]](https://github.com/vexilligera/strokenet) |

| [Neural Painters: A learned differentiable constraint for generating brushstroke paintings](https://arxiv.org/abs/1904.08410) | arxiv 1904 | [[Code]](https://github.com/reiinakano/neural-painters-pytorch) |

| [Learning to Sketch with Deep Q Networks and Demonstrated Strokes](https://arxiv.org/abs/1810.05977) | arxiv 1810 |  |

| [Unsupervised Image to Sequence Translation with Canvas-Drawer Networks](https://arxiv.org/abs/1809.08340) | arxiv 1809 |  |

- Vectorization

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [Depixelizing pixel art](https://dl.acm.org/doi/abs/10.1145/1964921.1964994) | SIGGRAPH 2011 |  |

| [Perception-Driven Semi-Structured Boundary Vectorization](http://www.cs.ubc.ca/labs/imager/tr/2018/PerceptionDrivenVectorization/perception-driven-vectorization.pdf) | SIGGRAPH 2018 | [[Webpage]](http://www.cs.ubc.ca/labs/imager/tr/2018/PerceptionDrivenVectorization/) |

| [PolyFit: Perception-aligned Vectorization of Raster Clip-art via Intermediate Polygonal Fitting](http://www.cs.ubc.ca/labs/imager/tr/2020/ClipArtVectorization/paper.pdf) | SIGGRAPH 2020 | [[Webpage]](http://www.cs.ubc.ca/labs/imager/tr/2020/ClipArtVectorization/) [[Code]](https://github.com/dedoardo/polyfit) |

| [ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis](https://ieeexplore.ieee.org/abstract/document/9444657) | TVCG 2021 |  |

| [TCB-Spline-Based Image Vectorization](https://dl.acm.org/doi/10.1145/3513132) | TOG 2022 |  |

| [Image vectorization and editing via linear gradient layer decomposition](https://cragl.cs.gmu.edu/gradientlayers/Image%20vectorization%20and%20editing%20via%20linear%20gradient%20layer%20decomposition%20(Zheng-Jun%20Du,%20Liang-Fu%20Kang,%20Jianchao%20Tan,%20Yotam%20Gingold,%20Kun%20Xu%202023%20SIGGRAPH)%20small.pdf) | SIGGRAPH 2023 |  |

## 4. Vector Graphics Generation (3D)

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [3Doodle: Compact Abstraction of Objects with 3D Strokes](https://arxiv.org/abs/2402.03690) | SIGGRAPH 2024 | [[code]](https://github.com/changwoonchoi/3Doodle) |

| [Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Rendering](https://arxiv.org/abs/2405.15305) | arxiv 24.05 | [[webpage]](https://yiboz2001.github.io/Diff3DS/) |

- Wire Art

| Paper | Source | Code/Project Link |

| --- | --- | --- |

| [Wired Perspectives: Multi-View Wire Art Embraces Generative AI](https://openaccess.thecvf.com/content/CVPR2024/papers/Qu_Wired_Perspectives_Multi-View_Wire_Art_Embraces_Generative_AI_CVPR_2024_paper.pdf) | CVPR 2024 | [[code]](https://github.com/WinKawaks/DreamWire) [[webpage]](https://dreamwireart.github.io/) |

| [Fabricable 3D Wire Art](https://doi.org/10.1145/3641519.3657453) | SIGGRAPH 2024 |  |