We present a generative model for stroke-based drawing tasks which is able to model complex free-form structures. While previous approaches rely on sequence-based models for drawings of basic objects or handwritten text, we propose a model that treats drawings as a collection of strokes that can be composed into complex structures such as diagrams (e.g., flow-charts). At the core of the approach lies a novel auto-encoder that projects variable-length strokes into a latent space of fixed dimension. This representation space allows a relational model, operating in latent space, to better capture the relationship between strokes and to predict subsequent strokes. We demonstrate qualitatively and quantitatively that our proposed approach is able to model the appearance of individual strokes, as well as the compositional structure of larger diagram drawings. Our approach is suitable for interactive use cases such as auto-completing diagrams.
Video
Poster
(Click poster to view full size)
Completion Results
Flowchart samples generated by our model. Given a number of strokes drawn by the users (in black color), our model autogressively completes the drawing stroke-by-stroke. At every prediction step, the generated strokes are illustrated with different colors.
In the accompanying figures (right or below), we visualize the attention weights during the prediction of a particular stroke (in red color). Darker blue corresponds to larger attention weights.
