Use of generative AI in type design: The state-of-the-art Survey

Within the scope of the searched project between the years 2010 and 2024, I’ve found 80 projects [20] and identified four main applicable AI tasks that can be leveraged in type design. In this article, I present the survey methodology, application description and methods, aswell as references to the particular projects.

Every type designer who stumbled upon a variable font ¹ project has faced a frequent issue. Some parameters of variable font masters ² don’t match, which prevents the calculation of the required instances in the design space ³. Unlike variable font interpolation, the interpolation methods of AI don’t necessarily face the issue and don’t even require masters to match all the properties.

The interpolation process calculates intermediate positions P_i of glyph outlines control points P with coordinates (x, y) at a given value v along an axis. The linear interpolation ⁴ formula commonly used in vector graphics is P_i​(v) = P_min​ + v ⋅ (P_max​ − P_min​) ⁵ where:

• P_min​ is the position of the control point in the master at the minimum value of the axis.
• P_max​ is the position of the control point in the master at the maximum value of the axis.
• v is the normalised value along the axis ranging from 0 to 1.

For the latent space font, interpolation is usually used the same formula of linear interpolation P_i​(v) = P_min​ + v ⋅ (P_max​ − P_min​). However, the common notation of the formula in machine learning is f_ip​ = (1 − λ) ⋅ f(a) + λ ⋅ f(b)⁶ where:

• f_ip​represents interpolated latent vector
• f(a) and f(b) represent the two fonts
• λ represents a parameter ranging from 0 to 1 that controls the interpolation level

Font-MF [3] employs a Gaussian process latent variable model (GP-LVM) [13] to represent a manifold [9:157] of 46 fonts from Google Fonts library [1]. Manifold (Latent Space) works here like a map of the fonts, and the interpolation is performed by navigating through this map ⁷. Hence, interpolation is not performed between two fonts through latent space but directly within the latent space by traversing through it. Font-MF [3] method has the potential for type foundries to exploit their existing fonts as a tool to explore new ideas. The interface of the manifold map is arbitrary and can be replaced by commonly known sliders representing the axes of their fonts. The limitation of the Font-MF system lies in the requirement for the same topological structure of the fonts. That means horrific typographic creatures are not allowed.

DeepSVG[5] is the first to use deep learning-based methods for vector graphics. DeepSVG has brought latent space operations to interpolate between two vector graphics in order to provide vector animations between two SVG images. The system leverages the autoencoder [9:499] technique, which involves two components. First, the encoder works like a memory of the system that has a memorised representation of all the seen fonts. The encoder is tasked to give two latent vectors of the two given fonts that represent the recollection of the two fonts. The two vectors are linearly interpolated, and values are forwarded to the second network—decoder. In the end, the decoder generates a new font—the interpolated instance of the master fonts. Latent vectors f(a) and f(b) represent represent hierarchical SVG command structures. Since DeepSVG comes with a robust animation system, it can be used not only for the generation of ideas but also to visualise the ideas as animations between given fonts.

DeepVecFont [29] inspired by the success of DeepSVG [5] uses a similar technique. Additionally, DeepVecFont latent space memorises both the control points (the sequential representation) and raster images of the glyphs (the raster image representation) ⁸. The interpolated latent space position is fed to decoders and later refined by a neural differentiable rasteriser that refines the initially synthesised glyphs to match the target style more closely. Latent vectors f(a) and f(b) represent font styles and structural characteristics. Unlike DeepSVG [5], which generates SVGs, DeepVecFont is capable of generating complete glyphs, which makes it more suitable for prototyping new font ideas.

SVGformer [4] uses a transformer-based model with geometric self-attention to capture both semantic and geometric relationships in SVG data. Latent vectors f(a) and f(b) capture continuous geometric and semantic features of SVGs.

VecFontSDF [31], instead of encoding curves to latent space directly, uses raster images that are used to calculate the signed distance function (SDF) of parabolic curves for each glyph. Then, the parabolic curves are converted into Bézier curves to create the final vector glyphs. Although the system generates appealing shapes, the lost original Bézier drawings are costly, making the system unsuitable for font production.

DeepVecFont-v2 [30], DualVector [16], and VecFusion [27] are the descendants of the dual-modality idea of DeepVecFont [29]. The image features and sequence features are likewise combined into a unified latent space representation. After the interpolation, the interpolated feature is fed to both image and sequence decoders and refined. Since all three systems use a robust system leveraging the type design tool FontForge, the implementation of its interpolation techniques into the type design process is more straightforward than in the previous systems.

In our table related to a font completion task have been found since 2010, starting with morphable template models [25] that complete fonts from users’ skeleton sketches. It is important to mention the repository aggregating the few-shot font generation project created by Song Park [22].

[11] proposed an end-to-end font style transfer system that generates large-scale Chinese font with only a small number of training samples called a few-shot font training stacked conditional GAN model to generate a set of multi-content glyph images following a consistent style from a few input samples, which is one of the few. [6] focused on compositional scripts and proposed a font generation framework that employed memory components and global-context awareness in the generator to take advantage of the compositionality. [2] proposed a framework that introduced deep metric learning to style encoders. [14] proposed a GAN based on a translator that can generate fonts by observing only a few samples from other languages. [21] proposes a model using the components of the initial, middle, and final components of Hangul. [23] employed attention system so-called local experts to extract multiple style features not explicitly conditioned on component labels. [24] introduced the deep implicit representation of the fonts, which is different from any of the other projects since their data representation remained in the raster. However, the generation ended with raster images. [26] proposed an approach by learning fine-grained local styles from references and the spatial correspondence between the content and reference glyphs. [18] proposed a model that simultaneously learns to separate font styles in the embedding space where distances directly correspond to a measure of font similarity and translates input images into the given observed or unobserved font style. [15] proposed a self-supervised cross-modality pre-training strategy and a cross-modality transformer-based encoder that is conditioned jointly on the glyph image and the corresponding stroke labels. [10] proposed a cross-lingual font generator based on AGIS-Net [8].

The above-mentioned attempts remain in the raster domain, which is not considered suitable for the font industry application ⁹.

FontCycle [32] leverages graph-based representation to extract style features from raster images and graph transformer networks to generate complete fonts from a few image samples. Although the graph-based representation seems to be a prospective approach to encoding spatial information of glyph shapes, the absence of original Bézier curve drawings is a limitation, which makes the system lag behind the other approaches.

DeepVecFont [29] works with dual-modality representation that efficiently exploits advances of raster image modality domain and vector outlines suitable for type design. These features make DeepVecFont a hot candidate for font type designers, helping them complete their fonts from initially provided few glyphs.

VecFontSDF [31] calculates quadratic Bézier curves from raster image input by exploiting the signed distance function. Similarly to FontCycle [32], the VecFontSDF [31] system drawback from the industry implementation is that it lacks real vector drawing inputs.

DeepVecFont-v2 [30] as a descendant of capable DeepVecFont [29] improved the limitation of sequential encoding of a variable number of drawing commands by introducing transformers instead of original LSTM. The system is capable of exceptional results, which makes it a good candidate for font completion of complex fonts.

DualVector [16], similar to DeepVecFont [29] and DeepVecFont-v2 [30], leverages both raster and sequential encoding. However, the input uses solely raster images of glyphs, which is again unfortunate, as original drawings aren’t used.

VecFusion [27] leverages recent advances in diffusion models to encode raster images and control point fields. However, the loss of vector drawings as input is again drawn back from use in the type design industry.

A multimodal or cross-modal generation represents a domain that aims to generate its outputs from different modalities of their inputs. For instance text-to-image, text-to-speech, image-to-text etc. In the font domain, it could be text-to-font, image-to-font.

[7] ’s proposed framework, based on GANs, enables the generated fonts to reflect human emotional information distinctive from scanned faces. This approach is considered as image-to-font as the input is scanned images, and the output is generated fonts.

[12] aimed to understand correlations between impressions andfont styles through a shared latent space in which the font and its impressions are embedded nearby. This approach is considered text-to-font as the input is impression defined in texts, and the output is generated fonts.

[17] aimed to generate fonts with specific impressions with a font dataset with impression labels. Similar to previous work, it is considered as text-to-font as the input is impression defined in texts, and the output is generated fonts.

[28] tried to analyse the given impression of fonts by training the Transformer network. They didn’t generate fonts. This approach is considered font-to-text as the input is a font, and the output generates text describing the impression.