How to train a text-to-SVG model

The task

Text-to-SVG asks a model to emit a valid vector document from a description. It is a sequence-generation problem with a hard constraint: the output must parse, and it must render to something that looks like what was asked for.

That constraint is what makes the data requirements unusual. A near-miss in natural language is still readable. A near-miss in SVG is a file that will not open.

Preparing the data

Four steps before any training run:

• Normalise. Collapse the many ways SVG can express the same drawing into one: consistent path commands, absolute or relative coordinates chosen once, a fixed precision, attributes in a stable order.
• Decide about text. Live text nodes are the signal you are paying for, so keep them as text; outlining them at this stage throws away exactly what distinguishes the corpus.
• Choose a tokenisation. Path data can be tokenised as characters, as commands with numeric arguments, or as a learned vocabulary of sub-paths. Keep the raw and the normalised string so the choice can be revisited without re-ingesting.
• Pair with captions. A caption that describes the layout as well as the subject teaches more than one that names the subject alone.

What to measure

Render-validity rate first, because it is the floor: the share of generated files that parse and render at all. Then perceptual agreement between the render and the reference, and a text-image score for whether the output matches the prompt.

Hold out an evaluation set stratified by complexity before you start. Simple and dense pages fail differently, and an average over both hides which one your model is losing.

The common failure

Models trained on icon corpora produce single centred objects, whatever the prompt asks for, because that is the only composition they have seen. If the target is designed pages, the training data has to contain designed pages.