Building a layout dataset from editable design files

Layout ground truth for free

Layout models need to know where things are, how big they are, and what role they play. From a bitmap those labels have to be annotated by hand or inferred by a detector. From an editable vector file they can be computed, because the document already states them.

What to extract

Each of these comes straight out of the document, with no annotation pass:

• Bounding boxes for every element, from its geometry and transforms.
• Text roles, inferred reliably from type size, weight and position rather than guessed from pixels: headline, subhead, label, caption, footnote.
• Reading order, from document order combined with position.
• Z-order and grouping, from the element hierarchy itself.
• Page format and margins, from the viewBox and the extent of the content.

Turning one page into many examples

Because the file is editable, a single page yields supervised pairs that would otherwise need a designer: shift a block and re-render for a spacing counterfactual, lengthen a headline for an overflow case, swap a two-column grid for a three-column one and keep the content fixed.

The pairs are exact rather than approximate, which is what makes them useful as training signal rather than as augmentation noise.

The one thing to check first

That the text is live. A file whose copy has been converted to outlines looks identical and is useless for this task: the roles cannot be inferred, the strings cannot be read, and the boxes describe shapes rather than words.