Teaching a Language Model to Draw
Bachelor's Thesis Faculty of Mathematics & Computer Science University of Bucharest · 2026

A text model learns to draw,
guided by a vision critic.

Mădălin Ioana Faculty of Mathematics and Computer Science University of Bucharest Bachelor's Thesis, 2026

Abstract

An Artist language model writes an SVG sketch from a text prompt; being text-only, it never sees what it drew. A separate Critic vision model looks at the rendered image and answers in plain language, naming a single region to fix next. The Artist revises and the loop repeats. The hard part is regression: acting on the Critic's notes, a naive Artist often damages parts that were already correct. The contribution of this work is a code-enforced region lock that confines every revision to the one region the Critic named, so earlier progress is kept by construction and the drawing can only improve from step to step. The gain over a single pass is real, but bounded by how capable the underlying models are.

Keywords. Artist–Critic loop; region lock; monotonic refinement; SVG generation; vision–language critique

prompt → Artist → SVG → render → Critic → feedback (1)

1.Agent architecture
typed feedback graph prompt p s_t : SVG I_t = render(s_t) r_t, v_t, f_{t+1} feedback f_t Artist language model text -> SVG Renderer deterministic SVG -> pixels Critic vision model image -> JSON p, f_t -> A -> s_t render(s_t) -> C -> f_{t+1}
Figure 1. Typed agent graph. The Artist writes SVG, the renderer produces pixels, and the Critic returns a structured judgment (a score, a verdict, and one localized correction naming a single action and target region) that drives the next iteration.

st = A(p, ft),   (rt, vt, ft+1) = C(render(st)) (2)

The interface is deliberately narrow. The Artist emits SVG; the Critic replies with a score, a verdict, and exactly one correction tied to a named region. That named region is what region lock relies on: every path the Critic does not name is frozen in code, byte for byte, so a revision can only add to or refine the drawing. It cannot quietly undo work that is already correct.

2.Iterative protocol
Algorithm 1. Critic-guided SVG refinement.
  1. 1Input: text prompt p, maximum iterations T
  2. 2Initialize feedback f as the empty string, target region g as none.
  3. 3for t = 1, ..., T do
  4. 4Artist writes SVG from p and previous feedback f.
  5. 5Region lock: outside region g, restore the previous iteration's geometry.
  6. 6Render the locked SVG to a raster image.
  7. 7Critic returns score, verdict, feedback, and one target region.
  8. 8if accept, score ≥ 9, and every required part is present then return drawing.
  9. 9Set f and g to the Critic's feedback and target region.
  10. 10end for
  11. 11return final drawing and Critic report.

Note. The Artist never sees the image; it only ever reads the Critic’s words. Because a revision is confined to the one region the Critic named, accepted strokes carry over untouched. The run halts only on a clear accept, meaning a high score with every required part present, or when the iteration cap is reached.

3.Qualitative example

A single pass of the loop. The Artist sketches the subject stroke by stroke; the Critic looks at the result and answers in plain words. This is iteration three of a run on “a cat”: the drawing now reads clearly, but it still needs one more refinement pass.

(a) Artist sketch s3, rendered from SVG path data.

Prompt “a cat”
Iteration 3 / 4
Score 8 / 10
Verdict revise
…

(b) Critic output returned to the Artist as text.

Figure 2. Representative iteration from a cat prompt. The left panel is the rendered Artist SVG; the right panel is the Critic response used for the next refinement pass.
4.Region lock & score

The central contribution of this work is the region lock mechanism: once the Critic accepts a named region, that geometry is frozen in code and cannot be overwritten by any later Artist revision. The guarantee is structural, so the score sequence cannot decrease; this follows from the loop, not from the model. The graph below shows the result: Critic scores that rise across iterations because region lock enforces it, not because the model was trained to produce that.

Result 1. Region lock produces a non-decreasing score sequence. In the representative run, r = (3, 4, 5, 6, 7, 7, 8, 9); the plateau at r = 7 is where the Artist needed two passes to improve a locked region before the Critic issues accept at t = 8.
10 8 6 4 2 0 1 2 3 4 5 6 7 8 score r_t iteration t
Figure 3. Pgfplots-style score trace for one refinement run. The curve is redrawn in reading order; the final marker denotes the accepted drawing.
References
  1. Sequential Generation of Vector Drawings via an Artist–Critic Loop. Bachelor's thesis, Univ. Bucharest, 2026.
  2. learn-to-draw-step-by-step. Source code, GitHub, 2026.
A.Interactive reproduction

This page is also a small reproduction cell. Live runs stream stroke by stroke from the inference backend.

prompt
iterations
4
backend

backend: checkingLooking for backend

Local reproduction command:

make local ARTIST=gemma3:27b CRITIC=blaifa/InternVL3_5:8b

Full setup is in the setup notes ↗.