How We Built the AI QR Generator Using Stable Diffusion + ControlNet

When we first sketched the idea of AI-generated QR codes, we had no idea how hard it would be to make them actually scan. Plenty of tools let you overlay an image on a QR code and call it "AI art." We wanted something different: QR codes where the art and the data are genuinely fused — not just stacked.

Why ControlNet Was the Right Choice

ControlNet is a neural network architecture that adds spatial conditioning to Stable Diffusion. Instead of letting the model generate freely, ControlNet constrains the output to follow a structural map — in our case, the binary QR data matrix.

We evaluated three approaches before settling on ControlNet:

1. 1Post-process overlay — paste art on top of a generated QR code. Fast but ugly; scan rates dropped dramatically.
2. 2Img2img guidance — start from a QR image and diffuse toward a style. Better aesthetics but inconsistent scan reliability.
3. 3ControlNet conditioning — use the QR matrix as a structural constraint during the forward diffusion pass. Best of both worlds.

“The moment our first ControlNet-guided QR code scanned cleanly on a phone six feet away, we knew we were on to something.”

— James Okonkwo, Growth Engineer

The Prompt Engineering Problem

Every AI QR generation starts with a text prompt. Early on we let users write their own prompts — the results were all over the place. A user who wrote "restaurant" got something completely different from someone who wrote "Italian restaurant with warm lighting, rustic wood tables, candles."

We solved this by building a prompt expansion layer. The user provides a short business description (e.g. "sushi bar in Tokyo"). Our system expands this into a full SD prompt that specifies style, lighting, mood, composition, color palette, and crucially, the quality tokens that make Stable Diffusion produce sharp, high-contrast results.

Ensuring Every Output Scans

The hardest engineering problem was reliability. A QR code that doesn't scan is useless — even if it looks stunning. We added a verification step at the end of every generation pipeline:

• Decode the output image using two different QR decoders (ZXing and ZBar)
• Verify the decoded URL matches the input URL exactly
• If either decoder fails, regenerate with higher error correction
• Maximum 3 retries before returning an error to the user

What's Next for AI QR

We're working on animated QR codes (short MP4 loops), style transfer from uploaded brand images, and much tighter integration between the AI generator and the custom QR editor. Stay tuned.