How does text-to-image AI work, and how do you use it well?

What happens between the prompt and the picture

Most text-to-image systems have three parts. A text encoder turns the prompt into a list of numbers that capture its meaning. An image generator creates a picture conditioned on those numbers. A decoder turns the generator's internal representation into real pixels.

The most common generator type at the time of writing is a diffusion model. It starts from pure random noise and, over a series of steps, predicts and removes a little noise each time, guided by the text. Latent diffusion models, the family behind Stable Diffusion, do this in a compressed space instead of on full-size pixels, which makes them much cheaper to run. Other systems are autoregressive: they produce the image as a sequence of tokens, one after another, much like a language model writes words. Janus-Pro, for example, works this way. Both kinds take the same inputs from you, so the workflow below applies to either.

1. Your prompt is split into tokens and encoded.
2. A random starting point is created from the seed.
3. The generator refines that start for a set number of steps, each step pulled toward the prompt.
4. The decoder outputs the final image at the chosen size.

The settings that matter

Names differ between tools, and many hosted services hide some of these. When you can see them, these are the ones worth learning.

Guidance works by comparing two predictions at each step, one with your prompt and one without, and pushing the result further toward the difference. That is why the scale feels like a strength dial for the prompt, and why setting it too high exaggerates everything.

A workflow from idea to final image

1. Write the brief in one sentence: what the image is for, its shape (square post, 16:9 banner, 9:16 phone wallpaper) and the one thing it must show.
2. Draft a prompt with subject, action, setting, style, lighting and framing.
3. Generate a batch of four to eight small images with random seeds. Look for composition, not detail.
4. Pick the best one and note its seed. Keep the seed fixed from now on.
5. Change one thing at a time in the prompt (lighting, lens, color) and compare. With a fixed seed, differences come from your words.
6. When the composition is right, fix local problems with inpainting or image-to-image rather than regenerating from scratch.
7. Upscale the final image to the size you need, then check faces, hands and any text at full size.
8. Record the prompt, seed, model and settings next to the file so you can make variations later.

Here is a small example of step 5. The seed stays the same; only the lighting phrase changes between runs.

seed 1234
A: a lighthouse on a rocky coast, wide shot, overcast soft light
B: a lighthouse on a rocky coast, wide shot, golden hour, warm rim light
C: a lighthouse on a rocky coast, wide shot, blue hour, lamp glowing

Because the starting noise is the same, A, B and C keep a similar layout, so you can judge the lighting alone. With a new seed each time you would be comparing different compositions.

Choosing size and resolution

Each model is trained around certain sizes. Earlier Stable Diffusion models were trained mostly at 512 by 512 pixels; newer ones are trained around 1024 by 1024 and several aspect ratios. Asking for a much larger image than the training size often produces repeated subjects, such as two heads or a doubled horizon, because the model fills the extra space with more of what it knows.

The safer pattern is to generate at the native size and then enlarge with an AI upscaler. For example, cv.cm's AI image generator makes a 384-pixel image on your own GPU with an autoregressive model, then can enlarge it 2x or 4x with Real-ESRGAN. The upscaler sharpens edges and textures, but it cannot add objects or fix a bad composition, so get the layout right first.

What text-to-image still does badly

• Text inside the image. Letters may be misspelled or invented, especially in long phrases and in non-Latin scripts. Add text in an editor afterwards when it must be correct.
• Hands and counting. Extra or merged fingers are less common than before but still appear. Exact counts (seven birds) and spatial relations (left of, behind) are often wrong.
• Several distinct people. Details from one person leak to another. Build the scene in stages or use reference images.
• The same character twice. A new run is a new person unless you use reference images, fixed descriptors or fine-tuning. See the guide on consistent characters.
• Bias toward common looks. Without details, models fall back on the most frequent faces, places and styles in their training data. Be explicit about age, region, body type and setting.
• Real people, brands and logos. Many services block them, and using them can create legal risk even where a tool allows it.

Hosted service or local model?

Hosted tools such as Midjourney, GPT image, Imagen or online Stable Diffusion and FLUX services are easy to start with and run large models, but you send your prompt to a server and follow its content rules and terms. Local tools run a model on your own GPU: nothing leaves your device, and you can see every setting, but the model must fit in memory and is usually smaller.