How we test a game client with no screen attached

You cannot check a game by reading its code. You have to play it. We wanted to be able to do that from a script, at three in the morning, without a monitor.

A client that draws to nowhere

Start the client in headless mode and it opens no window. It draws into an offscreen buffer instead — the same renderer, the same shaders, the same interface — and writes the result to a PNG on request. A run can be told how long to live, or how many frames.

A script instead of a mouse

A headless run takes a script: a plain text file of timed actions.

• click here, drag there, press this key, type that line;
• select the nearest NPC, or the nearest monster;
• run a client command;
• take a screenshot.

The client also writes a machine-readable trace of everything that mattered — what was clicked, what was targeted, what the server answered — so a check can assert on facts instead of squinting at pixels.

Talking to the server with no client at all

A separate tool, the netprobe, speaks the game protocol directly: log in, list characters, enter the world, walk, chat, fight, buy, sell, use items, move things in the warehouse, accept and hand in quests — and print what the bag, the purse and the quest log look like after each step. It is how a server change gets checked in a second, before anyone opens a client.

Checkers for the parts

• One tool renders any model to an image and can parse every model in the game to prove none of them crash.
• One does the same for maps, parsing every map file.
• One renders the game's windows offscreen and runs scripted input against them.
• One tests the content-pack system, forgeries included: 308 checks.
• One tests the spawn editor, including a fake server it can hurt on purpose: 637 checks.
• The server has its own suite of over two thousand tests.

Why it matters to you

Many of the fixes in this run came out of a screenshot somebody had to look at: a strip of dark pixels beside the chat window, an icon in the wrong place, a window drawn under its neighbour. A human still reads the image. But a script can now walk to the place, open the window, and take it — every time, in the same state — so that a problem once found stays found.

A footnote about builds

One evening several full builds of the client ran at once and used 16 GB of memory and 24 GB of swap, and the operating system killed the desktop to save itself. Every compile now queues for a machine-wide slot and runs under a memory cap. A build that waits is working as intended.