Software Engineer Trains a Transformer, Only to Find Search Rules Work Better

A software engineer with no machine-learning experience trained a Transformer during an eight-day competition. The result is somewhat counterintuitive: a network can be better on training metrics yet perform worse when placed into a real search. Gaurav Sen documented his participation in the CodinGame Fall Challenge 2026 in a blog post. He writes that Claude agents wrote all of his code, while he set up the surrounding harness.

He set three simple goals for himself: the network had to understand the board; it had to finish within 50 milliseconds, with no single source file exceeding 100,000 characters; and a network that looked better in training also had to be better in actual play.

The network was placed at the end of the search. With 50 milliseconds per turn, the search made about 14,000 edits to a plan; the judge built a table of candidate plans against possible opponent replies, and each cell was rolled forward one turn and scored by the network—about 64 times in the midgame. The network answered only two things: how likely we were to win from this position, and by how much.

The way the board was turned into tokens was also very specific. Each house was a token, carrying its units, ownership, and position; each triangle was a token, carrying ownership and whether it could be captured; and the whole board was one token. Attention let each token look at the other tokens and also learned a relational bias for each pair of tokens. Two houses one step apart versus four steps apart were different, and it was also marked when a house was a corner of a triangle.

The training route followed AlphaGo's three steps: supervised learning, self-play, and distillation. Gaurav started with distillation. On November 4, a small network learned how a slow but strong search scored positions. On November 5, the bot played against itself on AWS machines with GPUs, and its rating climbed from 1500 to 1758. Its first online placement was 198/426 in the Gold league, with a score of 17.54; the bot it replaced had a score of 16.77 and ranked 172. Higher score, worse ranking. Good self-evaluation does not equal good actual play.

On November 7, the network switched to learning from 1,918 games played by top players. This bundle went 248-120-32 against the previous build and improved placement by 1.20 points. It also included a lead guard and a timeout fix, so the network cannot take all the credit.

The 50 milliseconds and 100,000 characters were hard constraints. The online student had about 99,000 parameters, was trained at 7 bits per weight, and its weights were entropy-coded and packed into about 18,010 characters. A variant with a second embedding packed to 102,999 characters and was cut outright. On the final day, a 42-minute run distilled the student into five smaller networks. The selected d24L2 had 9,515 parameters and took 10.74 microseconds per evaluation, versus 98.73 microseconds for the online student—about nine times faster. On held-out positions, it selected the teacher's top move 79.8% of the time, with a value correlation of 0.990.

Good training numbers still had to be tested back in the bot. In 96 games on a shared test machine, d24L2 with deeper search won 64.5% against the online bot; in 48 games with one game per core, it won 55.7%. It also failed the opening test, with an online placement of only 25.69.

RL was more direct. An overnight policy went 819-203 against its parent on its own, but when placed into the search it achieved only 46.0%. The various RL attempts all fell below the 55% gate: self-play retrain 45.7%, local RL loop 46.6%, 49.5% under rule v3, distilling 4x search into the judge 50.6%, and the AlphaGo-style pipeline 46.0%.

Gaurav's summary is that of 11 network and RL ideas, 5 passed testing against the online bot, and 1 improved placement. The single largest placement improvement came from structure: the bot first planned against the opponent's best reply to its own plan, and then replanned.

At equal compute, this rule won 67.6%. Ten times the search won only 54.8%, and ten times versus four times the search won 54.2%, also below the 55% gate. The network was part of the bot, but more work went into the decision rules around it.

In the comments, someone asked how a software development engineer with 1 year of experience can compete with people who have 10 to 20 years of experience, and whether to learn a niche or go deep into ML. Seekful.org's reply was practical: when moving from SWE to ML, there is less theory to fear than many people think, but the order matters—matrix multiplication, derivatives, chain rule; with these, backprop is no longer mysterious. Another question asked whether SLMs are more effective in small projects, privacy, and speed scenarios. In this case, the small network was forced by the 50 milliseconds and 100,000 characters: system constraints came first, model size second.

Training numbers can only be a reference; performance in the system is the real test.