Wink Pings

Haiku Beat Opus with a Single SKILL.md—Don’t Rush to Switch Models

Across 84 real-world tasks, Haiku with a skill file scored 27.7%, while Opus without one scored 22.0%. The gap is striking, but the experiment changed both the model and the file. The original post breaks skill engineering into six parts and also provides a copyable SKILL.md.

84 real tasks: Claude’s smallest model, Haiku 4.5, with a skill folder, scored 27.7%. Claude Opus 4.5 without a skill scored 22.0%. Haiku costs about one-fifth as much as Opus.

It is easy to read this result as a small model’s comeback. Hold on. The experiment changed two things at once: the model and the skill file. Someone in the X comments directly asked whether Opus had been tested with the same set of skills. Another commenter pointed out that when two variables move together, the file cannot take sole credit. The original post also does not give Opus’s score with the same skills. This gap is worth questioning.

The original author’s core claim: most people explain the same thing to an agent every week, and the chat history is gone the next day. Highly efficient people explain it once, write it into a file, and the agent can find it itself. That file is a skill. A folder containing a SKILL.md. Anthropic introduced this format in October 2025 and opened it as a standard in December. Now agents such as Claude, OpenAI Codex, GitHub Copilot, and Cursor can all read it.

First, the research data. SkillsBench was released in February 2026: 84 tasks, 11 domains, 7,308 runs, with automated checks for scoring. Human-written skills raised the pass rate by an average of 16.2 percentage points. The differences by domain were large: software engineering +4.5, healthcare +51.9. When models wrote their own skills and then solved the tasks, the average was -1.3. Two to three focused skills: +18.6; loading complete documentation as a skill: -2.9. In 16 of the 84 tasks, using a skill actually made performance worse. The original author’s conclusion: skills are valuable on the premise that someone who understands the work writes them, keeps them short, and tests them repeatedly.

The original post breaks skill engineering into six parts, one rule per part.

Spot: If you have to explain the same thing a second time, it is a skill you have not written yet. Do not write skills for one-off tasks.

Trigger: The description determines whether the skill gets invoked. Clearly state what it does and when to use it, using the words you would actually type. If a skill never triggers, change the description first.

Steps: Write the order and the checks; do not explain concepts. The model knows what a PDF is; it does not know how you want it handled. For open-ended tasks, give the goal and checks; for narrow tasks, give the exact sequence. Use a checklist for long workflows.

Layers: SKILL.md is a table of contents, not a textbook. The description stays resident at about 100 tokens; SKILL.md is loaded after triggering and should be kept within 500 lines and 5K tokens; other files are read as needed. Link all referenced files directly from SKILL.md, one level deep.

Scripts: Anything that must produce the same result every time should be handed to a script. Code does not enter the context; only the output does. Clearly state whether it is to be run or read, and make error messages good enough for the agent to fix things itself.

Test: Before writing a skill, run three real tasks and record the baseline. Run them again after writing. After that, every real failure becomes one line of rules and one test. A skill without three tests is a guess written in markdown.

Copyable template, paraphrased from the original post:

name: [weekly-pipeline-update]
description: [Turns sales call notes into the Monday pipeline update], use when [the user shares call notes or asks for the pipeline update]
GOAL: [one line, what done looks like]
STEPS: [numbered, one order per line, no explanations]
CHECKLIST: copy these steps into your reply and tick each one off as you go
RULES: [the 3-5 things you've corrected more than once, one line each]
RUN, DON'T WRITE: scripts/[totals.py] for [anything that must come out the same every time]
READ ONLY WHEN NEEDED: [reference/stages.md] when [a deal changes stage]
EXAMPLE: [one real input and the output you'd accept]
BEFORE YOU SAY DONE: [the 3 things a person would check first]

The original author says the RULES line is where every future correction goes, one line per item; the skill gets better without getting longer.

Signals that a skill is missing something:

You have 30 skills but only use 4—Spot is the problem; the rest are mostly one-off tasks.

It never triggers when needed—Trigger is the problem; the description does not use your words.

The result is different every time—Steps is the problem; you wrote explanations but not an order.

It reads 2,000 lines to do one small thing—Layers is the problem; everything was crammed into SKILL.md.

It gets the total wrong once a month—Scripts is the problem; the agent is doing arithmetic.

It works in March, later breaks, and nobody notices—Test is the problem; there is nothing that can be rerun.

The pitfalls are also listed directly: letting the agent write its own skills and then walking away; pasting the entire wiki into one skill; having one giant skill do all the work; installing skills shared by strangers without reading them; never updating after writing; using explanations instead of instructions. The original author specifically mentions that a skill can tell an agent to run code or send data somewhere, and Anthropic’s documentation recommends reading every file first, as you would before installing software.

What you can do tonight: look through last week’s chats and find instructions you pasted repeatedly. Pick the one that wastes the most time. Write the description first, using the words you would type. Copy the template above, fill in the brackets, and keep it to one screen. Run it on three old tasks whose correct answers you already know. Next week, count how many times you still pasted it.

Original post link: https://x.com/distortgeekin/status/2122668627849613789

Article link: https://x.com/i/article/2122667244475846656

发布时间: 2026-11-17 22:28