The Basics › Chapter 7: When AI Gets It Wrong
Chapter 7

When AI Gets It Wrong

Why this matters

In the last chapter, you built your first skill file (a .md file in your Skills folder) and tested it. The output probably wasn't perfect. That's exactly the point of this chapter.

This chapter is worth the entire price of the course.

Every AI course teaches you how to set things up. Almost none of them teach you what to do when things go wrong. And things WILL go wrong. Your quote will have the wrong format. Your email will sound too formal. Your follow-up will include a discount when you told it not to.

Most people's reaction: "AI is useless" or "let me just do it manually."

Your reaction after this chapter: "That's a skill file fix. Two minutes."

The difference between people who give up on AI and people who build systems that run their business comes down to this one chapter.

What we're doing

Learning how to fix mistakes permanently by updating your skill files instead of just correcting individual outputs.

The mindset shift

When Claude gets something wrong, your instinct will be to say:

No, that's wrong. Remove the discount and make the email shorter.

Claude will fix it. This time. But next time you ask for the same thing, it'll make the same mistake again. Because the skill file hasn't changed.

The right move:

  1. Notice the mistake - what specifically went wrong?
  2. Ask why - what's missing from the skill file that caused this?
  3. Fix the skill file - add a rule or clarification
  4. Save and test - confirm it's fixed

That's it. The mistake is now impossible. Not unlikely. Impossible. Because the rule is written down and Claude reads it every time.

A real example

Let's go back to Dave the painter.

Dave asks Claude to write a follow-up email to a client who hasn't accepted a quote. Claude writes:

Hi Sarah, just following up on the quote I sent last week. I'd be happy to offer a 10% discount if you'd like to go ahead this month.

Two problems:

  1. It used "just following up" (Dave's skill says never use that phrase)
  2. It offered a discount (Dave's skill says never offer discounts unprompted)

The wrong approach: Tell Claude "remove the discount and don't say just following up." Claude fixes this one email. Next time, same mistakes.

The right approach: Open the follow-up skill file and check. Dave looks at his skill and realises:

  • The "never use 'just following up'" rule IS there. So why did Claude use it? Maybe the rule needs to be more prominent. He moves it to the top of the Rules section and bolds it.
  • The "never offer a discount" rule is there too, but it says "never offer a discount unprompted." Claude might be interpreting "following up" as a situation where a discount is appropriate. Dave changes it to: "Never mention discounts, promotions, price reductions, or any financial incentive. Ever. Under any circumstances. If the client asks for a discount, tell me and I'll handle it."

Dave saves the file. Tests again. Problem solved. Permanently.

The process, step by step

1. Notice the mistake

Don't just accept output that's "close enough." If it's not right, it's not right. Common things to watch for:

  • Wrong tone (too formal, too casual, too salesy)
  • Missing information (forgot to include GST, left out terms)
  • Wrong formatting (not your quote layout, wrong email structure)
  • Made-up information (invented a price, assumed a detail)
  • Broke a rule (offered a discount, used banned phrases)

2. Ask why

Before you touch the skill file, ask yourself: "Why did Claude do this?"

  • Is the rule missing entirely? → Add it
  • Is the rule vague? → Make it more specific
  • Is the rule buried? → Move it to the top
  • Did Claude misinterpret the rule? → Reword it more clearly
  • Is there a contradiction between two rules? → Resolve it

3. Fix the skill file

Open the skill file (the .md file) in your Skills folder. Make the change. Be specific. Be explicit. If you think "Claude should know that," it probably doesn't. Write it down.

Vague rule (bad): "Keep it professional"

Specific rule (good): "Write like you're texting a client you've known for years. Use their first name. Short sentences. No corporate language. Sign off with 'Cheers' and my first name."

4. Save and test

Save the file. Ask Claude to do the same task again. Check if the fix worked. If it didn't, the rule needs to be clearer.

💡 Quick diagnostic checklist

When the output is wrong, check these three things in order:

  1. Is the rule in your recipe file? If not, add it. Problem solved.
  2. Is the rule clear enough? "Keep it professional" is vague. "Write like you're texting a client you've known for years" is specific.
  3. Are two rules contradicting each other? "Be casual" and "use formal language for legal terms" might confuse things. Pick one or specify when each applies.

The compound effect

Here's why this matters so much:

Week 1: Your quoting skill has 5 rules. Output is okay but needs corrections.

Week 2: You've added 3 more rules based on mistakes. Output is better.

Week 3: Two more fixes. Output is consistently good.

Week 4: One more tweak. Output is now as good as what you'd write yourself.

After a month of small fixes, your skill file is dialled in. Claude barely makes mistakes anymore. And every new person who joins your team gets that same quality from day one, because they're using the same skill file.

This is the compound effect of fixing skills. Every correction is permanent. Every fix makes the system better. And unlike training a human employee, the AI never forgets what you taught it.

Teaching forward thinking

Once you get comfortable fixing mistakes, start thinking ahead. Instead of waiting for Claude to get something wrong, ask yourself:

What COULD go wrong with this skill?

Then add rules to prevent it before it happens.

Examples:

  • "If the client's address is missing, ask me for it before writing the quote. Don't guess."
  • "If the job is under $500, don't include the formal terms and conditions. Just the price and a simple confirmation."
  • "If a client mentions they're in a strata building, add a note about strata approval being the client's responsibility."

This is the difference between an AI that follows instructions and an AI that thinks ahead. You build that thinking into the skill file.

💡 How many rules is too many?

There's no limit, but keep it readable. If a skill file is so long you can't scan it in 30 seconds, it might need splitting into two skills. A quoting skill with 20 rules is fine. A single skill trying to handle quoting, invoicing, AND follow-ups is too much. Make three separate skills.

What if I'm not sure what went wrong?

Ask Claude. Seriously. Say: "You just wrote this quote. Here's what's wrong with it: [describe the problem]. Look at my quoting skill file and tell me what rule is missing or unclear that caused this." Claude will usually tell you exactly what to add. Then you add it. Then you test.

What you just learned

  • When AI gets something wrong, fix the skill file, not just the output
  • Every fix is permanent - the mistake can't happen again
  • The compound effect means your skills get better every week
  • Be specific in your rules - vague instructions get vague results
  • Think ahead: add rules to prevent problems before they happen
  • You can ask Claude to help you fix your own skill files

Try it yourself

  • Go back to the skill you built in Chapter 6
  • Run it 3 times with different scenarios
  • Note every mistake or thing you'd change
  • For each mistake, add a specific rule to the skill file
  • Test again. See the difference.
  • Think of 3 things that COULD go wrong and add rules to prevent them
Chapter 6: Your First Skill Connecting Your Email