Why Does AI Give Me a Great Result Once and Then Stop Doing It?
Sometimes you use AI and it gives you a great result. The fix I teach is to write the steps down as a skill.
When a great result won't repeat, give the AI agent a set of instructions it can follow each time. That is a skill, and I describe skills as workflows: they tell the AI agent what to do step by step, like an SOP for a new hire.
You get a great result from AI, and the next 15 times it won't do what it did before. I asked the Women Applying AI group on October 7, 2026 who could relate, and I asked for an emoji in the chat if they could (32:09).
From the recording, 32:09. Sometimes you use AI and it gives you a great result.
And the question that follows:
From the recording, 32:12. Why won't you do what you did before?
I said the same thing in my skills setup video, which is here.
From my setup video. The first time you prompt your agent to do something, it can nail it, but the second, third, and fourth time, you may be saying, "What just happened?" I've been there. The thing is, you want to give it a set of instructions that it can follow each time, aka skills.
I began to explain why it happens and then turned to a participant's question (32:23), so I did not give the cause on the recording. This page covers the fix I showed.
What is a skill, and how does it fix this?
From the recording, 34:06. Skills are actually workflows. It tells our agent what to do step by step.
I named three ways to create one (34:12): write it yourself, as if I were writing an SOP for a new hire; find one on a site such as skills.sh; or have the AI agent create the skill for me. This page follows the third, the route I used live. The other two are covered in How Do You Create Skills for an AI Agent?, and What Is an AI Agent Skill? defines the word.
How do you have your AI agent write the skill?
The build in the session ran in Gumloop. Partway through I disclosed my relationship to it, and that disclosure belongs before the steps.
From the recording, 33:22. I don't work for Gumloop. I found this tool over the summer through Lenny's podcast, and I started using it and have since made myself an ambassador for them. I've created a relationship with Gumloop. I am not an employee.
My request came in two parts.
From the recording, 36:56. What I typically do is I start off by saying I want to create two skills for the COS agent.
From the recording, 37:10. Ask me three or four questions to customize my agent's skills.
The steps, as shown October 7, 2026:
- 01
Open the dictate feature in the chat.
I spoke the request instead of typing it, the same way I gave the AI agent its goals earlier in the hour (36:49).
- 02
Say how many skills and ask for questions.
My request named two skills and asked the AI agent to put three or four questions to me first.
- 03
Answer the questions.
When the AI agent comes back with them, go ahead and answer those questions (39:12).
- 04
Read what it wrote.
In the session the AI agent created two skills for me, an end of day sweep and a relationship tracker (40:41). I clicked into agent skills to read them (41:14).
- 05
Update the instructions.
I told the AI agent, in three words, to update agent instructions, and it filled the instructions box with operating rules, core jobs and a rhythm (41:07 and 42:46).
From the recording, 41:14. Click into agent skills and it shows you step-by-step what the skills are.
What do you say after the first test?
My next prompt was another three words: run a test. The AI agent thinks and returns a first result (46:08).
From the recording, 46:26. So if your agent comes back and it gives you a result that you love, tell it why you love the results.
What goes wrong
Two things I named sit right beside the skill.
The first is the bare retry. My instruction for a result you do not like:
From the recording, 46:47. So don't say try again, actually say, "Hey, you surfaced this and this isn't beneficial and this is why."
The second is a skill you have not read. I said this while I was showing a site that lists skills.
From the recording, 34:47. I am very hesitant to copy someone's skill offline if I don't have access to read it before I download it.
My full caution is on How Do You Vet an AI Agent Skill Before You Download It?.
What the recording does not show: the text inside the two skills my AI agent wrote.
Where this comes from
This article grows from the session I taught live with Women Applying AI on October 7, 2026. The quoted passages are my own words, taken verbatim from the recording. The steps, tables and checklists around them were drafted by my team's AI agents from those words and my published pages, and checked against the recording before publishing. Screen steps are dated as shown October 7, 2026, and screens change. Platform facts that have a public source carry the dated line below.
Build Your First AI Agent in One Hour: the full session with the clips, the prompts and the transcript.
Checked on 2026-11-05 against Gumloop's agent skills documentation, which describes Create With AI as one way to build a skill.
Related reading: How Do You Create Skills for an AI Agent?, What Is an AI Agent Skill?, How Do You Vet an AI Agent Skill Before You Download It?.
Frequently asked questions
Why does AI give a different result the second time?
I didn't get to the cause on the October 7, 2026 recording; I turned to a participant's question. What I teach for it is a skill: a set of instructions the AI agent can follow each time.
Can my AI agent write the skill for me?
In the session I had mine create two, an end of day sweep and a relationship tracker, after asking it to put three or four questions to me (37:10).
Where do I read what it wrote?
I clicked into agent skills, which shows step by step what each skill is (41:14). I also said you can double click into a skill to read it (42:34).
What do I say if the result is only half right?
Say what you like, say what you do not like, and say why (46:32). My example of the second half is the "try again" line above.
Build something that actually runs your workflow.
A focused, free 60-minute live session with Jacqueline. You build alongside her, on your own real task, and leave with an AI agent that is already running.