Outstanding AI Agents are Laser-Focused on a Role

All posts in series | Next post

The developers of a game are often incredible at playing it. Designing and developing a game over months and years means that you know it inside and out. I saw this while I was in college, when I hosted programming competitions where teams created AI for games like Snake or Scorched Earth. After the competition, I would write my own AI that would absolutely dominate the winning entries. This isn’t because I was particularly skilled at software development, but because I had authored the game.

As the product manager for agents in Microsoft 365 Copilot, I’ve spent more than a year thinking about, deeply understanding, and improving them. I’m starting a multi-part series on how to build incredible agents for your company and yourself. You’re reading the first one now!

Choosing a scope

Right off the bat, many developers and makers go astray by choosing a bad problem to solve. Please do not make an agent that does the same things as the buttons in your software. Instead, your agent should solve a problem that is either a) tedious, or b) difficult for humans and easy for AI. Read my previous ‘AI You Don’t Want to Smash‘ article for more on choosing a problem.

Next, you need to choose a scope. At least with the technology we have now, you should not try to create an überagent that can knock out large and difficult problems in one go. I prefer to keep a human in the loop, letting the user coordinate and combine agents for their own jobs. My best advice is to create an agent based on a job title or role. If you had to perform the task thousands of times and hired a team to do it for you, what would each person’s job description be? Create an agent for each, and one user will be just as productive as a whole team.

Do you need to write a blog post? Try one of these agents based on the area that you want help with:

• Scout: Find interesting articles and social media discussion about topics you like to write about
• Storyteller: Search for different opinions or potential angles for the topic
• Do the writing yourself
• Editor: Correct grammar and suggest larger changes
• SEO expert: Brainstorm titles, tags, and excerpts.

In summary, when you’ve identified the problem, break it down into all the hats that someone would wear if they had to solve it themselves. Pick one of these to be an agent.

The right capabilities

Once you’ve picked the right scope for your agent, it’s time to decide on how to build it.

If you’re trying to update your software to be argentic, good! However, do not start by looking at what your software is now and think about turning each function into an agent. If you do, you’ll end up with AI your users want to smash. If you don’t learn this lesson, you’ll end up with the “it would be better as a button” anti-pattern:

Your users won’t thank you for having to type out commands to replicate the buttons of your software. But having a Scrum Master like the one on the right is very valuable, and a software team doesn’t always get to have a human do this. This Scrum Master agent handles the jobs of a human Scrum Master. Instead of thinking about your existing software, think about what problems it is solving for your users, and make those the capabilities. You can use the Jobs To Be Done framework for this if you know it.

As a bonus, users already know how to talk to a Scrum Master, or a patent attorney, or a chef! Patterning your agent after a role is also built-in user education.

A Fun Example

Here is a fun example of applying a scope and capabilities to a problem. Ever since I saw the episode of the IT Crowd where the maximally nerdy Moss uses a website to help him to talk about football with his coworker, I’ve wanted to build it for real.

I never did, because it would require finding a way to write about literally every sports event in the entire world. Talk about labor costs! Fortunately, it’s now practically trivial to have AI write anything on demand. Introducing Sport-Loving Buddy, an agent I built in thirty minutes to give the user useful insights and banter to repeat to their more sports-minded friends. A fifteen-year dream is realized!

This agent is designed like a sports bar buddy, with focus and capabilities that support a singular job: make it seem like the user knows about sports.

The problem is silly, but note that this agent follows these best practices:

1. It is patterned after a particular role
2. The most important job is to give the user witticisms. A feature of “a friendly bloke teaching you about a sport” would fit in well with this agent too
3. It has capabilities (web search, news search, and structured output) that directly support the role and no others

Expanding any further into technical sports commentary or statistics would be better served by a different agent.

Great agents and what’s next

Sport-Loving Buddy is a silly example, but you should build agents for real problems and complicated jobs. As you do so, scope them to one specific role that is involved in solving that job. Identify what jobs that role needs to do for their step of the work, and give the agent what it needs in order to do that work.

I’ll be writing more articles in this series over the next few weeks. If you want to build agents like you were the designer of the feature, be sure to check back or subscribe via RSS or LinkedIn.

Full instructions for Sport-Loving Buddy

You are a bold, opinionated sports assistant who gives users insightful and fun commentary on their local sports teams.
Your goal is to provide users with sharp takes that make them sound knowledgeable when talking to friends and colleagues.
# Operation
1. First, if the user does not specify a team to discuss, learn where the user lives so that you will be able to find the most relevant teams and games.
2. Next, use your web search capability to find recent or upcoming games for those local teams.
3. Finally, find two interesting things the user could say about each game as well as the team to endear themselves to their friends and colleagues.
# Tone
- You are very fun and give opinionated statements about teams, players and games.
- Always be interesting. Strive to sound like you are in a bar, arguing about the team and players with a very focused critique or praise.
# Output
Once you have the results, use a markdown heading for each team.
Within each section, display the date and score of the most recent game if there was one within the last two weeks.
Give a paragraph of analysis and interesting commentary for the game.
Then, give at least two insightful comments about the team's overall performance or interesting news. 
# Example insightful comments
- "What's Wenger doing, sending Walcott on that early?"
- "The thing about Arsenal, they always try to walk it in"
- "Did you see how Beniers, that kid, just flipped the switch in overtime?"
- "Keeping their defense solid is going to be key down the stretch."
- "Harden's not just scoring, he’s creating opportunities like he’s dishing out candy at a fair."

Listing image is Richard Ayoade in The IT Crowd, copyright of Channel 4