AI’s Hardest Problem: Teaching Us to Use It Better

Not only is AI smarter than us in most ways, but it is so much smarter than us that we struggle to use it well.

Terence Tao, one of the world’s best mathematicians, calls o1-preview a “mediocre, but not completely incompetent, graduate student.” Others have shown that it can complete all the reasoning of an entire graduate thesis in less than two minutes. But if you’re like me, you don’t have many problems that need a graduate student to solve. Making a future o2 model as smart as a post-doc isn’t going to make me use it any more often. I’m not smart enough to need it in my life.

o1 is also very close to being able to solve any problem that any human can. The Center for AI Safety is paying $5,000 for questions that today’s AI can’t solve. I’m confident these last problems will be stamped out next generation. And still, I’m betting you don’t have a use for this AI in your life. It’s harder than it sounds! The next hard problem in Artificial Intelligence isn’t training a larger model, but training ourselves.

Training Ourselves

A big part of how I spent my summer was wallowing in AI. The technology, the commentators, the jokers on Twitter, the arXiv papers, and I even built a few OSS apps myself. I’m confident that we’re only going to reach the combined potential of humans and AI if we train ourselves to work better with the AI. Specifically, we have to start doing two things:

1. Start recognizing when AI could solve the problems in front of us
2. Start tackling harder problems

What AI can do for us now

First, you must subscribe to a frontier model immediately. The free models are not good enough. I recommend Copilot Pro, but ChatGPT or Claude will work too. $20/month will be the best value of your life – once you start using it well! Next, you have to use the model for about 10 hours. There’s no shortcut here; trying it for a few things for 2 hours won’t cut it. You need to get past those failures before you can move on.

AI can only do two things: give us words or interact with computers. But that’s quite a lot! Let me give you some ideas on how to spend your 10 hours.

1. Write something with the AI’s help. Please don’t just give it a prompt and use the output. Instead, start from that book idea you’ve always wanted to write, and ask the AI for options. “Give me 5 ideas for the character’s major challenge” or “Suggest 10 things that could happen next.” And secondly, ask for the AI’s feedback. “Is this too complicated for a 5-year-old?” or “What would a magazine editor have to say about this article?” AI is a bad writer on its own, but it will help you be a great writer.
2. Learn core computer concepts. Not programming itself, the AI will mostly do that. There’s a lot of surrounding knowledge that technologists have picked up by osmosis. You can read a book like Understanding the Digital World, but since you have a world-class tutor in your AI, have it teach concepts to you. Start with a prompt like “write a comprehensive table of contents for a book that teaches core computer concepts, in nested markdown.” Copilot, ChatGPT, and Claude all give a good answer. Then start chatting. Ask questions as you learn. Skip over things that get very technical and detailed, which you can get back to later if you need them. With this step, you’re not only learning how to effectively use AI output (more on this in the next section), but you’re also using AI to research far beyond how you’d use a Google search.
3. Frequently turn on advanced voice and simply chat (Copilot or ChatGPT only at this time). Ask about one of your interests or describe the problem you’re facing at work as you would to a friend. This will help you get used to using AI for everything.

With this foundation, it’s time to level up and let AI assist you to do anything.

Leveling up our game

Although AI can’t change your tire, there is unparalleled value in giving you words and writing software. Most change in the world happens through words. And most of the market cap of the United States’ businesses is through electronic services, not physical labor (even physical labor is often coordinated through software). So now that you’re proficient at using AI, how do you want to use it?

Any way you want, of course! To get you started, let me give you some ideas of what you’re capable of:

• Turn a video you take of your bookcase into a searchable spreadsheet of titles and authors
• Change media categories, like Atlantic news essay to podcast audio or film to novel
• Sort through your email into a daily digest
• Understand anything about an image or video
• Think of meals based an a picture of your refrigerator
• Help you make up new bedtime stories
• Suggest new flavor pairings based on your favorite recipe
• Encourage you through a difficult time
• Write a song about your day, in your favorite micro-genre

If you’re doing it on a computer or you’re doing it through words, AI can probably help. Once you master using AI frequently to improve or accelerate what you were going to do anyway, you can begin to take on entirely new tasks you couldn’t do before:

• Create a web application, for fun or profit
• Apply for grants, or jobs
• Make original art for your wall
• Understand scientific disciplines down to the latest published papers
• Develop a game or write a novel

With some effort, you’ll begin to constantly see AI enabling solutions for tasks that never occurred to you as solvable before. But AI isn’t stopping there…

What’s Coming

If o1-preview is a “mediocre graduate student,” what will o2 be capable of? I do not know, but I do know that we’ll have to level up again. If you master the approaches described in this post, you’ll be able to take it on. It’s amazing to think what problems we will be able to solve with future AI assistance if we put in the time to train ourselves now.