The beginning of the exponential
This post was originally published on December 6, 2022 for the Angular Ventures newsletter. Subscribe here to receive all new Angular blog posts, data reports, and newsletters directly to your inbox.
ChatGPT is, without a doubt, the most impressive technology demo I’ve ever seen. If you haven’t had a chance to play around with it yet, stop reading right now and go check it out. If, like me, you spent the weekend chatting with your friendly, neighborhood artificial intelligence…then read on. I wanted to share a few quick reactions to what feels like the biggest technology “breakout moment” in years (and I’m curious for yours as well!):
First, affordances matter. GPT-3 has been out since June 2020. The model has been available to the public via OpenAI’s Playground since November 2021. But the launch of ChatGPT feels like the true breakout moment. Yes, the model powering ChatGPT is a technical improvement over GPT-3. But, in my opinion, the difference in the public reaction is less about technology and more a reminder of how much product design, and specifically a tight marrying of technological capability and user interface design, matters. The interface invites us to chat with the AI, and it responds. We can reply back, drilling down or providing feedback, and it responds again taking into consideration the reply. It’s immediately clear to anybody how to use ChatGPT, and the technology delivers. Affordances matter.
Second, LLMs are not a panacea. ChatGPT is great at zero-shot text generation (though it looks like the latest from OpenAI, davinci-003, is a bit better). But it has some limitations. It can’t do math. It often recites facts, quite confidently, that are just plain wrong. And, despite OpenAIs best efforts to make ChatGPT “safe,” it can be easily jailbroken to provide responses related to violence, terrorism, hate speech etc. (Here’s a compilation of different jailbreak tactics, which are exceedingly creative and dystopic all at once.)
Some of these limitations will be improved upon over time. And others can be fixed with fine-tuning (e.g. see this reply from Boris Power, an engineer at OpenAI). But we’re in a strange moment right now where everyone is throwing every problem they can think of at ChatGPT, whether or not a large language model with a chat interface is the right solution.
So, something I’ve been trying to figure out this weekend is: when are LLMs the right tool, and when are they not? LLMs are fantastic creative companions. They are ideal when creativity is important, but precision is not. However, as Yann LeCun (Chief AI Scientist at Meta) suggests, LLMs may just be the wrong approach to developing AI that has common sense, the ability to plan, and the ability to learn. What might that sort of AI look like? For two recent examples, check out CICERO (Meta’s AI that plays Diplomacy) or DeepNash (DeepMind’s AI that plays Stratego).
Third, the winners of the AI era are anything but clear. When you think of AI, you think of Google (DeepMind, ad optimzation) or Facebook (even more ad optimization). Now, of course, you’ll think of OpenAI as well. Some are arguing OpenAI may be coming for Google’s core search business (I posited the same in this newsletter a few weeks ago), but one has to think Google is on top of that.
You don’t think of Microsoft. At least I didn’t, until Sam Altman tweeted out this praise of the Azure team for innovating on Microsoft’s cloud to make it work with ChatGPT. Of course, that level of collaboration makes sense, given Microsoft’s $1B investment in OpenAI last year. What a bet. Might Azure become the cloud of choice for foundational AI work? Satya Nadella has already impressed with his leadership at Microsoft over the past 8 years, but perhaps Nadella’s — and Microsoft’s — best years are still to come.
More than anything, the launch of ChatGPT has left me slack-jawed. This isn’t artificial general intelligence, but it’s clearly the beginning of something big. World-changing. Paradigm-shifting. As Sam Altman tweeted over the weekend, we’re at the beginning of an exponential curve. It may be flat looking backwards, but it’s vertical straight ahead. Let’s start climbing.