Claude 2 from Anthropic is out, and it's great
Claude's 100K context limit is a gamechanger for some tasks compared to ChatGPT
Two big intertwined announcements from Anthropic yesterday:
They released Claude 2, their most recent LLM
Every user in the USA and UK now has access to the ChatGPT-like web interface claude.ai to talk to Claude 2 (previously, you could sign up for a waitlist for claude.ai’s predecessor, the Anthropic console).
The TL;DR of this newsletter is:
If you’re in the US or UK, go sign up now for claude.ai and start using it for all the same kinds of tasks that you might want to use ChatGPT for, plus anything that requires a higher context limit than ChatGPT.
It’s very good; probably on par with ChatGPT overall, better at some kinds of tasks and worse at others.
Back up a minute, who is Anthropic and what is Claude?
You might be wondering: Who is Anthropic and why do I care about them? First, a refresher on ChatGPT-related terms:
OpenAI is the single most famous and influential generative AI-native company (as opposed to the Googles and Amazons of the world, which are gen AI heavyweights, but were not founded as gen AI companies).
GPT-3.5 and GPT-4 are separate Large Language Models (LLMs) that OpenAI developed and released.
ChatGPT is the B2C interface or application that OpenAI released late last year, which allows anyone to create an account and interface very simply with the GPT-3.5 and GPT-4 models.
So, similarly:
Anthropic is arguably the most important and viable competitor to OpenAI. It was founded two years ago by an exodus of senior OpenAI staff. Here’s a “fun” (if you think existential risk is fun) NYT article about them and their path to launching Claude. (Several of the Anthropic founders and staff are friends and/or former housemates of mine.)
Claude 1.0, 2.0, etc. are separate LLMs that Anthropic developed.
claude.ai is Anthropic’s new B2C application that anyone on the internet can use to interact with Claude 2.0.
What makes Claude so good that I should consider switching from ChatGPT?
No one outside of Anthropic has had access to Claude 2.0 for more than about 2 days as of this writing, so there is no definitive evaluation of Claude 2.0 vs GPT-4. However:
Similar performance. In general, anecdotally, ChatGPT-4 and the new Claude perform similarly on many the kinds of tasks I use ChatGPT for. E.g., in a previous post (where, incidentally, I explain context limits if you need a refresher!) I asked GPT-4 to help me come up with a pop culture analogy to help explain LLM knowledge cutoff dates. Here’s Claude’s answer to the same question:
Vastly higher context limit. So, similar performance on small tasks. BUT, Claude 2 has a VASTLY higher context limit than GPT-4. Claude can process 100,000 tokens (about 75,000) words, compared to ChatGPT 3.5’s 16,000 tokens (about 12,000 words) and ChatGPT 4.0’s 4,000 tokens (about 3,000 words). This is an enormous deal; on some tasks it makes Claude as magical compared to ChatGPT as ChatGPT is compared to spellcheck. You can upload three 70-page PDFs and ask for a combined summary of all three! You can copy in a 20-page memo and ask for feedback!
Claude also has a higher OUTPUT limit. ChatGPT can only output about 600 words at once (though there are workarounds); the new Claude can output a few thousand tokens/words at once.
Attaching files. The new Claude interface lets you upload up to 10 PDF and other files as input into any given query. In theory this is kind of minor, because you could just copy and paste the text of a PDF (or whatever) into the old Claude. But man is it convenient.
Based on #2 and #3, here’s an example of how I might use it for research for my novel. (Note that I would still have to fact-check for hallucinations for any critical pieces of information I need to make sure aren’t wrong.)
The next time I write a proposal to a client, for instance, I’m going to use Claude: I’ll attach all my previous proposals, give it a bit of information about the new client and the project, and ask it to write me the entire new proposal using the previous proposals as templates. A fundraising team could do the same with grant applications and grant reports.
Any downsides?
First, claude.ai does not have a paid tier yet — which means it’s already hitting usage limits the way the free version of ChatGPT does. Which means that I’m already getting this error message occasionally:
Additionally, I’m sure there are also types of tasks that ChatGPT is better at than Claude — certain types of writing styles or coding problems, areas of greater depth of knowledge, types of hallucinations it’s not as prone to, etc. For now we’ll all just be surfacing these by experimentation. I’d love to hear anything you discover on this front in the comments!