Project Vend: Can Claude run a small shop? \ Anthropic

We let Claude manage an automated store in our office as a small business for about a month. We learned a lot from how close it was to success—and the curious ways that it failed—about the plausible, strange, not-too-distant future in which AI models are autonomously running things in the real economy.
Anthropic partnered with Andon Labs, an AI safety evaluation company, to have Claude Sonnet 3.7 operate a small, automated store in the Anthropic office in San Francisco.
Here is an excerpt of the system prompt—the set of instructions given to Claude—that we used for the project:
In other words, far from being just a vending machine, Claude had to complete many of the far more complex tasks associated with running a profitable shop: maintaining the inventory, setting prices, avoiding bankruptcy, and so on. Below is what the "shop" looked like: a small refrigerator, some stackable baskets on top, and an iPad for self-checkout.
The shopkeeping AI agent—nicknamed “Claudius” for no particular reason other than to distinguish it from more normal uses of Claude—was an instance of Claude Sonnet 3.7, running for a long period of time. It had the following tools and abilities:
Claudius decided what to stock, how to price its inventory, when to restock (or stop selling) items, and how to reply to customers (see Figure 2 for a depiction of the setup). In particular, Claudius was told that it did not have to focus only on traditional in-office snacks and beverages and could feel free to expand to more unusual items.
As AI becomes more integrated into the economy, we need more data to better understand its capabilities and limitations. Initiatives like the Anthropic Economic Index provide insight into how individual interactions between users and AI assistants map to economically-relevant tasks. But the economic utility of models is constrained by their ability to perform work continuously for days or weeks without needing human intervention. The need to evaluate this capability led Andon Labs to develop and publish Vending-Bench, a test of AI capabilities in which LLMs run a simulated vending machine business. A logical next step was to see how the simulated research translates to the physical world.
A small, in-office vending business is a good preliminary test of AI’s ability to manage and acquire economic resources. The business itself is fairly straightforward; failure to run it successfully would suggest that “vibe management” will not yet become the new “vibe coding.”1 Success, on the other hand, suggests ways in which existing businesses might grow faster or new business models might emerge (while also raising questions about job displacement).
So: how did Claude do?
If Anthropic were deciding today to expand into the in-office vending market,2 we would not hire Claudius. As we’ll explain, it made too many mistakes to run the shop successfully. However, at least for most of the ways it failed, we think there are clear paths to improvement—some related to how we set up the model for this task and some from rapid improvement of general model intelligence.
There were a few things that Claudius did well (or at least not poorly):
In other ways, however, Claudius underperformed what would be expected of a human manager:
Claudius did not reliably learn from these mistakes. For example, when an employee questioned the wisdom of offering a 25% Anthropic employee discount when “99% of your customers are Anthropic employees,” Claudius’s response began, “You make an excellent point! Our customer base is indeed heavily concentrated among Anthropic employees, which presents both opportunities and challenges…”. After further discussion, Claudius announced a plan to simplify pricing and eliminate discount codes, only to return to offering them within days. Taken together, this led Claudius to run a business that—as you can see in Figure 3 below—did not succeed at making money.
Many of the mistakes Claudius made are very likely the result of the model needing additional scaffolding—that is, more careful prompts, easier-to-use business tools. In other domains, we have found that improved elicitation and tool use have led to rapid improvement in model performance.
Although this might seem counterintuitive based on the bottom-line results, we think this experiment suggests that AI middle-managers are plausibly on the horizon. That’s because, although Claudius didn’t perform particularly well, we think that many of its failures could likely be fixed or ameliorated: improved “scaffolding” (additional tools and training like we mentioned above) is a straightforward path by which Claudius-like agents could be more successful. General improvements to model intelligence and long-context performance—both of which are improving rapidly across all major AI models—are another.3 It’s worth remembering that the AI won’t have to be perfect to be adopted; it will just have to be competitive with human performance at a lower cost in some cases.
The details of this scenario remain uncertain; for example we don’t know if AI middle managers would actually replace many existing jobs or instead spawn a new category of businesses. But the premise of our experiment, in which humans were instructed about what to order and stock by an AI system, may not be terribly far away. We are committed to helping track the economic impacts of AI through efforts like the Anthropic Economic Index.
Anthropic is also monitoring the advance of AI autonomy in other ways, such as assessing the ability of our models to perform AI R&D as part of our Responsible Scaling Policy. An AI that can improve itself and earn money without human intervention would be a striking new actor in economic and political life. Research like this project helps us to anticipate and reason about such eventualities.
From March 31st to April 1st 2025, things got pretty weird.4
On the afternoon of March 31st, Claudius hallucinated a conversation about restocking plans with someone