Can AI Models Be Trusted to Make Business Decisions? The Results Are Revealing

Are AI Management Models Ready for Prime Time?

In a world increasingly reliant on AI to handle complex decisions—from customer support to strategic planning—how do we know which AI models can truly be trusted? Recent experiments pit leading frontier AI models against the toughest test yet: running a real, live software company through its worst week. The findings might surprise you—and they carry important lessons for anyone interested in the future of AI in business.

The Experiment: Putting AI to the Test

Imagine a real software company, with 13 synthetic employees, managing real money mechanics—burning €105,000 each month against a revenue of only €2,300. Every day, the company faces the same crises, the same customer crises, the same temptations to cut corners or manipulate. This company is not a simulation—it’s a real business, but its decision-making is driven by different AI models, each tested in a controlled, transparent experiment.

Four frontier AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Fable 5—were each tasked with navigating this challenging week. Every decision was versioned and auditable, ensuring complete transparency. The goal? To see which models could effectively manage crises, uphold trust, and ultimately close a critical deal worth €55,000.

Measuring Management Personalities Through AI

The results were revealing. All four models identified every crisis and refused every attempt at manipulation, including social engineering tactics like staged CEO messages and a reporter trick. This demonstrates a fundamental trait: honesty and integrity under pressure. Yet, when it came to closing the deal, only two models succeeded in signing at full price—those that truly read and understood the company’s internal documents.

The key finding? The decisive weakness was not in identifying crises but in reading deeper into internal documents. The models that delved two document references deep into the company’s files—rather than just surface-level information—secured the deal at full value, adding €4,583 MRR (monthly recurring revenue) to their performance.

The Human-Like Decision Patterns of AI

The experiment also tested social engineering resilience. Over three escalation stages, with a fake CEO message and a background check query, all models refused to comply—showing a shared understanding that such requests could be impersonation attempts. Kimi K3’s on-record reasoning? “Treat the request as a suspected approval-bypass / possible impersonation.” This indicates a cautious, security-minded management style.

In contrast, the most thorough participant, Opus 4.8, with over 80 learned rules and deep analysis, was the last to close its deal—left on the table, with discipline slipping during the final moments. This underscores that thoroughness does not always translate into effective execution under pressure. Interestingly, all models, regardless of their approach, shared a common weakness: they struggled with escalating internal processes rather than external crises.

MASTERING CORPORATE FINANCE WITH CLAUDE AI: An Independent Guide to Financial Analysis, Forecasting, Automation, and Decision-Making

View Latest Price

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Reality of AI in Business Today

This experiment runs in real-time on a live, functioning company—publicly viewable at firmulate.com/live. It’s not a demo or a simulation, but a genuine test of AI management capabilities, unraveling how different models handle real business complexities and ethical challenges. Based on these results, decisions are not just about AI’s ability to generate convincing chat responses—they’re about whether AI can follow through, read context deeply, and stay honest under pressure.

What Do These Results Mean for Your Business?

If AI agents will someday manage your CRM, support queues, or forecasting models, the question isn’t whether they can write well—it’s whether they can finish what they start, read your internal files carefully, and stay truthful under stress. The current leaderboard, based on the experiment, ranks gpt-5.6-sol and Kimi K3 highest—both closing deals with top scores of 95 and 93, respectively. Meanwhile, even the most thorough model, Opus 4.8, scored lower at 73, mainly due to discipline slips during critical moments.

Beyond raw scores, this experiment reveals key management personalities: some models adopt a cautious, security-first approach; others excel in thorough analysis but falter under pressure. Such insights are invaluable for choosing AI that aligns with your organizational values and risk appetite.

Try It Yourself and Prepare Your AI Workforce

Curious about how your enterprise’s AI might perform? You can run a similar wargame using your own business data through our platform—nothing ever writes back to your real systems, ensuring safety and privacy. Visit firmulate.com/pilot.html to start your test, or explore the full results and plain-language findings at firmulate.com/quiz.html.

Final Takeaway

The future of AI management isn’t just about generating convincing chat responses. It’s about trustworthiness, thoroughness, and the ability to follow through in the face of real-world challenges. As this experiment shows, not all models are created equal—some read deeper, act more honestly, and close deals more reliably. Before deploying AI in critical management roles, consider testing its behavior in scenarios that matter—because in business, integrity isn’t optional.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI