AI Companies Are Showing You What They Want You to See — Here's What the Data Actually Reveals

AI Companies Are Showing You What They Want You to See — Here's What the Data Actually Reveals

Nearly half of all real AI conversations would be scrubbed from the usage reports that AI companies publish and the press regularly cites. That single finding from a new independent research project called the AI Observatory should change how every business owner thinks about the AI tools they are adopting and the advice built around them.

A team of researchers from MIT, Stanford, and the Data Provenance Initiative recently published findings from the AI Observatory, a public platform that aggregated and analyzed 24,521 real AI conversations across 85,633 conversational turns, drawn from seven consent-based real-world datasets. Those conversations involved 5,000 people interacting with 52 different models, including ChatGPT, Claude, Gemini, and Grok, between 2023 and 2025. The goal was straightforward: provide independent, corroborated data on how people actually use generative AI, rather than relying solely on what the companies themselves choose to publish. According to Anka Reuel, a Computer Science PhD candidate at Stanford's Trustworthy AI Research Lab and co-lead of the project, "There is no independent source to corroborate" the data AI companies release on their own.

What the AI Observatory found cuts against the dominant narrative that AI is primarily a productivity and work tool. When researchers applied Anthropic's own filtering methodology to the AI Observatory's dataset, 48% of conversations would have been excluded entirely because they fell outside work- or productivity-related topics. Those filtered-out conversations were far more likely to involve health and relationships (44.2% versus 31.2% in Anthropic's analysis), adult or illicit content (7.9% versus 2.1%), harassment and hate speech (27.5% versus 5.66%), and sexual content (16.7% versus 2.4%). OpenAI's own 2025 ChatGPT usage report similarly found that only 30% of consumer use was work-related. The researchers also found meaningful behavioral differences across platforms. People used Grok and Gemini more often for information retrieval. Grok drew heavy news and politics usage but was also where misinformation concentrated most densely. Claude attracted more coding-focused conversations. Gemini saw more social and roleplay interactions. ChatGPT was especially popular for homework help. And even within the same product, behavior shifted: conversations with ChatGPT on GPT-3.5 were shorter, while GPT-4o sessions grew longer and more iterative, consistent with that version's association with deeper emotional engagement. Over time within WildChat, one of the Observatory's largest datasets, prompts and responses grew longer, small talk increased, and AI self-disclosure (bots flagging that they are bots) decreased, suggesting a rise in AI companionship use.

For small business owners, this research matters in ways that go beyond academic curiosity. The AI strategies being sold to you, the benchmarks being used to justify AI tool adoption, and the product positioning from the major AI companies are all built on a filtered, curated view of how their tools are actually being used. Shayne Longpre, a recent PhD graduate from the MIT Media Lab and co-lead of the AI Observatory, put it plainly: "No single company report tells the whole story." If your marketing team is building a strategy on the assumption that customers or prospects are using AI primarily for efficient, task-oriented work, the real data suggests a far richer and more varied picture, one that includes emotional support, companionship, entertainment, and personal exploration.

This also has direct implications for how you deploy AI in your own customer-facing operations. If AI chat tools are increasingly serving as companions and emotional touchpoints rather than just task executors, the tone, warmth, and relational quality of any AI-assisted communication your business uses, whether that is a chatbot, an AI email assistant, or an AI-generated content series, matters more than you might have assumed. Getting the "functionality" right is table stakes. The experiences your customers are actually craving from AI are richer than a productivity tool provides. Understanding that distinction gives you a real edge in designing marketing funnels and customer interactions that meet people where they genuinely are.

There is also a cautionary note here about overconfidence in AI-generated insights. As David Widder, an assistant professor at UT-Austin's School of Information who researches people-AI interaction, noted, when all available data comes from companies with a financial stake in the story it tells, decision-makers are "completely operating in the wild and making these really consequential decisions without knowing what's actually happening beyond those company narratives." That applies to you as a business owner deciding which AI tools to invest in, which platforms to trust for customer engagement data, and which usage claims to build your growth strategy around.

This week, take one concrete step: go back to any AI tool you are currently using or evaluating for your business and look critically at whose data you are relying on to justify it. If the only usage data available comes from the tool's own developer, treat it as a starting point, not the full picture. Seek out independent reviews, third-party case studies, or agency partners who can give you an unfiltered view of how that tool performs across a real and diverse range of use cases, not just the curated highlights.

Your competitors are making AI decisions based on filtered narratives. The businesses that win are the ones who demand more complete data before committing their time and budget.

Originally inspired by: We still don't know how people are really using AI (https://www.technologyreview.com/2026/08/18/1142226/how-people-use-ai/) See how Leads to Conversion can help you make smarter, data-driven AI marketing decisions. Get your free AI audit