AI Can Fake the Skills That Look Like Expertise — Here Is What That Means for Your Business

AI Can Fake the Skills That Look Like Expertise — Here Is What That Means for Your Business

A tool that boosts academic grades by nearly a full point without teaching anyone anything sounds like a marketing problem disguised as an education story. It is both — and if you run a small or mid-size business that uses AI to produce content, proposals, or customer-facing work, this research is directly relevant to you.

A randomized experiment published in August 2026 and involving 1,053 freshmen at Bocconi University put GPT-4o to a controlled test. Students were split into four groups: a control group, a group that received a short lesson on causal reasoning, a group given access to GPT-4o, and a group that received both. Their task was to write marketing recommendations for the university's merchandise shop in up to 180 words. Students who used GPT-4o scored nearly a full point higher on a 1-to-5 grading scale. Their answers included roughly two more ideas on average, were rated as more logically coherent, and more closely aligned with the recommendations of three subject-matter experts. Even after researchers controlled for argumentation quality, idea count, idea diversity, and text properties, a measurable GPT-4o advantage remained. The researchers attributed the improvement to higher content quality in the output, not to any increase in what the students themselves actually knew.

Here is where the story gets more complicated for anyone using AI in a professional context. A separate analysis of more than 500,000 US college grades found that top grades increased most sharply in writing-heavy and programming-heavy courses with large homework components after ChatGPT launched. A 30-month study of more than 26,000 students in China found that AI-assisted homework scores improved, but exam scores dropped 18 to 24 percent over the long term among students who leaned on AI to do the thinking. Controlled experiments also found that just ten minutes of using AI as an answer machine measurably eroded problem-solving skills. When the AI was removed, those participants performed worse than people who had never used it. The pattern across all of this research points in the same direction: AI-assisted performance without independent reasoning tends to degrade the underlying skill over time.

For small and mid-size business owners, this research carries a direct warning about how AI is being used inside their operations. If your team is using tools like ChatGPT to produce polished-looking deliverables — marketing copy, client proposals, social content, strategy documents — without any critical reasoning layer built into the workflow, the outputs may score well on the surface while the underlying thinking quietly weakens. Graders in the Bocconi study rewarded structure, coherence, and completeness. They penalized originality and falsifiability. The same dynamic can play out in a business context when AI-generated content that sounds authoritative gets approved because it looks right, not because someone stress-tested the logic or verified the claims.

The causal reasoning lesson in the Bocconi experiment is the counterweight worth paying attention to. Students who received that training did not earn higher traditional scores. In fact, their average scores dipped slightly. But they were significantly more likely to explain why a proposed action should work, under what conditions it might fail, and they generated more diverse ideas that diverged meaningfully from their peers. When combined with GPT-4o access, the two approaches complemented each other — the reasoning training held up even alongside the AI boost. That is a practical model for any business owner building AI into their team's workflow: the tool handles polish and volume, and the people handle the reasoning, the verification, and the strategic pressure-testing.

The actionable move this week is to introduce a mandatory reasoning checkpoint into any AI-assisted content or strategy workflow your team already uses. Before any AI-generated draft gets approved or published, assign one person to answer three specific questions in writing: Why will this approach work? Under what conditions could it fail? What is one assumption in this piece that we have not verified? This mirrors exactly what the causal reasoning training in the Bocconi study produced — and it is what prevented the skill erosion documented in every follow-up study when AI handled all the thinking.

AI can make your work look expert-level. The businesses that win long-term will be the ones that stay expert-level underneath the output.

Originally inspired by: The skills that earn top grades are the ones AI can fake best (https://the-decoder.com/the-skills-that-earn-top-grades-are-the-ones-ai-can-fake-best/) See how Leads to Conversion can help your team use AI without losing the strategic edge that drives real growth. L2C builds human in the loop, agentic learning systems, so you get AI assistance with human, game changing inginuity

Your turn

What is your traffic actually doing?

Send us your details and we will come back with a short, specific read on what your traffic, your pages and your pipeline are doing today — and the first three things we would change. A real person reads every submission, and you get the read whether or not we ever work together.