The Prompt Engineering Tax on Financial Advice

AI-generated financial advice is competent at the basics but fails when things get messy. A recent study from the MIT Sloan School of Management analyzed how large language models (LLMs) provide financial guidance, finding that while AI offers generally good advice, its utility is highly dependent on the structure of the input. For practitioners, this highlights a significant friction point: AI can handle the standard 'savings and diversification' playbook, but it struggles with the non-linear events that define actual financial lives.

Good at Basics, Fragile in Shocks

The models consistently hit the standard notes: saving during working years, drawing down in retirement, and investing in diversified stock funds. However, the "good" advice stops at the surface level. When researchers introduced complexity—such as unemployment shocks or the need for active portfolio rebalancing—the models faltered. This isn't a trivial edge case; it represents a core limitation in how these models reason about dynamic, high-stakes changes. For a user, this means AI can help you build a "weatherproof" plan in theory, but it might leave you stranded when the weather actually turns. In a production environment, relying on an LLM for reactive financial strategy without human oversight is a significant risk. We can't treat LLMs as autonomous financial planners yet; they are currently high-quality calculators for static scenarios.

The Hidden Cost of Input Quality

One of the most revealing findings is that advice quality improved significantly when researchers used structured "academic" prompts compared to regular user prompts. In practice, this means there is a massive disparity between a finance professor's query and a regular user's "how do I save money?" prompt. The data shows that people with prior AI experience had roughly $100,000 more wealth at age 60 than those without it. Furthermore, less financially literate users saw about $50,000 less wealth at age 60. This suggests that AI doesn't naturally level the playing field; it may actually widen the wealth gap because the "right" way to interact with the model requires a level of literacy that many of the people who need the help most may not possess. If the tool rewards those who already know how to speak its language, it's not a democratizer—it's a gatekeeper.

The Interface is Where the Real Product Lives

The real story here isn't about whether LLMs can give financial advice, but about who has the power to shape the interface that delivers it. If a user has to be a prompt engineer to get high-quality results, the tool isn't truly "accessible" to the general public. For this to work at scale, the complexity of the "academic" prompts must be abstracted away into standardized, high-quality UI components. Until then, we are looking at a tool that favors the already-informed, potentially leaving vulnerable users with "good enough" advice that fails them during the specific life events that matter most. This is a classic case of a technology that works in a lab but requires a significant "translation layer" to be useful—and safe—in the real world. The value isn't in the model itself; it's in the engineering required to make that model's capabilities usable for the non-expert.