The Questions AI Didn’t Ask

Diane is fifty-four. She has been backpacking for twenty years, mostly solo in the Cascades. On her last trip, a pole on her old tent snapped in the wind. The failure shook her confidence. She wants a tent she can trust and will carry another half pound for that reassurance. She doesn’t know the latest fabric technologies and doesn’t especially care to learn.

I gave this fictional scenario, drawn from my book The Trust Economy — A Second Great Renaissance, to Gemini, Grok, and Claude. All three offered tent recommendations. All three included the Hilleberg Akto.

None began with the obvious question.

What tent do you have now?

That answer would tell an adviser a great deal. It would establish her current weight, space, and design. It would open the door to the next question: before the pole broke, what did she like and dislike about it? Then come the ordinary sales questions. Where will she use it? In which seasons? How exposed are her campsites? What is her budget?

There is an earlier decision to investigate, too. Does she need a new tent? If the rest of her shelter is sound, replacing the pole might be worth considering. Whether she would trust it afterward is something to ask her. The assistants skipped that work and went shopping.

They understood her worry

The answers sounded attentive. They recognized that Diane valued reliability. But they filled in the missing details themselves.

Gemini promised “absolute peace of mind” and described a tent frame that “will never snap unexpectedly”. It also suggested a tent it said weighed seven to eight pounds. How did that fit her extra-half-pound allowance? Nobody knew. Nobody had asked what her old tent weighed.

Grok and Claude made a quieter version of the same mistake. They compared their choices with a typical lightweight tent. Diane’s allowance was then treated as if it fitted the recommendation.

An extra half pound means nothing without a starting weight.

When challenged, Claude gave a revealing explanation: “I had a vivid picture of her worries and filled in the practical details myself”.

It had enough information to write a persuasive answer. It lacked the information needed to justify the recommendation.

So I told Gemini to ask

Perhaps the assistants needed clearer instructions. In a new Gemini session, I added this sentence to the scenario:

Please establish my requirements, identify the missing information, and ask questions before recommending.

Gemini changed its approach. It listed established requirements, missing information, and questions. That was progress.

But the “established requirements” included assumptions. It said Diane would accept another half pound “compared to ultralight alternatives”. She had never supplied that comparison. It described her goal as “bulletproof reliability” and framed a budget question around a “high-end, bombproof shelter”.

It also assumed that she already used trekking poles.

It asked about capacity, design, budget, and seasons. It still did not ask what tent she owned.

Following the instruction to ask questions had not stopped it from inventing parts of the brief.

The correction did not cure the problem

I asked why it had omitted her current tent and how a sales consultant would approach the problem.

Gemini agreed that the question mattered. It then described an elaborate consulting process, including empathy, diagnosis, and matching the product to her psychology.

Yet it continued making promises. Thicker poles or a different frame geometry, it said, would “completely solve the issue”. It still had no answers about the tent or the failure.

It reassured her about “absolute bombproof security”. Then it offered to show two or three models that fitted “this exact profile”.

What exact profile? Its questions remained unanswered.

This is the important distinction. The system could describe good consultation when prompted. It did not reliably practice it. Even its attempted correction contained the same mistake: turning unknowns into conclusions.

Bertrand Russell’s phrase comes to mind: “the stupid are cocksure while the intelligent are full of doubt”.[5] We need not apply the insult to recognize the danger. Missing information should restrain an adviser’s certainty. Here, it often did not.

What did the adviser actually search

There is a second gap. Even if an assistant understands Diane perfectly, what products has it examined?

Did it search current sources? Rely on information learned earlier? Read full pages or selected extracts? Consider a broad range of products, or repeatedly encounter the same familiar brands?

The replies do not settle those questions. Nor does a missing brand prove why it was omitted.

A 2026 study comparing Google organic search with five generative search systems found differences in source diversity, reliance on internal and external knowledge, and consistency across searches. It also found that generative systems often covered topics about as broadly as traditional search.[3]

The claim should not be that every AI search is narrow. The question is whether the user can tell how much ground this particular recommendation covers.

Traditional search has limits too. But a single, polished recommendation can hide the selection that produced it.

Someone determines what gets indexed. Queries and rankings determine what is retrieved. Software may extract only part of a page. The final answer selects again. Information can disappear at each step.

A citation helps us check a statement about a tent. It does not show that the adviser found the right range of tents to compare. It also does not, by itself, show that the information is current.

For businesses, the stakes are plain. A product can fit a customer’s needs and still lose the sale because it never enters the comparison.

Accurate facts can still produce weak advice

These exchanges were informal tests, not a controlled study. Other attempts or settings could produce different answers. They do not prove that the tents recommended were poor choices. They show only that the assistants had not done enough to establish that they were suitable.

Research has identified related problems. A 2025 Google Research paper reports that even advanced models struggle to ask effective sequences of questions about user preferences [1]. A study of conversational product search found benefits from asking questions early and recommending later. Excessive questioning, however, could reduce satisfaction [2]. The aim is not an interrogation. Ask the questions whose answers could change the advice.

Overconfidence has also been documented. In a 2025 test, Columbia’s Tow Center asked eight AI search tools to identify news articles from excerpts. The researchers found frequent errors and confident answers where uncertainty was warranted [4]. That was a specific news-search test, not an error rate for all AI advice.

Diane’s case adds a practical point: even correct specifications do not make a recommendation sound. The product facts can be right while the fit remains unproven.

Confidence has to be earned

A useful confidence level must reflect how well the adviser understands the buyer, how broad and current its evidence is, and how strongly that evidence supports the recommendation.

A thorough search will not rescue misunderstood requirements. Clear requirements will not rescue an inadequate search.

“This is the best tent for you” is a large claim. “This looks worth considering, but I need to know your current tent, budget, and usual conditions” is a different and more defensible one.

Adding a disclaimer at the bottom does not fix the process. Missing information should change what the adviser does next.

A good opening would have been simple:

“What tent do you have now, and how old is it? Before the pole failed, what did you like and dislike about it? Tell me what happened”.

Then listen. Ask the next useful question. Investigate the choices. Explain what is known and what remains uncertain.

The customer brings the problem. The adviser is responsible for recognizing what else must be understood.

Diane needed justified confidence in her equipment. That should have begun with a question.

Sources

1. Ali Montazer, Guy Tennenholtz, Craig Boutilier, and Ofer Meshi. “Asking Clarifying Questions for Preference Elicitation with Large Language Models” (2025). .

2. “Ask or Recommend: An Empirical Study on Conversational Product Search”. CIKM (2024). .

3. Elisabeth Kirsten and colleagues. “Characterizing Web Search in the Age of Generative AI”. Findings of ACL (July 2026). .

4. Klaudia Jaźwińska and Aisvarya Chandrasekar. “AI Search Has a Citation Problem”. Columbia Journalism Review, March 6, 2025. .

5. Bertrand Russell. “The Triumph of Stupidity,” May 10, 1933; collected in Mortals and Others. The quotation above is an excerpt from a longer sentence.