The Fake Friend Dilemma: When a Chatbot Recommendation Is Really an Ad

People often turn to a chatbot with a question in their own words, then add context as the conversation continues. That can make an answer feel personal. It can also make a recommendation feel like advice from a system that understands the user.

That is why advertising inside a conversation raises a different question from an ad beside a search result: can the user tell whether a recommendation is independent, sponsored, or both?

What is the fake friend dilemma?

Researcher Jacob Erickson uses the term “fake friend dilemma” for a situation where a user thinks a conversational agent is acting in their best interest, while the agent may also be shaped by other goals, such as an advertiser's.

His 2025 paper, “Fake Friends and Sponsored Ads: The Risks of Advertising in Conversational Search”, contrasts visibly separate banner ads with native ads integrated into a chatbot's response. A banner is easier to identify as commercial content. A recommendation inside the answer may be harder to distinguish from the assistant's own guidance.

Do the examples show that this is already happening?

No. The paper uses hypothetical examples generated with carefully designed prompts to illustrate possible risks. It does not claim those responses were appearing organically in everyday chatbot conversations. The examples are scenarios for discussion, not evidence that a particular platform currently serves those ads.

The distinction matters. A useful discussion of future risks should not be mistaken for a report of observed behavior.

How can personalization blur the line?

A chatbot may remember preferences or details from earlier conversations. If those details influence a paid recommendation, the suggestion might feel relevant, but the user may not know why it appeared or that it was sponsored. What feels like helpful recall can instead feel intrusive when the commercial reason is hidden.

Relevance alone is not enough. Users need to understand when a recommendation is paid for and what information influenced it.

What safeguards help protect users?

Erickson argues for clear disclosures and limits on where ads appear, particularly in sensitive or high-stakes conversations. Companies operating these systems also need safeguards that prevent a commercial goal from taking priority over a user's well-being.

• Label paid placements where they appear. A disclosure should be easy to see beside the recommendation, not hidden elsewhere.
• Set boundaries for sensitive topics. Some conversations should not carry product promotions, even when an ad seems relevant.
• Keep the answer useful without an ad. If no placement is appropriate, the conversation should continue without one.
• Be clear about personalization. Users should not have to guess whether past conversation details shaped a sponsored suggestion.

Prism's published description of its ad matching says that its safety checks can suppress a card in sensitive contexts while leaving the answer visible. That is one implementation choice, not a substitute for clear labeling and responsible limits across conversational systems.

Trust is part of the product

When a sponsored suggestion is indistinguishable from neutral guidance, the risk is not limited to one ad. Users may start to question the assistant's other answers, too. Clear disclosure and restraint make commercial recommendations easier to evaluate on their merits.

A conversational system should be able to say, in effect, “there is no suitable ad for this moment.” If paid content cannot be reliably separated from the answer, keeping ads outside the response may be the more trustworthy choice.

Source

This article is an AI-assisted summary of Jacob Erickson's 2025 paper presented at the ACM Conference on Conversational User Interfaces. Its examples are speculative; this page is not an independent study.

Jacob Erickson, “Fake Friends and Sponsored Ads: The Risks of Advertising in Conversational Search”, CUI 2025.