Keyword Bias in ChatGPT

ChatGPT exhibits clear keyword bias when it executes searches on behalf of the user using its web search tools. During the building phase of Kismet’s AI Visibility tool, we saw specific web searches the agent would make on behalf of the user that varied widely from our initial expectation resulting in valuable insights for marketers who want to capture traffic in the new world of AEO (Answer Engine Optimization).

When ChatGPT users execute a prompt, ChatGPT often executes a series of search queries regardless of the user’s subscription level. Using OpenAI’s responses API, we changed the parameters to match outputs most similar to a user on ChatGPT’s overwhelmingly popular free tier, resulting in the ability to have greater visibility into the choices the ChatGPT agent makes on behalf of the user to answer their prompts.

When looking up “What is the best hotel for a bachelorette party in Austin?”, ChatGPT executes a series of search queries, most often searching for “Rooftop Pool”, “2025”, and other relevant keywords from its training corpus. This wasn’t just a fluke – after multiple rounds of testing, the takeaway was clear: ChatGPT thinks rooftop pools are essential research for a bachelorette party.

Kismet’s AI Visibility tool (Now in Beta), allows users to run their own prompts to see how often they are cited from the most popular AI search engines, noting when their web entities are mentioned in their responses. Across all these responses, Kismet plans to help their clients build the appropriate web assets on their own page to be directly cited by ChatGPT.

Right now, independent sellers of their own hotel inventory face mounting challenges of making sure their websites are directly cited in responses to ChatGPT users. Kismet's platform drives direct conversions without losing web traffic to OTA middlemen by capitalizing on our AEO expertise of having the best information available to AI agents.