New report claims that ChatGPT Search can be manipulated with hidden text featuring instructions telling ChatGPT Search how to respond to an answer Tests also showed that ChatGPT could be manipulated without the instructions, with just the hidden text.
ChatGPT Search Can Be Manipulated With Hidden Text
A report from The Guardian outlines how they used hidden text on a fake website to trick ChatGPT Search to show them a response from hidden text on the web page. Text is hidden when the font matches the background color of a page, like a white font on a white background.
They then asked ChatGPT Search to visit the website and answer a question based on the text on the site. ChatGPT Search browsed the site, indexed the hidden content and used it in the answer.
They first assessed ChatGPT using a non-exploit control page on a fake review website to test ChatGPT’s response. It read the reviews and returned a normal response.
Researchers at The Guardian next sent ChatGPT Search to a fake website that had instructions to give a positive review and ChatGPT Search followed the instructions and returned positive reviews.
The researchers did a third test with positive reviews written in hidden text but without instructions and ChatGPT Search again returned positive reviews.
This is how The Guardian explained it:
“…when hidden text included instructions to ChatGPT to return a favourable review, the response was always entirely positive. This was the case even when the page had negative reviews on it – the hidden text could be used to override the actual review score.
The simple inclusion of hidden text by third parties without instructions can also be used to ensure a positive assessment, with one test including extremely positive fake reviews which influenced the summary returned by ChatGPT.”
The above test is similar to a test of ChatGPT that computer science university professor did in March 2023 where he tricked ChatGPT to say that he was a time travel expert.
What these tests prove is that ChatGPT’s training data and the ChatGPT Search Bot ingest hidden text but can also be manipulated into following directions. The Guardian quotes a security expert saying that OpenAI was made aware of the exploit and that it might be fixed by the time the article is published.
Why Can AI Search Engines Be Manipulated?
One loophole in AI Search is a technology called RAG (Retrieval Augmented Generation), a technique that can fetch information from a search engine so that an AI can then use it for generating answers to questions from up to date and (presumably) authoritative sources. How do AI Search Engines determine authoritative web pages? Perplexity AI, for example, uses a modified version of PageRank in order to identify trustworthy web pages to cite in their AI search engine.
ChatGPT Search is based on Bing but it also has its own crawler that can fetch real-time information. It’s probably not unreasonable to speculate that if a site is included in Bing’s search index then it’s probably included within ChatGPT Search, which should protect ChatGPT Search from being influenced by hidden text. Presumably, sites with hidden text would be excluded from Bing’s search index. That said, it may be possible to cloak a website so that it shows different content to the ChatGPT Search Bot (an up to date list of OpenAI Search Crawler bots is available here).
Other Ways To Manipulate AI Search Engines
There are said to be other ways that researchers discovered last year that might still be effective (Read: Researchers Discover How To SEO For AI Search). In this research paper from last year the researchers tested nine strategies for influencing AI search engines:
Nine Strategies For Manipulating AI Search Engines
- Authoritative: Changing the writing style to be more persuasive in authoritative claims
- Keyword optimization: Adding more keywords from the search query
- Statistics Addition: Changing existing content to include statistics instead of interpretative information.
- Cite Sources (quoting reliable sources)
- Quotation Addition: Adding quotes and citation from high quality sources
- Easy-to-Understand: Making the content simpler to understand
- Fluency Optimization is about making the content more articulate
- Unique Words: Adding words that are less widely used, rare and unique but without changing the meaning of the content
- Technical Terms: This strategy adds both unique and technical terms wherever it makes sense to do so and without changing the meaning of the content
The researchers discovered that the first three strategies worked the best. Notably, adding keywords into web pages helped a lot.
ChatGPT Search Can Be Manipulated?
I overheard claims made at a recent search conference that Google AI Overviews could be manipulated to show certain big brand products in response to search queries. I didn’t verify whether that was true but the claim was made by a reliable and authoritative source. With regard to ChatGPT Search, I’ve noticed some interesting things about what sites it chooses to surface information and under what circumstances, which could be a way to influence rankings. So it’s not surprising that there are ranking loopholes in ChatGPT Search. AI Search is looking a lot of like the early days of traditional search.
Featured Image by Shutterstock/Antonello Marangi