ChatGPT now reads Reddit in only 1% of software answers
Crockett Ford
sunbeam Research
4 min read
Since August, the common observation has been that ChatGPT still reads Reddit, it just stopped citing it. We found that—at least in software-related answers—this is no longer true.
We monitored 26,056 ChatGPT answers to SaaS and software-related prompts, resulting in 521,125 retrieved pages and 107,763 citations between October 13th and November 11th.
Between October 22nd and 24th, Reddit retrieval by ChatGPT plummeted from 9.1% of answers to 1.1% and held steady. Prompts that relied on Reddit the most saw retrieval drop from 60% to only 8%.
Listicles from a handful of sites took Reddit’s place. And what ChatGPT reads matters even when it isn’t cited: it is 1.9x as likely to name a brand when it reads the brand’s site without citing it.
ChatGPT stopped reading Reddit and started reading more listicles
Between October 13th and 21st, Reddit was retrieved in about 9.1% of answers. On the 22nd it fell to 5.9%, and by the 24th it had reached about 1%. It hasn’t been above 1.7% on any day since.
The prompts where ChatGPT read Reddit most often before the drop saw the biggest change. In those answers, Reddit went from being retrieved 60% of the time on average to only 8%.
Something had to take Reddit’s place. For example, on the prompt “Is Ramp legit?”, ChatGPT went from reading Reddit in 88% of answers to 0%. Bankrate’s retrieval rate increased from 38% to 68%.
In Reddit-heavy prompts, Zapier’s retrieval rate went from 2% to 8% of answers, and Rankite’s from 0% to 8%. Across all prompts, ChatGPT now retrieves Rankite in about 1 in 7 answers, more often than it retrieved Reddit before the drop, but cites it in only 3.5%.
Some tech publications dropped, with TechRadar falling from 23% to 11%, and The Verge from 5% to 1%. Review platforms were mixed. Capterra rose from 24% to 27%, while G2 fell from 31% to 26%.
But listicles didn’t simply replace Reddit. They increased across the board. Five sites that publish listicles went from 18% to 35% retrieval across all answers, not just the prompts where Reddit was previously read.
What ChatGPT reads influences its recommendations
ChatGPT retrieves roughly 5 pages for every 1 it cites. Those retrieved pages influence ChatGPT’s output, even if the page never makes it to the final answer as a citation.
ChatGPT is 1.9x as likely to name a brand when it retrieves a page from the brand’s site, even if it doesn’t cite it. With a direct citation, a mention is 2.1x as likely, only slightly more.
Third-party listicles have a similar effect, albeit smaller. On average, each uncited listicle that mentions a brand makes ChatGPT about 1 percentage point more likely to name it, and each cited one about 4 points. But it can add up.
Customer.io went from being mentioned in 0% of answers to 94% of answers for the same prompt after ChatGPT started reading more than twice as many listicles that mention it. Most of those listicles were never cited.
Tracking citations can’t give you the whole picture about why ChatGPT names some brands and not others.
What marketers should know
This is yet another point against leaning on Reddit as a strategy to increase AI visibility, at least in ChatGPT.
AI retrieval behavior changes often and fast, and our findings here are strongest in SaaS-related AI answers. We do not suggest re-orienting your entire strategy based on a change observed over a few weeks. But a few things are worth acting on:
- Google’s AI Mode is still reading Reddit. Its Reddit retrieval rate stayed steady around 28% throughout the entire sample, and ChatGPT could resume reading it at any time.
- The best strategy is a diverse one across owned media, earned media, and UGC. Don’t over-index on any specific type, or you risk becoming a casualty of sudden changes.
- If third-party listicles aren’t part of your current strategy, check which ones ChatGPT is retrieving in your category and whether you’re on them. Look at retrieved pages, not just citations. Rankite, the fastest-growing of the five sites, is cited in only 1 of every 4 answers that retrieve it.