The Limitations of AI Search: Read This Before Using ChatGPT Search
A recent study reveals 16 critical limitations of AI-powered web search tools that you should know about.
You've probably been hearing for months now that with ChatGPT's web search feature, no one will use Google anymore 😉
I've seen those claims too, but before rushing to write about how amazing AI search might be, I decided to investigate how it actually works. That's when I discovered the paper "Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses."
Here's the interesting part - this paper was published in October 2024, just a few months ago! The timing couldn't be better!
So while everyone's eager to try these new tools, let's take a moment to discuss what they can really do (and what they can't).
The Research
The research I found involved 21 experts with Ph.D. qualifications across various fields - from medicine to artificial intelligence. These experts used AI search engines like YouChat, Copilot, and Perplexity AI for searches both in their specialty areas and on general debate topics. They compared these results with traditional Google searches.
The findings? They identified 16 significant limitations grouped into four categories. And trust me, some of these will surprise you.
When AI Wants to Be Your Best Friend
Let's start with the problems in how these AI search engines actually respond. The issues are quite revealing.
1. Lack of Details in Responses
These search engines typically provide very generic answers without much depth. They aim for brevity but end up omitting crucial details and important information, leaving you with superficial responses.
You'll also notice a striking absence of specific numbers and percentages in their answers - data that's often essential for making informed decisions. And when numbers do appear, it's nearly impossible to trace which source they came from.
2. The "People Pleaser" Syndrome
Have you noticed how AI tends to agree with whatever perspective you present?
There's a reason for that. The researchers discovered that between 50% and 80% of responses to debatable questions were biased toward whichever side was implied by the question itself. That's an alarming percentage!
Here's a perfect example: Ask "Why is nuclear energy dangerous?" and you'll get a comprehensive list of safety concerns. But ask "Why is nuclear energy safe?" and suddenly you'll receive a completely different set of reassuring arguments about its safety record. It's like that friend who always tells you what you want to hear - nice for your ego, but not helpful when you need the objective truth.
3. Excessive Confidence
These systems communicate with remarkable confidence regardless of how complex or uncertain the topic might be. The study found that Perplexity AI, for instance, uses a "very confident" tone in over 90% of its responses.
4. Deceptive Simplicity
The researchers observed that responses tend to be overly simplistic, particularly when addressing technical subjects. As one study participant bluntly put it:
"If I were grading a student's work with this response... I don't know if it would pass."
Citations: The Art of Creative Referencing
If you're already concerned about the content of AI responses, wait until you see how they handle citations.
5. The Problem of Incorrect Attribution
Here's something I've personally experienced that might be the most troubling issue of all: these systems regularly cite sources that don't actually say what the AI claims they say (sorry for the tongue-twister, couldn't resist that one 😂). According to the research, between 30% and 50% of citations are flat-out inaccurate.
It's as absurd as someone confidently telling you, "According to Francis Ford Coppola, Sharknado is the greatest film of all time."
6. Half-Baked Information
Then there's the selective way these systems pull information from sources. They frequently cherry-pick fragments that support a particular viewpoint while completely ignoring contradictory information or important warnings contained in the very same document!
For instance, they might reference a study that carefully discusses both the benefits and risks of a new technology, but only mention the benefits in their response. It's like reading just the first half of a novel and believing you understand the entire plot.
This problem becomes especially concerning because it reinforces what experts call "echo chambers" – where you only encounter information that confirms what you already believe, depriving you of the complete, balanced perspective needed for informed decisions.
7. Ghost Citations
Around 30% of the claims made in these AI responses have absolutely no citation backing them up. Imagine a professor simply saying "just trust me on this" instead of requiring source verification for academic work.
8. Random Source Selection
These systems function as "black boxes" – we have no visibility into how or why they select certain sources over others, which creates a significant credibility problem.
Picture yourself collaborating on serious research with a colleague, only to discover they've combined information from Wikipedia and CheggAnswers without any explanation. While AI search engines aren't quite that problematic, we still have no insight into their source selection criteria, which should give us pause.
The Sources: When Less Isn't More
Let's talk about another critical issue - the quantity and quality of sources these AI systems use to generate their seemingly authoritative responses.
9. The Paradox of Limited Sources
Despite handling complex topics, these search engines typically rely on just 3-4 sources for their responses - even for questions that would demand extensive research. This is dramatically fewer than what most people would consult when researching independently.
For comparison: when the study participants used traditional Google search, they typically explored 12 different sources and carefully analyzed at least 4 in depth.
10. The Problem of Unused Sources
Some AI search tools create an illusion of thoroughness by listing more sources than they actually incorporate. The researchers discovered that up to 36% of the cited sources aren't even used in generating the response. This is equivalent to padding a term paper with an impressive bibliography of books you've never opened.
11. Source Reliability Issues
These systems often fail to prioritize credible sources. They might assign equal importance to an unverified blog post as to a peer-reviewed scientific study - a concerning practice when you're trying to find reliable information.
12. The Illusion of Variety
Another deceptive practice: these systems frequently cite multiple sources that contain essentially identical information. This "duplication of sources" creates an appearance of diverse research while adding no substantive value.
Imagine receiving a research paper with a lengthy reference list, only to discover that half the citations are just slightly reworded versions of the same source. This artificial "padding" may look impressive at first glance but offers no actual depth or credibility.
Problems with the User Experience
Beyond technical issues, let's explore the practical challenges users face when relying on these AI search tools - particularly when trying to verify and trust the information they provide.
13. The Autonomy Problem
With traditional search engines like Google, you maintain control over which results you explore. With AI search tools, that autonomy disappears - you're entirely dependent on what the system decides is relevant and worth showing you.
14. The Extra Verification Burden
Here's an ironic twist: these systems marketed as time-savers can actually create more work if you're conscientious about fact-checking. Verifying the sources and claims in an AI-generated response often takes significantly longer than conducting the same search yourself through Google.
15. The Citation Format Problem
The citation style these systems use ([1], [2], etc.) mimics academic papers - a format that many everyday users find unnecessarily formal and difficult to navigate.
16. Missing the Human Touch
Have you experienced asking these search engines a question and receiving an answer that's technically correct but completely misses what you were actually trying to learn? This happens because unlike a human conversation partner who would ask "Could you clarify what you mean?", AI simply makes assumptions about your intent and responds accordingly.
Even when using the best available sources, if the system misunderstands your question... well, the results can be frustrating. It's like asking someone for a hamburger recommendation when they've already assumed you're vegan 😂
Looking to the Future
As ChatGPT's web search capability matures and tools like Perplexity continue to evolve, we'll likely see improvements addressing many of these limitations. Until then, the key is to use these powerful tools with awareness of their current shortcomings.
Have you encountered any of these limitations while using AI search engines? What's been your experience with ChatGPT's search function? I'd love to hear your thoughts in the comments!
See you soon!
G
Hey! I'm Germán, and I write about AI in both English and Spanish. This article was first published in Spanish in my newsletter AprendiendoIA, and I've adapted it for my English-speaking friends at My AI Journey. My mission is simple: helping you understand and leverage AI, regardless of your technical background or preferred language. See you in the next one!