2. What sources of evidence are there?

2.5. Internet search tools

3.5 Internet search tools

Can’t we just rely on internet search tools like Google or Google Scholar for finding evidence?

To some extent this depends on you and how much time you have to browse the internet for information and analyse what you find, to decide whether or not you can trust it. If internet searching is your only option, it is better than nothing!

Some of the key issues are:

General internet search tools do not confine themselves to peer-reviewed information from the academic and scientific communities and so, while they will include links to some high-quality evidence, the amount of time it will take to locate this amid everything else that is listed can make them inefficient. The onus is on you to learn how to search them efficiently, and to sift through the results and analyse them to discern the validity and currency of the evidence.

Disreputable publishers, sometimes referred to as 'predatory', have emerged online in recent years. They exploit the open access model of publishing where the author pays an Article Publishing Charge (APC). The disreputable publisher takes the money but fails to follow through with the peer-review and editing process that is the standard expected from a reputable scientific journal. This has led to a proliferation of freely available poor-quality research. While these 'predatory journals' usually would not meet the inclusion criteria for databases such as MEDLINE, they are included in Google search results. Again, it would require time and effort to weed out articles from predatory journals from your Google results. Databases can offer an efficient means of avoiding articles from predatory journals.

How can I investigate the quality of this journal?

  • Is the journal indexed in bibliographic databases like PubMed  ?
  • Is the journal listed in DOAJ : the Directory of Open Access Journals?
  • Use the tools and strategies from Think Check Submit

Using search engines, you run a risk of missing key evidence, because these tools do not take a systematic approach to indexing in the way bibliographic databases do.

  • If your results include an article from a journal, you cannot presume that the search engine looked at all the articles and all the issues of the journal.
  • The most useful results may not be first in the list; the results list may give higher ranking to some items because they are paid to.
  • The relevant, quality results can be swamped by low-quality results, so it can be very hard to pinpoint what you need.
  • Search engines often link to a finite number of results; if key resources were to come lower down the list, we might miss them.

Those producing an evidence synthesis need to report the search strategy, so that it is explicit and reproducible. This is not possible with internet search engines like Google, for reasons described below.

Internet search engines and bibliographic databases work in different ways, and while brilliant for finding information generally, search engines have imitations for EBVM.

Google crawls the internet and retrieves results using an algorithm which, for commercial reasons, is a closely guarded secret. We do know Google uses robots rather than biomedical graduates to populate its indexes and does not publish a Journals List we can check to see if key journals in our field are being included, so we cannot be sure it indexes all the relevant journals.

While great for searching for information generally, Google generates searches that are not reproducible – the ranking of search results on Google are subjective and vary according to IP address, location, and previous search history (i.e. which computer is used, where it is located geographically, and the previous searches conducted on it). This cannot be used for situations where a search should be explicit and reproducible, such as that used to produce an evidence synthesis.

Google Scholar is a search engine, not a bibliographic database, but it indexes articles. This means it can reveal useful results, but unlike bibliographic databases, it does not publish a list of the sources it is searching, so we cannot say with confidence we have performed a systematic search of all the relevant veterinary journals.

It searches the full text of web resources, so may retrieve results not found via bibliographic databases (which search the bibliographic data only, e.g. author, title, abstract, keywords). It can also be useful for citation searching.

Google will often give you results from Wikipedia , the online encyclopaedia written collaboratively by internet volunteers. Wikipedia has some great research-based information on it, but anyone with internet access can make changes to Wikipedia articles, and people often contribute anonymously using a pseudonym. This has pros and cons: it can be updated very quickly, and articles are dynamic and so can be updated to reflect new evidence. However, the quality of articles depends on the skill and knowledge of individual authors, which can be hard to ascertain from anonymous contributions.

Databases are more reliable than internet search engines since they focus on scientific literature and list the sources they search. Although internet search engines are free, they are less reliable.

Some bibliographic databases are freely available and will provide a more robust search of the evidence when compared to internet search engines.