Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to detect whether request is from prerender.io(crawlers) or from real user(browser) in JavaScript?

I wanted to detect whether request is from prerender.io(library used to render the angularjs application when request is from crawlers) or from real users.If request is from prerender, then i have to redirect to the page which is only designed for SEO purpose.

I tried setting cookies to detect but it doesn't work since prerender.io executes javascript code and even the cookie/session storage works in prerender.io.

After some research i found that we can detect user agent and since prerender.io calls site in headless browser(i.e phantomJS)

if (/PhantomJS/.test(window.navigator.userAgent)) {
          //  console.log("PhantomJS environment detected.");
 } else {
        //  console.log("PhantomJS environment not detected.");    
  }

but is it a permanent/proper/best fix for this issue? is there any other solution?

like image 263
Nishan shetty Avatar asked Sep 09 '25 15:09

Nishan shetty


1 Answers

In their FAQ, you can see the UserAgent they use:

What is your crawler's user agent?

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3159.5 Safari/537.36 Prerender (+https://github.com/prerender/prerender)

You can use userAgent to know if the request comes from their crawler.

like image 109
Cyril Gandon Avatar answered Sep 12 '25 04:09

Cyril Gandon