Puppeteer browser referer list

Question

I would like to load random list of referers from my default location path , for example: 'referers.txt' instead of adding direct "facebook url as referer.

My code:

browser = await puppeteer.getBrowserInstance(port);
const page = await browser.newPage();
page.setDefaultTimeout(PAGE_DEFAULT_TIMEOUT * 1000);
page.on('error', handlePageCrash(page));
page.on('pageerror', handlePageCrash(page));
page.setExtraHTTPHeaders({ referer: 'https://www.facebook.com/' });

Answer 1

Instead of a txt you should choose JSON to store your list of referer values in an array.

referers.json

["https://www.google.com", "https://www.facebook.com", "https://www.instagram.com"]

Then you will be able to pick a random element form the array by: array[randomIndex]. To generate a random number for the length of your array you have multiple possibilities, Math.floor(Math.random() * array.length) only one of them.

referers.js

const puppeteer = require('puppeteer')
const referers = require('./referers.json')

async function fn() {
  const randomReferer = referers[Math.floor(Math.random() * referers.length)]
  console.log(referers)
  console.log(randomReferer)
  const browser = await puppeteer.launch({ headless: false, devtools: true })
  const page = await browser.newPage()

  page.setExtraHTTPHeaders({ referer: randomReferer })

  await page.goto('https://www.instagram.com/')
}
fn()

output example:

[
  'https://www.google.com',
  'https://www.facebook.com',
  'https://www.instagram.com'
]
https://www.facebook.com