Puppeteer: log HTML DOM object in evaluate()

Question

When I run a puppeteer script , if I want to do a log inside the page.evaluate I can use a code like bellow

page.on('console', consoleObj => console.log(consoleObj.text()));

Unfortunately , it does not work if I want to log an object:

For example the code below does not log properly a js obj:

page.on("console", log => {
   console[log._type](log.text());
});

await pageBis.evaluate(() => {
   let selector = `select.form1 option[value="optionToSelect"]`;
   let optionObj = document.querySelectorAll(selector)[0];
   console.log(`optionObj : ${JSON.stringify(optionObj)}`);
});

It displays :

optionObj : {}

Do you know how to handle it please?

Answer 1

Problem

If you open about:blank on your default browser and run the following,

console.log(JSON.stringify(document.querySelector('body')))

and it will return {}, because it's trying to convert a HTML element to a string which is not possible.

Solution: domjson

There are lots of ways to do this. You can use such library from within the browser with addScriptTag and console out or use as you wish.

Usage:

// add the script to the window like <script src="...">
await page.addScriptTag({url:"https://www.unpkg.com/domjson"})

// run the code inside browser, we have domJSON available on window now
await page.evaluate(()=>{

  // use it
  const bodyJson = domJSON.toJSON(document.querySelector('body'));

  // log it
  console.log(JSON.stringify(bodyJson, true, 2))
})

Result:

{
  "meta": {
    "href": "about:blank",
    "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36",
    "version": "0.1.2",
    "clock": 1,
    "date": "2018-10-22T15:08:03.973Z",
    "dimensions": {
      "inner": {
        "x": 1920,
        "y": 476
      },
      "outer": {
        "x": 1920,
        "y": 993
      }
    },
...

domjson was last published on npm 4 years ago, but still works to satisfy the needs of this question and does not probably pose any great security issues, also have no external dependencies. Their github page is last updated 20 days ago with several fixes.

Note:

• There are several other modules to deal with dom like jsdom etc.
• Whatever you do inside .evaluate, it runs in the browser context.
• Whatever you do outside it will run in the nodeJS context. So you will have to treat them differently. Learn more about context here.
• DOM means HTML DOM (Document Object Model), it is (only) available inside the browser, so it's best to use it there if possible.