Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Extract javascript variable value from html document with python

I need to parse an HTML document that contains javascript code with json object.

Something like this:

<html>
   <head>
   </head>
<body>
    <script type="text/javascript">
        myJSONObject = {"name": "steve", "city": "new york"}
    </script>

   <p>Hello World.</p>
</body>
</html>

How can I extract the myJSONObject value with python?

like image 210
Shahaf Avatar asked Sep 03 '25 13:09

Shahaf


1 Answers

You can use lxml to parse the HTML, and then extract the JSON:

>>> import lxml.etree,json
>>> s = '''<html><body><script type="text/javascript">
             myJSONObject = {"name": "steve", "city": "new york"}
           </script></body></html>'''
>>> js = lxml.etree.HTML(s).find('.//body/script').text
>>> jsonCode = js.partition('=')[2].strip()
>>> json.loads(jsonCode)
{u'city': u'new york', u'name': u'steve'}
like image 55
phihag Avatar answered Sep 05 '25 03:09

phihag