Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get the inner html of an element using scrapy

Tags:

python

scrapy

This is my HTML document

<div class='my-class'>
    <p>some text</p>
</div>

I want to get the inner HTML of div.my-class element, which is:

<p>some text</p>

The inner html is not always a <p> it could be some other element.

Here is what I have tried but not able to get the desired output:

res = response.css('div.my-class').get(); 

/* result */
<div class='my-class'>
 <p>some text</p>
</div>

//-------------------------------------------

res = response.css('div.my-class::text').get(); 

/* result */
some text
like image 538
Hooman Bahreini Avatar asked Dec 05 '25 08:12

Hooman Bahreini


1 Answers

Here is a way to get the children of the element of class my-class:

html = "<div class='my-class'><p>some text</p></div>"
response = Selector(text=html, type="html")
print(response.xpath('//*[@class="my-class"]/*').get())
like image 63
Karl Avatar answered Dec 07 '25 23:12

Karl



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!