Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using multiple CSS selectors for the same ArticleItem in Scrapy

Tags:

python

scrapy

The site I am scraping has an inconsistent layout. I'm currently using this but its not returning all the titles -

article['title'] = sel.css('p[class=title] ::text').extract()

I need to use this to scrape span classes also -

article['title'] = sel.css('span[class=newstitle] ::text').extract()

Is there a way to combine two css selectors in a single ArticleItem?

like image 437
metalayer Avatar asked Oct 22 '25 23:10

metalayer


1 Answers

As simple as list concatenation:

article['title'] = response.css("p.title ::text").extract() + \
                   response.css("span.newstitle ::text").extract()
like image 97
Arthur Burkhardt Avatar answered Oct 24 '25 12:10

Arthur Burkhardt



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!