Is it possible to to get an empty string in a list when there is no element, using CSS selector?

Question

I want to scrape some items, which are on the same page, using Scrapy. HTML looks like this:

<div class="container" id="1">
    <span class="title">
        product-title1
    </span>
    <div class="description">
        product-desc
    </div>
    <div class="price">
        1.0
    </div>
</div>

I need to extract name, description and price.

Unfortunately, sometimes product doesn't have the description and HTML look like this:

<div class="container" id="2">
    <span class="title">
        product-title2
    </span>
    <div class="price">
        2.0
    </div>
</div>

Currently I am using CSS selectors which returns list of all elements existing on the website:

title = response.css('span[class="title"]').extract()

['product-title1', 'product-title2', 'product-title3']

description = response.css('div[class="description"]').extract()

['desc1','desc3']

price = response.css('div[class="price"]').extract()

['1.0','2.0','3.0']

Is it possible to get for example an empty string in place of missing 'desc2' when description object isn't there, using CSS selector?

Answer 1

I recommend you to rewrite you code:

for section in response.xpath('//div[@class="container"]'):
    title = section.xpath('./span[@class="title"]/text()').get(default='not-found') # you can use any default value here or just empty string
    desctiption = section.xpath('./div[@class="description"]').get()
    price = section.xpath('./div[@class="price"]/text()').get()