Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to pretty print BeautifulSoup's string output

I have the following scraping code:

import requests, bs4

def make_soup():
    url = 'https://www.airbnb.pl/s/Girona--Hiszpania/homes?place_id=ChIJRRrTHsPNuhIRQMqjIeD6AAM&query=Girona%2C%20Hiszpania&refinement_paths%5B%5D=%2Fhomes&allow_override%5B%5D=&s_tag=b5bnciXv'
    response = requests.get(url)
    soup = bs4.BeautifulSoup(response.text, "html.parser")
    return soup

def get_listings():
    soup = make_soup()
    listings = soup.select('._f21qs6')
    number_of_listings = len(listings)
    print("Current number of listings: " + str(number_of_listings))
    while number_of_listings != 18:
        print("Too few listings: " + str(number_of_listings))
        soup = make_soup()
        listings = soup.select('._f21qs6')
        number_of_listings = len(listings)
    print("All fine! The number of listings is: " + str(number_of_listings))
    return listings

new_listings = get_listings()
print(new_listings)

I think def get_listings() returns listings as string, so I cannot use BeautifulSoup's prettify() on it and new_listings gets printed as one block of text.

Is there any way to print new_listings in HTML-esque format or at least have each tag printed at separate line?

like image 247
barciewicz Avatar asked Sep 03 '25 09:09

barciewicz


1 Answers

type(new_listings)
# list

Shows that new_listings is a list. Try:

print(new_listings[0].prettify())
like image 76
ilja Avatar answered Sep 04 '25 22:09

ilja