Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get only the first link of a URLs list with BeautifulSoup

I parsed an entire HTML file, extracting some URLs with Beautifulsoup module in Python, with this peace of code:

for link in soup.find_all('a'):
    for line in link :
        if "condition" in line :

           print link.get("href")

and i get in the shell a series of links that observe the condition in the if loop:

  • http:// ..link1
  • http:// ..link2
  • .
  • .
  • http:// ..linkn

how can i put in a variable "output" only the first link of this list?

EDIT:

The web page is : http://download.cyanogenmod.com/?device=p970 , the script have to return the first short URL (http://get.cm/...) in the HTML page.

like image 225
Gabriele Salvatori Avatar asked Oct 21 '25 02:10

Gabriele Salvatori


2 Answers

You can do it with a oneliner:

import re

soup.find('a', href=re.compile('^http://get.cm/get'))['href']

to assign it to a variable just:

variable=soup.find('a', href=re.compile('^http://get.cm/get'))['href']

I have no idea what exactly are you doing so i will post the full code from scratch: NB! if you use bs4 change the imports

import urllib2
from BeautifulSoup import BeautifulSoup
import re

request = urllib2.Request("http://download.cyanogenmod.com/?device=p970")
response = urllib2.urlopen(request)
soup = BeautifulSoup(response)
variable=soup.find('a', href=re.compile('^http://get.cm/get'))['href']
print variable

>>> 
http://get.cm/get/4jj
like image 123
root Avatar answered Oct 24 '25 18:10

root


You can do this more easily and clearly in BeautifulSoup without loops.

Assuming your parsed BeautifulSoup object is named soup:

output = soup.find(lambda tag: tag.name=='a' and "condition" in tag).attrs['href']
print output

Note that the find method returns only the first result, while find_all returns all of them.

like image 24
jdotjdot Avatar answered Oct 24 '25 18:10

jdotjdot