How to automatically download the files that have a download button on a webpage?

Question

I need to download all the files that have a download button on a webpage, such as this page:http://a.vmall.com/topic/list/d6aac030f17e49d990b8a1996e872d53, every app has a download button, that requires you to click "download" on the website, then it begin to download

How do I write a python or selenium script to automatically download the all files that has a "download" button? thanks

Answer 1

selenium is useless here, just use BeautifulSoup to get the link.

For example:

import re
import requests
from bs4 import BeautifulSoup

link = 'http://a.vmall.com/topic/list/d6aac030f17e49d990b8a1996e872d53'

r = requests.get(link)
soup = BeautifulSoup(r.text, "html.parser")

for i in soup.find_all('a', {'class': "app-btn-down"}):
    print(re.search('http://.*\.apk', i.get('href')).group(0))

Output:

http://122.11.38.214/dl/appdl/application/apk/72/723e6026cf0f4a2eb4b7f564f6c82715/com.google.android.wearable.app.cn.1510301352.apk
http://122.11.38.214/dl/appdl/application/apk/17/1768953363fc4bc39957bc2d1a8e0cb0/com.flightmanager.view.1510161532.apk
http://122.11.38.214/dl/appdl/application/apk/49/494ce5d2b4074fc8a47709fc10145d80/com.tencent.qqlite.1510191652.apk
http://122.11.38.214/dl/appdl/application/apk/6d/6de0e6ea05ee4b53a6a730912f1cb732/com.tencent.news.1510271932.apk
http://122.11.38.214/dl/appdl/application/apk/82/82ff5be0aee442feb6501f604e30c815/com.eg.android.AlipayGphone.1510291547.apk
http://122.11.38.214/dl/appdl/application/apk/5a/5ae799a7fb4545e4903abaf2b5800751/ctrip.android.view.1511051017.apk
http://122.11.38.214/dl/appdl/application/apk/64/643e6561255a473aaa3348b36d045404/com.tencent.mm.1510201247.apk

Then you could save there links, or use other package such as urllib or requests to download them.