Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python selenium - Proxy connection with host, port, username, password

What I want: A headless browser with private navigation and loggin to HTTPS proxy automatically

What I tried:
- PhantomJS -> Deprecated
- Google Chrome -> Proxy work but not with Headless
- Firefox -> Headless OK Private nav OK but can't use username and password in proxy connection

I have 4 variables, pxy["host"] ("xx.xx.xx.xx:xx"), pxy["username"], pxy["password"]
With Firefox I made some test
Test 1 - capabilities with prompt

capabilities['proxy'] = {'proxyType': 'MANUAL',
                'httpProxy': pxy["host"],
                'ftpProxy': pxy["host"],
                'sslProxy': pxy["host"],
                'noProxy': ''
                }

That's open a prompt in firefox to type login and password. I tried to use alert functions to send keys in alert form but it not work with firefox.

Test 2 - capabilities with credentials

capabilities['proxy'] = {'proxyType': 'MANUAL',
                'httpProxy': pxy["host"],
                'ftpProxy': pxy["host"],
                'sslProxy': pxy["host"],
                'socksUsername': pxy['login'],
                'socksPassword': pxy['password']
                }

That's make an error:

selenium.common.exceptions.InvalidArgumentException: Message: Invalid proxy configuration entry: socksPassword

Test 3 - firefox preferences

ip = pxy["host"].split(":")[0]
port = pxy["host"].split(":")[1]
firefox_profile.set_preference("network.proxy.type", 1)
firefox_profile.set_preference("network.proxy.http", ip)
firefox_profile.set_preference("network.proxy.http_port", port)
firefox_profile.set_preference("network.proxy.socks_username", pxy["login"])
firefox_profile.set_preference("network.proxy.socks_password", pxy["password"])

That's doesn't do anything, just keep my ip...

Test 4 - Extensions ?
I think I can use an extension like closeproxyauth.xpi (too old) to set my proxy but I'm not sure what I did. When I use firefox_profile.add_extension(extension=extension_path), No extension appear in browser but path is correct.
I tried to use driver.install_addon(extension=extension_path, temporary=True) but he never found my path

like image 779
Timothée Dautrême Avatar asked Dec 20 '25 18:12

Timothée Dautrême


1 Answers

Dup of: How to set proxy AUTHENTICATION username:password using Python/Selenium

Selenium-wire: https://github.com/wkeeling/selenium-wire

Install selenium-wire

pip install selenium-wire

Import it

from seleniumwire import webdriver

Auth to proxy

options = {
'proxy': {
    'http': 'http://username:password@host:port',
    'https': 'https://username:password@host:port',
    'no_proxy': 'localhost,127.0.0.1,dev_server:8080'
    }
}
driver = webdriver.Firefox(seleniumwire_options=options)

Warning
Take a look to the selenium-wire cache folder. I had a problem because it take all my disk space. You have to remove it sometimes in your script when you want.

like image 115
Timothée Dautrême Avatar answered Dec 23 '25 06:12

Timothée Dautrême



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!