What I want: A headless browser with private navigation and loggin to HTTPS proxy automatically
What I tried:
- PhantomJS -> Deprecated
- Google Chrome -> Proxy work but not with Headless
- Firefox -> Headless OK Private nav OK but can't use username and password in proxy connection
I have 4 variables, pxy["host"] ("xx.xx.xx.xx:xx"), pxy["username"], pxy["password"]
With Firefox I made some test
Test 1 - capabilities with prompt
capabilities['proxy'] = {'proxyType': 'MANUAL',
'httpProxy': pxy["host"],
'ftpProxy': pxy["host"],
'sslProxy': pxy["host"],
'noProxy': ''
}
That's open a prompt in firefox to type login and password. I tried to use alert functions to send keys in alert form but it not work with firefox.
Test 2 - capabilities with credentials
capabilities['proxy'] = {'proxyType': 'MANUAL',
'httpProxy': pxy["host"],
'ftpProxy': pxy["host"],
'sslProxy': pxy["host"],
'socksUsername': pxy['login'],
'socksPassword': pxy['password']
}
That's make an error:
selenium.common.exceptions.InvalidArgumentException: Message: Invalid proxy configuration entry: socksPassword
Test 3 - firefox preferences
ip = pxy["host"].split(":")[0]
port = pxy["host"].split(":")[1]
firefox_profile.set_preference("network.proxy.type", 1)
firefox_profile.set_preference("network.proxy.http", ip)
firefox_profile.set_preference("network.proxy.http_port", port)
firefox_profile.set_preference("network.proxy.socks_username", pxy["login"])
firefox_profile.set_preference("network.proxy.socks_password", pxy["password"])
That's doesn't do anything, just keep my ip...
Test 4 - Extensions ?
I think I can use an extension like closeproxyauth.xpi (too old) to set my proxy but I'm not sure what I did.
When I use firefox_profile.add_extension(extension=extension_path), No extension appear in browser but path is correct.
I tried to use driver.install_addon(extension=extension_path, temporary=True) but he never found my path
Dup of: How to set proxy AUTHENTICATION username:password using Python/Selenium
Selenium-wire: https://github.com/wkeeling/selenium-wire
Install selenium-wire
pip install selenium-wire
Import it
from seleniumwire import webdriver
Auth to proxy
options = {
'proxy': {
'http': 'http://username:password@host:port',
'https': 'https://username:password@host:port',
'no_proxy': 'localhost,127.0.0.1,dev_server:8080'
}
}
driver = webdriver.Firefox(seleniumwire_options=options)
Warning
Take a look to the selenium-wire cache folder. I had a problem because it take all my disk space. You have to remove it sometimes in your script when you want.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With