Chrome Headless + Proxy Server

Question

I'm using BrowserMob-Proxy with Chrome but when I change it to headless mode the request/responses are just empty. It seems to be a SSL problem, since if I try with a http website it works just fine. Any idea what might be or how I'm able to fix it?

The code i'm using:

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium import webdriver
from browsermobproxy import Server
from pprint import pprint

MOBPATH = "/Users/tiagocardoso/Code/scraper/python/browsermob-proxy-2.1.4/bin/browsermob-proxy"

try:
    mobserver = Server(MOBPATH)
    mobserver.start()
    proxy = mobserver.create_proxy()

    chrome_options = webdriver.ChromeOptions()


    chrome_options.add_argument('--headless')
    chrome_options.add_argument('--disable-gpu')
    #chrome_options.add_argument('--no-sandbox')
    #chrome_options.add_argument('--window-size=800,600')
    #chrome_options.add_argument("--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36")

    chrome_options.add_argument('--ignore-certificate-errors')
    chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy))

    # Disable Images
    prefs = {
        "profile.managed_default_content_settings.images":2
    } 
    chrome_options.add_experimental_option("prefs",prefs)

    # Tried with those stuff...
    capabilities = DesiredCapabilities.CHROME.copy()
    capabilities['acceptSslCerts'] = True
    capabilities['acceptInsecureCerts'] = True

    driver = webdriver.Chrome(chrome_options=chrome_options,desired_capabilities=capabilities)

    proxy.new_har('google', options={"captureContent":True, "captureBinaryContent":True})
    driver.get('https://www.google.com')
    pprint(proxy.har) # returns [] if using headless

finally:
    driver.close()
    mobserver.stop()

Answer 1

I'm using java, but I managed to solve it by adding

 chromeOptions.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);

Answer 2

I had a similar problem a while ago and I ended up using a virtual display on the headless environment.

Here's an explanation on how to get it working for launching browsers in a CLI environment : https://gist.github.com/addyosmani/5336747