rotating proxies with scrapy with authentication

Question

just a noob question but I can't seem to find the answer googling. how can i use this package https://pypi.org/project/scrapy-rotating-proxies/ if the proxy requires a user/password? do I just put it in the rotating list like that?

ROTATING_PROXY_LIST = [
    'https://username:password@proxy:port',
    'https://username:password@proxy:port',
    # ...
]

thanks

Answer 1

what worked for me is using the rotating proxy list path:

ROTATING_PROXY_LIST_PATH = 'path/to/proxylist.txt'

and then in the text file, I have my list formatted like this:

USERNAME:PASSWORD@HOST:PORT
USERNAME:PASSWORD@HOST:PORT
USERNAME:PASSWORD@HOST:PORT