Data payloads in requests module?

Question

This is a site-specific issue that I'm curious about—as I'm still learning much of the dynamics of web-requests.

I know how to add a data payload to a http request using the request library for Pyhton as such:

payload = {
    "data": "whatever_data_string_here"
}

r = requests.get('url', data=payload)

What I still largely have little clue regarding is when and how to apply this type of approach. I was playing around with different site types trying to find unique ways to understand different approaches in requesting data, and have become stumped on the following site:

aaimedicine(dot)com/fap/

Ignoring their poor choice in abbreviation, I'm trying to figure out how to iterate through requests for the different specialty types in their form.

The general functionality of the site is that you can select an option from the Specialty dropdown form, click Search, and be shown relevant results.

To the best of my ability, it seems that data is retrieved through a data submitted with the request header, that looks like the following:

specialty=Massage_Bodywork&specother=&state=ALL&Submit=Search+%C2%BB

I've found that by inspecting the request headers in the console view, but don't exactly know how to apply that to my http request.

I've tried the following:

for option in options:
    r = requests.get('http://www.aaimedicine(dot)com/fap/', data={"Form Data": "specialty={}&specother=&state=ALL&Submit=Search+%C2%BB".format(option)})

This doesn't respond with the data I'm looking for, and just gives the html of the original url without before a choice was submitted.

I'd like to know how to make these types of requests so that iterating over the options would be possible.

Answer 1

I've realized that this type of POST request can be made in this instance by taking the following header source:

specialty=Speciality_Choice_Value&specother=&state=ALL&Submit=Search+%C2%BB

And utilizing it through the request module as such:

payload = {
        "specialty": "Speciality_Choice_Value",
        "specother": "",
        "state": "ALL",
        "Submit": "Search+%C2%BB",
    }

r = request.post('theurl.com', data=payload)