How do I scrape data from an ArcGIS Online map?

Question

I want to scrape the data from an ArcGIS map. The following map has a popup when we click the red features. How do I access that data programmatically?

Link : https://cslt.maps.arcgis.com/apps/MapSeries/index.html?appid=2c9f3e737cbf4f6faf2eb956fa26cdc5

Answer 1

Note: Please respect the access and use constraints of any ArcGIS Online item you access. When in doubt, don't save a copy of someone else's data.

The ArcGIS Online REST interface makes it relatively simple to get the data behind ArcGIS Online items. You need to use an environment that can make HTTP requests and parse JSON text. Most current programming languages either have these capabilities built in or have libraries available with these capabilities.

Here's a general workflow that your code could follow.

1. Use the app ID and the item data endpoint to see the app's JSON text:

   https://www.arcgis.com/sharing/rest/content/items/2c9f3e737cbf4f6faf2eb956fa26cdc5/data

2. Search that text for webmap and see that the app uses the following web maps:

   • d2b4a98c39fd4587b99ac0878c420125
   • 7b1af1752c3a430184fbf7a530b5ec65
   • c6e9d07e4c2749e4bfe23999778a3153

3. Look at the item data endpoint for any of those web maps:

   https://www.arcgis.com/sharing/rest/content/items/d2b4a98c39fd4587b99ac0878c420125/data

4. The list of operationalLayers specifies the feature layer URLs from which you could harvest data. For example:

   https://services2.arcgis.com/gWRYLIS16mKUskSO/arcgis/rest/services/VHR_Areas/FeatureServer/0

5. Then just run a query with a where of 0=0 (or whatever you want) and an outFields of *:

   https://services2.arcgis.com/gWRYLIS16mKUskSO/arcgis/rest/services/VHR_Areas/FeatureServer/0/query?where=0%3D0&outFields=%2A&f=json

   Use f=html instead if you want to see a human-readable request form and results.

   Note that feature services have a limit of how many features you can get per request, so you will probably want to filter by geometry or attribute values. Read the documentation to learn everything you can do with feature service queries.