How to download CSV via terminal (SSH)?

Question

I'm trying to download the Chicago Crime stats data (CSV format) from their government website. This is the link for download:

https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD

but it only works when you copy it to the browser and hit enter.

I am wondering how to download the csv file on terminal? Can I use:

curl -O https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD > Chicago.csv

I want to save the Chicago.csv to my current work directory on ssh.

Answer 1

Your command works, however it take a long time to "compute" the file, which is huge (5360469 rows, and after 215 MB downloaded, I only got 881705 rows, so the final file size should be about 1.3GB).

If you try with another set (let's say "Flu Shot Clinic Locations - 2012", 1058 rows, 192kB) you can see that your command works perfectly, even if it does not write to Chicago.csv.

Take a look at man page:

-o, --output <file>
          Write  output to <file> instead of stdout.
-O, --remote-name
          Write output to a local file named like the remote file we get. (Only the file part of the remote file is used, the path is cut off.)

When you use the following command:

curl -O https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD > Chicago.csv

The data is written to rows.csv?accessType=DOWNLOAD, stdout remains empty, so Chicago.csv file will remain empty.

Instead, you should use either:

curl -o Chicago.csv https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD

Or:

curl https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD > Chicago.csv

Answer 2

have you tried wget? like this:

wget --no-check-certificate --progress=dot https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD