wget .vs curl using linux

Question

I'm running a test on linux using wget/curl. I can get wget to run, but I can't do the same thing with curl.

wget version:

 wget -v -O osu.tst  -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4'  'http://ohiostate.bncollege.com/webapp/wcs/stores/servlet/TBListView?catalogId=10001&storeId=33552&termMapping=N&courseXml=<?xml version="1.0" encoding="UTF-8"?><textbookorder><courses><course dept="CHEM" num="100" sect="16030" term="S12" /></courses></textbookorder> '

curl version:

curl -v -b fftfcook -A "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4" -L  "http://ohiostate.bncollege.com/webapp/wcs/stores/servlet/TBListView?catalogId=10001&storeId=33552&termMapping=N&courseXml=<?xml version="1.0" encoding="UTF-8"?><textbookorder><courses><course dept="CHEM" num="100" sect="16030" term="S12" /></courses></textbookorder>"

curl returns a partial page of content while wget gets what the browser generates. I've tried to modify the cookies/etc.. but wget is a straight "get" so it should work in curl as a straight get as well. the user agent is set to the same...

Any pointers would be helpful.

Answer 1

The problem seems to be that the GET data is not URL-encoded. Only a limited set of characters can appear in the URL or GET data without such encoding, though when entered into the browser's address bar the browser often handles this for you.

Rather than tacking the GET data onto the end of the URL, use the -d and --data-urlencode options for each value, and pass the -G option to treat these as GET data. So you'd end up with the following:

curl -v -b fftfcook -A "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4" -L  'http://ohiostate.bncollege.com/webapp/wcs/stores/servlet/TBListView' -G -d 'catalogId=10001' -d 'storeId=33552' -d 'termMapping=N' --data-urlencode 'courseXml=<?xml version="1.0" encoding="UTF-8"?><textbookorder><courses><course dept="CHEM" num="100" sect="16030" term="S12" /></courses></textbookorder>'

The various -d and --data-urlencode options will be joined together to form the GET data on the URL. Note also the changes from double to single quotes, to prevent the double quotes in the form data from causing a problem.