Scraping through VIEWSTATE

Question

I've come across an issue while I was writing a utility to scrape a web page.

I'm sending POST requests to retrieve data, I mimic the behavior of the web I'm scrapping (According to info collected with fiddler).

I've been able to automatically replace all params on my POST except VIEWSTATE. my guess is that the web is performing some logic according to the given VIEWSTATE, and that's why I'm not getting the expected result (I tried entering the value fiddler gives inside VIEWSTATE and then I do get the expected results, however I want to automate this process)

Is there a way I can edit a VIEWSTATE string without damaging it?

I tried decoding & encoding back using base64 (and finally URLEncode before POST action), couldn't keep it valid.

Answer 1

The viewstate is probably encrypted.

• http://msdn.microsoft.com/en-us/library/aa479501.aspx

The viewstate should be encrypted, anyway. It's a protection aganst XSRF attacks of the type you seem to be trying to do :-)

The bottom line is the ViewState you send back must be the same as the viewstate the server sent to you. That's pretty much what it is for. In other words, to do what you are trying to do you have to keep a copy of the viewstate the server sent you, and send the same viewstate back. The server will then send you a new viewstate, which you must submit with your next request, and so on.