Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python cgi FieldStorage slow, alternatives?

Tags:

python

cgi

I have a python cgi script that receives files uploaded via a http post. The files can be large (300+ Mb). The thing is, cgi.FieldStorage() is incredibly slow for getting the file (a 300Mb file took 6 minutes to be "received"). Doing the same by just reading the stdin took around 15 seconds. The problem with the latter is, i would have to parse the data myself if there are multiple fields that are posted.

Are there any faster alternatives to FieldStorage()?

like image 854
Ash Avatar asked Dec 02 '25 05:12

Ash


1 Answers

"[I] would have to parse the data myself"

Why? CGI has a parser you can call explicitly.

Read the uploaded stream and save it in a local disk file.

For blazing speed, use a StringIO in-memory file. Just be aware of the amount of memory the upload will take.

Use cgi.parse(mylocalfile).

like image 146
S.Lott Avatar answered Dec 03 '25 21:12

S.Lott



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!