Multiple threads and Sqlite - database disk image is malformed

Question

When I try to read (select) from sqlite database from ~1000 processes i keep getting errors like

SQL error: database disk image is malformed

or less often

SQL error: file is encrypted or is not a database

from some of threads.

I test this program & db on 3 hosts:

1. with older 1-core CPU - no problems, no errors
2. with 1-core VPS - errors occur
3. with 8-core i7 - errors occur

DB is always in /tmp/ folder so it's basically held in RAM. Integrity check says DB is fine.

What can cause this problem and is there a solution?

Answer 1

Solved. Problem was due to connection to database was opened before fork(). Many processes used one connection. Now it's opened after fork() and everything is ok.

Sqlite3 provides support to multiple selects at the same time only in the case of multiple connections to this db. In this case multiple processes were trying to query db through the same connection. On slower hosts it wasn't a problem but on newer machines probably process run too quickly and previous queries were overwritten/interrupted by new ones. Just guessing :)