Preparing SQL Statements with PDO

Question

My code looks like this:

// Connect to SQLite DB
DB('/path/to/sqlite.db');

DB('BEGIN TRANSACTION;');

// These loops are just examples.
for ($i = 1; $i <= 10000; $i++)
{
    for ($j = 1; $j <= 100; $j++)
    {
        DB('INSERT INTO "test" ("id", "name") VALUES (?, ?);', $i, 'Testing ' . $j);
    }
}

DB('END TRANSACTION;');

And here is the DB() function:

function DB($query)
{
    static $db = null;

    if (is_file($query) === true)
    {
        $db = new PDO('sqlite:' . $query, null, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_WARNING));
    }

    else if (is_a($db, 'PDO') === true)
    {
        $result = $db->prepare($query);

        if (is_a($result, 'PDOStatement') === true)
        {
            if ($result->execute(array_slice(func_get_args(), 1)) === true)
            {
                if (stripos($query, 'INSERT') === 0)
                {
                    return $db->lastInsertId();
                }

                if (stripos($query, 'SELECT') === 0)
                {
                    return $result->fetchAll(PDO::FETCH_ASSOC);
                }

                return $result->rowCount();
            }
        }

        return false;
    }

    return true;
}

The problem is that the DB() call inside the inner loop is taking to long to complete, I was thinking that doing something like this could speed up considerably the execution:

DB('BEGIN TRANSACTION;');

for ($i = 1; $i <= 10000; $i++)
{
    $queries = array();

    for ($j = 1; $j <= 100; $j++)
    {
        $queries[] = 'INSERT INTO "test" ("id", "name") VALUES (?, ?);' /*, $i, 'Testing ' . $j;*/
    }

    DB(implode("\n", $queries));
}

DB('END TRANSACTION;');

The problem is I don't know how I could prepare (replace and escape) all the question marks with the respective variables, is there any way I can accomplish this?

Answer 1

Why do you use prepared statements if you "prepare" them in the loop ? (in the DB function)

Make a prepare befor the loop, and in the loop just give the values and execute. Of course this would require a rewrite of your DB method.

Oh and btw. is your ID column the primary key ? if so you have another problem couse "i" would be for 100 "j" the same :)

For example:

$sth = $dbh->prepare('INSERT INTO "test" ("id", "name") VALUES (:id, :name)');
$j=0;
for ($i = 1; $i <= 10000; $i++){
   $j = ($j==100) ? 0 : $j++;
   $sth->execute(array(':id' => $i, ':name' => 'Testing ' . $j));     
}

Answer 2

If you are inserting lots of data into a table, try inserting the data in one query.

$query = 'INSERT INTO "test" ("id", "name") VALUES ';
$data = array();
for ($i = 1; $i <= 10000; $i++) {
  for ($j = 1; $j <= 100; $j++) {
    $query .= '(?,?),';
    $data[] = $i;
    $data[] = 'Testing '.$j;
  }
}

$query = substr($query, 0, -1);
DB($query, $data);

This should eliminate the overhead with single insert queries. There's a limit on the query lenght though, if you have troubles with query lenght, try issueing DB() calls more often in for loop.