php json encode large arrays gets truncated

Question

I am encountering a problem displaying json with json_encode. Problem is it gets truncated after a certain point. I'm guessing around 2500 lines. I've read a lot of q&a's in stackoverflow and others suggesting to increase memory limit. I've increased it already to 32m and it still gets truncated furthermore I don't think it's the problem for i've echoed the memory usage and with true i get 3.5mb and with false i get 1.5mb I also tried encoding it by 1000 but up until a certain point it still truncates.

my json looks like this

{"Id":"1"},{"Words":""},{"Position":"0"},{"Length":"0"},{"Pdf_Id":"1"}

just 2000 more

My code goes something like this:

json file

echo json_encode(array_values($db->getallqueryjsoncountStart("words_table","Allwords",4000,0)));


public function getallqueryjsoncountStart($table,$jsonparentname,$count,$start)
{

    $this->sqlquery = "select * from $table LIMIT $start,$count";
    $this->stmt = $this->conn->query($this->sqlquery);
    for ($i = 0; $i < $this->stmt->columnCount(); $i++) {
        $col = $this->stmt->getColumnMeta($i);
        $columns[] = $col['name'];
    }
    $this->initcnt = 0;
    $getqueryJson = array();

    while($this->row = $this->stmt->fetch()) 
    {
                for($x = 0;$x < sizeof($columns);$x++)
                {
                    $getqueryJson[][$columns[$x]] = $this->row[$columns[$x]];
                }
    }
    return $getqueryJson;
}

Answer 1

If the "Words" column actually contains some text there maybe some illegal character contained within the value.

Add the following options to the json_encode function:

JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT | JSON_HEX_AMP

For a description of what each does see the manual page

Also. it looks like the way you are structuring the $getqueryJson array is incorrect. Try changing as follows

$getqueryJson = array();
while($this->row = $this->stmt->fetch()) 
{
    $row = array()
    for($x = 0;$x < count($columns);$x++)
    {
        $row[$columns[$x]] = $this->row[$columns[$x]];
    }
    $getqueryJson[] = $row;
}
return $getqueryJson; 

Call the method like so.

$table_data = $db->getallqueryjsoncountStart("words_table","Allwords",4000,0);
echo json_encode($table_data, 
                 JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT | JSON_HEX_AMP);

This will produce a JSON string with the following format.

[{"Id":"1", "words":"", "Position":"0","Length":"0","Pdf_Id":"1"},
 {"Id":"2", "words":"", "Position":"0","Length":"0","Pdf_Id":"2"},
 {"Id":"3", "words":"", "Position":"0","Length":"0","Pdf_Id":"3"},
 ...
 {"Id":"4000", "words":"", "Position":"0","Length":"0","Pdf_Id":"4000"}]

If the problem persists, try calling json_last_error() right after calling json_encode.