Split files on folders into more folders with php

Question

I had a problem lately and couldn't get a solution for it , i have 100,000 image in 1 folder in my server called

/Images/

inside this folder there are 100,000 image something like this:

/Images/image1.jpg
/Images/image2.jpg
/Images/image3.jpg
...

using cpanel file manger i cant compress these files and i cant open the Images folder it overtime and never open , even with filezilla and smartftp the Images folder shows only 10,000 image , so the best solution i have now is to split the folder into more folders and compress them as parts.

i want it to be done something like this :

/Images/part1/image1.jpg
/Images/part1/image2.jpg
/Images/part1/image3.jpg
...

/Images/part2/image10001.jpg
/Images/part2/image10002.jpg
/Images/part2/image10003.jpg
...

...

so at the end i will have 10 folders in each folder i have 10,000 image.

Can i do this using PHP ?

thanks.

Answer 1

This should work for you:

1. I use glob() to get all files in the directory which matches the pattern

2. I created chunks of 10 with array_chunk() out of the array with all files. So i converted the array e.g. Array ( [0] => ... [2000] => ...) to an array which looks like: Array ( [0] => Array ( [0] => ... [9] => ...) )

3. I created as many folders as you have chunks with mkdir()

4. And at the end I moved all files in the directory's with rename()

<?php

    //Configuration
    $folderPattern = "part";
    $chunkSize = 10;
    $path = "images/";

    //get images
    $files = glob($path . "*.*");
    $chunks = array_chunk($files, $chunkSize);
    
    //create folders
    for($i = 1; $i <= count($chunks); $i++) {
        if(!file_exists($path . $folderPattern . $i))
            mkdir($path . $folderPattern . $i, 0700);
    }

    //move images
    for($i = 1; $i <= count($chunks); $i++) {
        for($x = 0; $x < count($chunks[$i-1]); $x++) {
            rename($path . basename($chunks[$i-1][$x]), $path . $folderPattern . $i . "/" . basename($chunks[$i-1][$x])); 
            echo $path . basename($chunks[$i-1][$x]) . " -> " . $path . $folderPattern . $i . "/" . basename($chunks[$i-1][$x]) . "<br />";
        }
    }

?>

Depending on your knowledge about basic php you may want to look into this:

• for loop
• concatenation
• if statements
• control structures

Answer tested:

• With 10'000 images
• Same configuration as in the answer
• Image size: 31'503 Bytes
• Time of execution: 15-17 sec (average: 16.189925909042 sec)

EDIT:

To change your folder structure from:

X folders with 10 images

to:

10 folders with X images

This script convert's X folders to X/10 folders e.g. 8139 -> 813 folders. So if you want to convert 8139 -> 813 run it 2x and if you want 8139 -> 81 run it 3x.

Note: If a file already exists e.g. /images/part1/xy.jpg and you want to move the file into this folder, it automatically appends - TEMP-[random number] to the name so it doesn't get lost. As example if you now want to move the file: /images/partXY/xy.jpg into the folder from above the file get's renamed to: /images/part1/xy.jpg - TEMP-906222766. So you can spot these files easy and rename it to what you want.

(If you want a full explanation to this code let me know in the comments)

<?php

    //Since it can take a few seconds more than 30 and default is (mostly) 30
    set_time_limit(120);


    //Configuration
    $path = "images/";
    $chunkSize = 10;
    
    //Get all dirs
    $dirs = glob($path . "*");

    //Get all files
    foreach($dirs as $dir)
        $files[] = glob($dir . "/*.*");
    
    //Define MAX folders
    $files = array_chunk($files, $chunkSize);

    
    foreach($files as $key => $innerArray) {
        $baseFolder = dirname(str_replace($path, "", array_column($innerArray, 0)[0]));
        
        for($i = 1; $i < count($innerArray); $i++) {
            foreach($files[$key][$i] as $file) {
                if(file_exists($path . $baseFolder . "/" . basename($file)))
                    rename($file, $path . $baseFolder . "/" . basename($file) . " - TEMP-" . mt_rand());
                else
                    rename($file, $path . $baseFolder . "/" . basename($file));
                
                echo $file . " -> " . $path . $baseFolder . "/" . basename($file) . "<br />";
            }
            
            //Delete dir
            rmdir($path . str_replace($path, "", dirname($files[$key][$i][0])));
            echo "<br /><br />Removed: " . $path . str_replace($path, "", dirname($files[$key][$i][0])) . "<br /><br />";
        }

    }

?>

Answer tested:

• With 10'000 images | à 1'000 folders e.g. 10 images per folder
• Same configuration as in the answer
• Image size: 31'503 Bytes
• Script called: 2 times:
  1. 1'000 folders -> 100
  2. 100 folders -> 10
• Time of execution: 29-33 sec (average: 31.29639005661 sec)