Suggested practice for storing a lot of user uploaded files?

Question

I 'm looking a scalable way to store a lot of files from users. My strategy is to save them with a random name and save the name and file relationships in the database.

My question is what 's a suggested practice for storing the files on disk? One limitation I 've faced is that if I save them all in one directory, the OS performance degrade which is a no,no.

Obviously I can create directories at specified intevals and split the load but that 's just a hack and I 'm sure there must be tried and true procedures to deal with storage. I just can't seem to find them :)

I'm looking to host the files privately and not on an external storage like S3.

Answer 1

As you already give each file a random name, chop it into subdirectories.

Let's say there is a random name:

$randomName = 'Akj823daosd03hdx';

Save it to:

/Ak/j8/23/daosd03hdx

This will limit the number of nodes in each directory. If you have a good random distribution, this should be equally distributed.

Depending on how many files you have you can also distribute those differently, but with random names its normally straight forward in such a manner. You can vary with the length and number of subdirectory and final file names depending on your needs.