Architecture of an image hosting site

Question

I'm sure many here are aware of image hosting sites, like imgur, min.us, photobucket etc.

Not that I want to develop one, but besides just uploading the file, organising it in some directory somewhere, what architectural considerations are involved in these sites? Especially when there's millions of page views a day (like imgur, I'd imagine)

I'm curious about this because it seems that a lot of sites (say, dating websites etc) would be pretty image intensive. Even if it's not for millions of page views, what are some basic architectural requirements of efficient image deliveries online?

Answer 1

If you are talking about internal server architecture, there are many considerations.

Is there any security? Is the retrieval of an image subject to any business logic? What web server will you be using? What type of request distribution are you looking at (will you be serving 1 of 10 images 50% of the time and 1 of 100,000 the other 50%?

Let's say no security, and you are using a simple web server that is not going to do any caching for you.

That (by which I mean caching) is going to be your main issue. You will want to write an isapi filter/module/whatever to keep your images with the highest x% hits in memory. x will depend on your distribution pattern.

As far as storage, do not store your images in a database. File systems are very very good at traversing directory structures. Just put your "hello.png" image in an 'images/png/h/e/l' directory using how ever number of letter directories you find fits you library to < 100 images per directory.