create binary file like maxmind

Question

I have a set of public data I need to give to my clients, (data is private and vary per clients). Every client has different structure and needs different type of data. I have 50+ clients and they all required different information which also are in different languages (French, Spanish, English, Italian and German).

Some of them get 10 fields per unique records, some get 2.

Some of them uses linux as their main (centos i think) and other uses windows server (local server and web server ), also this file will be use to local application (win32, php etc...).

after digging on a common solution i found out I can create something like maxmind (i think it's a binary file and you provide an index to retrieve the info.

I cannot create an API since some of the client uses their computers on a private network that has access to internet from specific hours (kind of like a school lab).

Now, is there a way to create a data file like maxmind?

Thanks

Answer 1

Your best bet is to, on your end, store all of your data inside a relational database. Depending on the size of the dataset and how it needs to be accessed, MySQL or SQLite might be good options. This will make it very easy to select which data you need on a per-client basis.

For each client, develop an SQL query to extract all the data they need (and no more). Write a script to run the query and populate either another database (to be sent to the client) or a simple CSV file. (A CSV might be easier if they plan on adding the data to their own database, then they can easily diff the changes from whatever you sent them last).

Now, you have a convenient storage solution for your data and a simple way to extract the data relevant to each client. You can easily automate the process by writing scripts to execute the per-client queries periodically and even mail them out.

I would recommend against using the MaxMind format unless you have a specific reason for it. You'll find much more support using a more common database interface.

Answer 2

Your question is:

is there a way to create a data file like maxmind?

The answer is, yes - if you're prepared to reverse engineer a Maxmind API you are familiar with. For me that would be the C# sample. You can take a look at how the code reads the format and decipher from this what format it expects.

As an alternative, create your own binary format. To create your own binary format is pretty simple, you can decide on an internal format (maybe use JSON or XML) and then covert it to Base64. All your clients will need to do is convert from base64 (the binary data) to a string format and then use a library to interpret the JSON or XML.

Use something like this for converting JSON to binary: http://wiki.fasterxml.com/SmileFormat