Better practice for huge size of table on Ruby on Rails 4 / ActiveRecord

Question

fBetter practice for huge size of table on Ruby on Rails 4 / ActiveRecord

How could I partition a big table in the postgreSQL with Active Record on Rails 4

I prefer PostgreSQL or other RDBMS, because I tried it in MongoDB. It is really slow on it.

Is Rails 4 supporting good solution for a one whole big table ?

(my case: more than 50 billions of records, size is about 20TB)

Data description

There are a User table containing name, personal_data, year fields.

The data can be divided by year and the data will be divided evenly.

Ideas

I think it is not practical to create multiple model like User_1950, User_2001,..., User_2015

I want to partition whole data by year

There are two approached I can think of

• split into different physical table in the same database. (sharding ?)
• split into different physical database on the different databases. (could it works well with Rails)

I want the solution can compatible with Active Record

it will act like

User.find(name: xxx, year: 1988) User.find(name: xxx, year: 2012)

So that I don't care about how to access the partitioned multiple tables.

As far as I know

I found a gem partitioned but which is not supporting Rails 4

Answer 1

Since you are storing the data by years, maybe you can use sharded data based on years. I would suggest octopus for use with activerecord.

Your queries will become something like:

User.using(:year_2012).find(name: xxx)