In a continuous delivery process, is there a proper way to automatically move data from production to development?

Question

In a common continuous-delivery process, the code is moving from a development instance to a staging instance to production instance.

For development purpose (reproducing bugs, testing performance with a full data set), most of the time developers fetch data from production database to their development environment. See, for example, this question.

In my company, we use three instances beside production in our continuous delivery process:

• latest: sync every night with our SCM trunk
• staging: with the last released version before deployment to production
• stable: with the exact same version of the software deployed in production (useful to reproduce bugs found on production)

The problem is that on the stable instance, for reproducing bugs we would like to have the exact same data set that is on production. So we would like to sync databases on a nightly basis.

Is it a good practice ? How to implement it ? Any pitfalls ?

Answer 1

Depending on the data you have in production, you may not want to replicate it back to non-production environments. (Or may not even be allowed to under certain regulations.) If you have customer data, personally identifiable information (PII), regulated data, financial data, credit card data, health data, SSN, or any other type of sensitive data, if you replicate it you need the full controls you have (or should have) in production - which you probably don't, and probably don't want.

Answer 2

There are several VDB solutions which I recommend you to look for. One of them is Delphix