Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS DMS replicate only selected columns

I'm using AWS DMS to CDC from MySQL on-premise database to AWS S3.

  1. Is it feasible to transfer only a few columns from the source table to target? I have a table with more than 50 columns and only need 10.
  2. If I want to transfer 5 tables from the source, is it best practice to create 1 replication task for each table, or put all 5 in one?
like image 369
datahack Avatar asked Sep 17 '25 13:09

datahack


2 Answers

  1. Yes, by using transformations.

  2. If your replication tasks are full-load-and-cdc, then, definitely no. Because each replication task will end up with the redo logs being transferred to your replication instance, consuming memory CPU power, and network bandwidth.

like image 98
tsz662 Avatar answered Sep 20 '25 05:09

tsz662


  1. You can do that by creating filter conditions. This video will help you out.

  2. That is totally up to you. No matter how replication tasks are created, it works normally. If your tables are so different from each one, you can consider separating them.

like image 44
sinwoobang Avatar answered Sep 20 '25 06:09

sinwoobang