Skip to content

Commit f243abb

Browse files
authored
Update ReadMe.md
1 parent b37a90a commit f243abb

File tree

1 file changed

+18
-0
lines changed
  • stored-procedures/sp_cdc_DynamoDB_to_Redshift

1 file changed

+18
-0
lines changed

stored-procedures/sp_cdc_DynamoDB_to_Redshift/ReadMe.md

+18
Original file line numberDiff line numberDiff line change
@@ -130,3 +130,21 @@ Sample IAM role will look like this.
130130
]
131131
}
132132
```
133+
134+
135+
136+
#### Observations from CDC (change data capture) performance testing between DynamoDB and Redshift.
137+
1. Not all records showed up in the KDS stream / Materialized View right away - Sometime it took two iterations.It is important to design KDS for throughput and latency.
138+
2. Single record update is not that different from group of records.
139+
3. Providing a minute for KDS to catch up and then run this process is recommended and
140+
4. Update/Delete/New record changes are running in the same time window and has no considerable difference between type of CDC image in terms of processing times.
141+
5. Initial execution is slower than next runs. This should normalize if this process is running every 15 mins on schedule.
142+
6. Better suited for Near - Real time use cases with 15 min window.
143+
144+
| Scenario | Iteration 1 | Iteration 2 | Iteration 3 | Iteration 4 |
145+
| --- | --- | --- | --- | --- |
146+
| Multiple tables pushing data to a single stream(5 tables) | 38 sec | 15 sec | 43 sec | 29 sec |
147+
| Number of updates processed in one iteration(200) | 65 sec | 14 sec | 28 sec | 19 sec |
148+
| Single record updated/added | 20 sec | 3 sec | 9 sec | 28 sec |
149+
| Number of inserts processed in one iteration(200) | 9 sec | 31 sec | 25 sec | 12 sec |
150+

0 commit comments

Comments
 (0)