Thanks for the above reply! Useful feedback/inputs for us.
> Looks like Debezium Server doesn't require Kafka, ZK for setup. However it supports only messaging queues as sinks (targets). So to stream CDC from Postgres to a DWH - one needs to a) setup/manage messaging infra as a part of their stack to capture CDC changes b) write/manage reading from the message queue and replaying the changes to the target store (ex: Snowflake, BQ etc). With PeerDB, you can skip these 2 steps. CREATE MIRROR can have targets that are queues, DWHs or databases.
> That is true, however the ones we tried aren't very simple to work with. For example - confluent was super tricky to work with. One has to setup a sink (to message queues), use another connector to move those changes to snowflake, use something else to normalize those changes to the final table. Overall total number of moving parts were quite a lot. decodable.co might give a better experience, will give it shot! :)
> On parallel snapshotting, very interesting, looks like it is a recent feature Debezium added (March 2023). I missed that one. In regards to parallelized reading of slot, Postgres enables you to read a single slot concurrently across 2 connections using 2 separate publications - each publication filters a set of tables. Same here, we are also excited for the benchmarks. Will keep you posted! :)
> Looks like Debezium Server doesn't require Kafka, ZK for setup. However it supports only messaging queues as sinks (targets). So to stream CDC from Postgres to a DWH - one needs to a) setup/manage messaging infra as a part of their stack to capture CDC changes b) write/manage reading from the message queue and replaying the changes to the target store (ex: Snowflake, BQ etc). With PeerDB, you can skip these 2 steps. CREATE MIRROR can have targets that are queues, DWHs or databases.
> That is true, however the ones we tried aren't very simple to work with. For example - confluent was super tricky to work with. One has to setup a sink (to message queues), use another connector to move those changes to snowflake, use something else to normalize those changes to the final table. Overall total number of moving parts were quite a lot. decodable.co might give a better experience, will give it shot! :)
> On parallel snapshotting, very interesting, looks like it is a recent feature Debezium added (March 2023). I missed that one. In regards to parallelized reading of slot, Postgres enables you to read a single slot concurrently across 2 connections using 2 separate publications - each publication filters a set of tables. Same here, we are also excited for the benchmarks. Will keep you posted! :)
Thanks again!