RE: What are the different components involved Diyotta data connect architecture?
What are the different components involved Diyotta data connect architecture?
The core components that make up the Data Connect integration are as follows
- Kafka connect framework
- Kafka
- Zookeeper
Even if you did not have Kafka in your infrastructure, the Data Connect extension package comes with all the above components. At a high-level, the execution architecture for real time data ingestion follows the below steps
- Source connector reads from the event source (RDBMS in case of CDC)
- The new events are captured in Kafka topic specific to each source object
- Sink connector reads from the Kafka topic and writes to sink
