RE: What are the different components involved Diyotta data connect architecture?
The core components that make up the Data Connect integration are as follows
- Kafka connect framework
Even if you did not have Kafka in your infrastructure, the Data Connect extension package comes with all the above components. At a high-level, the execution architecture for real time data ingestion follows the below steps
- Source connector reads from the event source (RDBMS in case of CDC)
- The new events are captured in Kafka topic specific to each source object
- Sink connector reads from the Kafka topic and writes to sink