Stambia Component for CDC

Change Data capture is a commonly used process to detect and track changes, made to your data, to perform a specific action.

Mostly this is used for replicating that data to an external application or a third-party database.

Stambia Component for CDC simplifies the management of change detection by providing a quick and easy way to set up your CDC processes and to manage the new data..


 

Find out how to capture your changes on data easily with our CDC ( Change Data Capture ) component for Stambia ELT. 

Use Cases and Key features in CDC

Real-time updates for your BI Databases

One of the most common use cases where Change Data Capture is required is in a BI Project.

By setting up change tracking, you reduce the overhead of extracting all your data and determining the delta in your Target BI Databases.

The changes as they happen are replicated to a staging area, which is the mirror of the Source tables.

With these staging tables, the data warehouse is populated based on business logic..

 Real-time updates for your BI Database - Change Data Capture with Stambia ETL
 

Real-time exchanges in a Data Hub Architecture

Real-time exchanges in a Data Hub Architecture with CDC by Stambia

On the other hand, in some cases, any data change in your databases, needs to get reflected in an application or another database.

To cater to this need, a Data Hub architecture needs to be set-up, to act as a central hub of information flow and exchanges between various systems.

For e.g., a manufacturing company, would want to set up CDC for their legacy database, to update Salesforce in real-time.

In this scenario, a solution that can connect and exchange data to and from different technologies and applications is beneficial.

 

Data Transformation of the changed record

A lot of times, the changed record can only be loaded to the destination, as it is.

Many replication tools are providing very basic data transformations capabilities.

The lack of ability to perform complex transformation on your data, results in having two layers i.e., replication and integration.

With this comes the need to manage two different set of tools catering to these two layers..

 Data Transformation of the changed record
 

Ability to do bi-directional replication

Ability to do bi-directional replication - Change Data Capture

 

Lastly, there is always a need, especially in a Data Hub Architecture, to have a bi-directional replication.

Let us take the same example of a manufacturing company.
Once the data changes from legacy system is updated on Salesforce, the process now should take the Salesforce Id, and update the legacy database back as an acknowledgment.

Ability of the tool to be able to perform such bi-directional replication comes then becomes essential..

t

How Stambia can implement CDC

 
Stambia Component for change Data Capture

Stambia Component for CDC simplifies the setting up of a CDC process and lets you utilize all the integration features that come with it, as a data integration tool.

The same solution packed with the CDC templates can fulfill the requirement, therefore, there is no downtime..

 

Key Features of the CDC Component

Detection of the Modification

Stambia being an E-LT tool, relies on the source database CDC capabilities.

A lot of database technologies now offer in-built APIs or processes that can be used to avoid any overhead on the source system.

As an example, PostgreSQL provides Write Ahead Logs (WAL) that can be used for data replication and subscribers can be set-up for it.

With the same agility as the other Stambia components, the Change Data Capture component is directly usable and offers a high level of productivity due to the model driven approach of Stambia and the use of native "transactional or redo log" files of databases or the use of simple triggers.

Stambia Change Data Capture component - Detection of data Modification
 

Get the history (snapshot) data

Get the history (snapshot) data

In Stambia Designer, with a click you can change your mappings from a CDC to History load.

Further, the templates help you in quickly choosing the load/integration approach, so that there is no need to manually design load and integration strategies.

Another easy feature for a history load is Stambia Replicator templates.

These come handy and save a lot of time, when your source and target structures are the same.

 
 

Manage all kind of transformation

When your source and target structures are different and you need to further perform transformations and complex logic on the data being replicated, Stambia mappings is the way.

In your mappings you can perform, aggregations, various derivations and so on.

The data can be moved to another on premise, a cloud database, or an application.

mapping stambia Manage all kind of transformation - onpremise or cloud application
 

Handle multiple subscribers

 Stambia Change Data Capture templates can manage multiple subscribers

Furthermore, the Change Data Capture templates can manage multiple subscribers.

This is particularly useful when an organization needs to use modifications in a source for multiple targets.

For example, a modification in the CRM application can be sent to the data warehousing database, and at the same time into an ERP or specific application..

 

Use CDC in different type of architectures

Stambia is flexible with any type of architecture, as an integration solution. With the CDC component, you can work on using it in a:

  • BI & Analytics Projects
  • Data Hub Architecture
  • Big Data Analytics
  • Cloud projects
  • Application Integration
Stambia ETL - ELT unified Data Integration Solution with CDC component
 

Technical specifications and prerequisites

SpecificationsDescription

Protocol

JDBC, HTTP

Structured and Sem-Structured

CDC Components are available for:

  • MS SQL
  • Oracle
  • MySQL
  • PostgreSQL
  • DB2 400
  • Teradata
  • Informix
  • HSQL

Connectivity

You can extract and load data to and from :

  • Any relational database system such as Oracle, PostgreSQL, MSSQL, ...
  • Any NoSQL database system such as MongoDB, Elastic, ...
  • Any Cloud system such as Amazon Web Service (AWS), Google Cloud Platform (GCP), Microsoft Azure, ...
  • Any ERP applications such as SAP, Microsoft Dynamics, ...
  • Any SAAS applications such as Salesforce, ...


For more information, consult the technical documentation

Standard Features

 From Stambia Runtime S17.6.6

Stambia Runtime Version

Stambia DI Runtime S17.4.7 or higher

Want to know more ?
Consult our resources

Anonymisation
Manage all your deployment, configurations, runtime environments. Scheduling and monitoring your data flows has never been so easy with Production Analytics
Stambia Production Analytics
Anonymisation
Continuous Integration, Delivery, and Deployment best practices for DevOps teams with Stambia ELT
DevOps and CICD Practices
Anonymisation
Ask advices to our experts in Data integration.
Contact us
Anonymisation
Discover our training and certifications
To know more about it
Anonymisation
Your customized demonstration
Get your demo

Subscribe to Stambia newsletter

Sign up for our email newsletters and stay connected to the best ideas in data integration

Send me the Stambia newsletter. I expressly agree to receive the newsletter and know that I can easily unsubscribe at any time.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.