One of the data targets which our customers find highly interesting is Snowflake. Snowflake is a cloud-based data warehousing platform designed to handle and analyze large volumes of data. It's a software-as-a-service (SaaS) solution that allows organizations to store, manage, and analyze data using cloud infrastructure, without the need to manage the underlying hardware or software.

How do companies use SNP Glue and Snowflake?

Here is an actual customer example using SNP Glue to integrate SAP with Snowflake: imagine a huge corporation, headquartered in Germany, but active around the globe. They have 30+ production SAP systems, from which they capture data (mostly using SAP’s own SLT). And the scope of this is massive, we are talking about tens of thousands of tables per system. Now, with SNP Glue, we can take all this data, and use Glue’s very own CDC (change data capture) – sometimes in tandem with SLT’s delta capture – to replicate all SAP data including constant changes to a Snowflake based data warehouse in the cloud.

Why would companies use Snowflake?

Snowflake is based on key features of SQL, “elasticity” (i.e. scalability) and comes with powerful data processing features to cover the “T” in ETL (Extraction – Transformation – Loading) which makes a lot of sense looking at modern integration architectures, which is mostly about ELT rather than ETL, simply to de-couple the complexities of data integration and data transformation. In short, Snowflake is a somewhat cloud-agnostic SAS offering of a Data Platform (formerly known as Data Warehouse).

Snowflake supports integration with various data science and AI tools through connectors and APIs. You can use popular AI libraries and frameworks alongside Snowflake for building and training models.

To summarize in simple words: Snowflake is an extremely powerful database on steroids on a clustered system, built to scale, and providing tons of goodies.

Keep your options open

Having said that, each hyperscaler has a competing technology, e.g. redshift (AWS), Synapse (Microsoft), Big Query (GC), or even DataBricks. Snowflake has some advantages, among many technical ones one of these is that Snowflake is truly cloud agnostic, thus not pushing customers to a vendor lock in.

Customers use SNP Glue to integrate with a cloud-based data warehouse pick Snowflake because it comes as a true SAS solution, understands SQL like a database (but scales like Hadoop) and again returns data fast as a database.

Make your SAP data integration easier

With SNP Glue it’s possible to have a native integration from SAP to Snowflake. The obvious starting point is the technology integration with security and authentication. Then comes the “real” data integration which begins with schema creation: SNP Glue can analyze SAP data sources and create a corresponding data model on Snowflake.

Data replication can be either table driven (i.e. table based data replication with or without delta capture), or event driven (in which case you would use Snowpipe for data streaming).

What makes Snowpipe cool

Table driven data integration is the classical integration method is full table loads followed by delta capture (a.k.a. CDC) and near-real-time data replication. After the initial full load, rather small packages of data are transferred. This all is typically very much based on classical database tables rather than on business objects. Depending on frequency of data transfer (and possibly resulting very small packages) a near-real-time extraction and data integration can be achieved but would come at the price of frequent so-called delta merges on Snowflake.

Snowpipe enables the continuous loading of data into Snowflake tables as soon as new data arrives. This is particularly useful for scenarios where you need up-to-date information for analytics, reporting, or other applications. Snowpipe can be integrated with various tools and workflows, making it useful for real-time analytics, reporting, and other data-driven applications.

To close this blog article properly I would like to use the age old “one more thing” closing remark: the same team which is building SNP Glue to natively integrate SAP data and Snowflake is working on a native application on Snowflake using Snowflake’s application framework. The goal is to implement (and vastly improve) the data streaming including delta merge on Snowflake, i.e. the integration of updated records into the data warehouse. More information can be found in our press release: SNP Launches “Data Streaming for SAP”, a Snowflake Native App in the Data Cloud.