Snowflake
Snowflake Code Bundles: Python without stored procedures
Snowflake Code Bundles let you package and run Python and Spark on Snowflake compute without stored procedures. Here is what ships, what it costs, and when to use it.
#pipelines
11 stories tagged pipelines.
Snowflake
Snowflake Postgres data mirroring explained
Snowflake Postgres data mirroring continuously replicates Postgres tables into Snowflake using Iceberg with 30-second lag. It replaces external CDC connectors with serverless compute and open formats.
Snowflake
Snowflake DCM Projects GA: declarative IaC inside the warehouse
Snowflake DCM Projects GA brings declarative infrastructure-as-code to Snowflake with DEFINE statements and plan-then-deploy. Eight capabilities remain in Preview, including masking policies and streams.
Snowflake
Dynamic Iceberg table replication, costed honestly
Dynamic Iceberg table replication now keeps Snowflake managed Iceberg pipelines in DR plans, but refresh and transfer costs move to the target account.
Snowflake
Snowflake Dynamic Tables refresh gets a cost test
Snowflake Dynamic Tables now claim up to 2.8x faster refresh on Gen2 warehouses, but the cost win depends on target lag and change volume.
Snowflake
Snowflake Iceberg ADLS writes are finally usable
Azure teams can now read and write externally managed Iceberg tables in Snowflake without moving storage, through GA ADLS support.
Snowflake
ICEBERG_MERGE_ON_READ_BEHAVIOR changes Iceberg DML
Set to AUTO, Snowflake's GA ICEBERG_MERGE_ON_READ_BEHAVIOR switch sends 3 of 4 Iceberg table cases to merge-on-read.
Matillion
Matillion's Data Productivity Cloud, explained for builders
Matillion's Data Productivity Cloud builds and runs pipelines with low-code, code and AI. How the pieces fit and where the cost goes.
Snowflake
Should you put your data lake in Snowflake Iceberg tables?
Snowflake Iceberg tables store data in open Apache Iceberg format in your own cloud storage, so Snowflake bills zero storage and other engines can read it. The catch: only Snowflake-managed catalog tables get full platform support, external-catalog tables lose clustering, cloning, and replication.
Snowflake
Snowflake Openflow: managed ingestion built on Apache NiFi
Built on Apache NiFi, Snowflake Openflow moves data from any source into Snowflake, running inside Snowflake or in your own VPC, each billed differently.
Snowflake
Dynamic Tables vs Streams and Tasks: which Snowflake pipeline to build
Dynamic Tables let you declare a SELECT and a target lag and let Snowflake refresh it; Streams and Tasks give you imperative control. Dynamic Tables bottom out at a 60-second minimum lag, so sub-minute freshness still means Snowpipe Streaming or Tasks.