The Engineering Desk covers the data platforms teams run in production: Snowflake, Databricks, Matillion, cloud cost and FinOps, and developer tooling. It works from release notes, vendor documentation, and pricing pages.
The Data Today Engineering Desk is a research workflow that monitors primary sources and drafts analysis. Every piece is reviewed, edited, and signed off by Lars Cornelissen, responsible editor at Data Today. Corrections go to [email protected].
Sources this desk works from:
Responsible editor: Lars Cornelissen, responsible editor.
Contact: [email protected].
OpenAI Custom GPT retirement moves builders to a plugin system by December 11, 2026. Plugins bundle skills, reference files, and MCP tools, but migration drops conversations, Custom Actions, and sharing settings.
Any MCP agent can now query Databricks data through the GA Genie One MCP server, with Unity Catalog permissions enforced on every request.
Snowflake Code Bundles let you package and run Python and Spark on Snowflake compute without stored procedures. Here is what ships, what it costs, and when to use it.
VS Code AI agents dominate 6 of 10 open GitHub issues from Microsoft team members, pointing to multi-agent session management.
Secure server-side AI memory lets AI assistants remember across devices while keeping encryption keys on your phone. Google cannot read the data.
Isaac ROS 5.0 is NVIDIA's open-source robotics framework update that brings AI agents into ROS development and adds a CUDA buffer backend for zero-copy GPU payload sharing between nodes, now available for ROS 2 Lyrical.
GGUF inference in transformers now matches llama.cpp on Apple Silicon, hitting 103.6 tok/s with ggml Metal kernels while keeping you in PyTorch.
Generally available Lakebase Search adds pgvector-compatible vector search and BM25 to Lakebase Postgres at 1B vector scale with 32x compression.
LLM block pruning via Ising glass optimization preserves MMLU near 77 at 50% compression of Llama-3.3-70B-Instruct, beating baselines by 23 percentage points.
Admins can now set daily and monthly AI credit limits per person in Snowflake, with GA per-user quotas blocking overspend within minutes.
LLM watermarking with Google's SynthID-Text changes how models respond to harmful prompts and tool calls. New research shows refusal behavior shifts under prompt injection, with some models up to 12.5 points more likely to comply.
Seven external tools now connect to Genie One and Genie Code through GA Databricks MCP connectors, governed by Unity Gateway with per-user OAuth.
Enterprise AI agent pilots fail 89% of the time before production. Of 1,000 budgeted projects, 120 ship and 34 meet ROI. The blockers are data, evaluation, and operations.
GPT-6 Astra now handles Perplexity's communications, code changes, and production monitoring with automated verification gates replacing human approval gates. The architecture has three layers and one known failure mode.
LLM model routing promises cost savings by sending easy prompts to cheap models. A new study of four open-source routers finds a fixed mid-tier baseline matches the best router on every benchmark, with no statistically significant task-specific advantage detected.
A 2026 fault dropped 3.49 GW from the PJM grid in seconds, twice the 2024 event. NERC's top alert treats AI data center load as an architecture problem.
A runtime layer that learns how agents reuse context cuts TTFT by up to 45 percent and lifts throughput by up to 57 percent on multi-agent workloads.
Automatic change data feed (Auto CDF) GA shifts CDC from write time to query time. MERGE and UPDATE run 15% faster, but read costs move the other way.
Between-day variance reaches 8.4 points, three times the 2.8-point within-day noise: LLM performance drift silently regresses API models.
Ingesting live satellite data, Google WeatherNext 3 produces hourly 5km AI weather forecasts and cuts precipitation error by up to 60 percent.
Snowflake Postgres data mirroring continuously replicates Postgres tables into Snowflake using Iceberg with 30-second lag. It replaces external CDC connectors with serverless compute and open formats.
Meta's data center robotics tests target cable swaps and server resets. One trial could absorb 80% of some technician tasks, but major limitations remain.
Across 31,352 hourly scores, LLM benchmark variance swings 8.4 points between days and 2.8 within a day, so one run can mislead model choice.
On the 550-query ESQ-Bench Oracle NL2SQL benchmark, GPT-4o falls from 79.8 to 57.2 percent as schemas grow, and 92 percent of passes diverge.