Delta Lake
PulseAugur coverage of Delta Lake — every cluster mentioning Delta Lake across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
Databricks introduces native FILE type for multimodal data in lakehouse
Databricks has introduced a new beta feature called FILE type, designed to store unstructured data like documents, images, and videos as a native column within tables. This innovation aims to integrate multimodal data d…
-
Stop copying Databricks patterns into Microsoft Fabric, advises analysis
Data engineers are advised to avoid replicating complex Databricks architectural patterns within Microsoft Fabric. The article argues that Fabric's unified, serverless, and SaaS-based architecture rewards simplicity ove…
-
Databricks MCP server grants AI agents direct lakehouse access
Databricks has released a new tool, the Databricks MCP server, which allows AI agents like Claude to directly access and interact with a user's data lakehouse. This integration enables conversational execution of notebo…
-
Dow builds carbon ledger on Databricks to track product emissions
Dow has developed a Carbon Footprint Ledger (CFL) utilizing the Databricks Data Intelligence Platform to streamline the calculation of Product Carbon Footprints (PCFs) for its extensive product portfolio. This system, b…
-
Data Versioning Tools for ML and Data Lakes Reviewed
This article provides a practical overview of data versioning tools for machine learning and data lakes. It examines platforms such as Dolt, DVC, Delta Lake, Iceberg, Hudi, and lakeFS, offering insights into their capab…
-
Databricks enables external engine access to Unity Catalog managed tables
Databricks has announced that external engines can now create, read, and write to Unity Catalog managed Delta tables in public preview. This feature, built on open UC Delta APIs, allows engines like Apache Spark, Apache…
-
MLOps CI/CD and Feature Engineering on Azure, AWS, GCP · 2 sources tracked
This cluster explores the implementation of CI/CD pipelines within MLOps across major cloud platforms like Azure, AWS, and GCP. It highlights how MLOps differs from traditional DevOps, emphasizing the importance of feat…
-
Data Lakes vs. Cloud Data Warehouses: Choosing the Right Architecture
This guide compares data lake and cloud data warehouse architectures, highlighting their differences in data storage, query performance, governance, and cost. Data lakes excel at storing raw, multi-format data for machi…
-
Databricks outlines unified data pipeline architecture with Lakeflow
Databricks has detailed its approach to data pipeline architecture, emphasizing a unified platform that integrates batch and streaming data processing. The company highlights that effective architecture separates data i…
-
Databricks launches OpenSharing for AI agents, models, and data
Databricks has introduced OpenSharing, an evolution of its Delta Sharing protocol designed for the agentic AI era. This new open-source protocol, now hosted by the Linux Foundation, expands beyond data sharing to encomp…
-
Text-to-SQL LLM risks: Data leaks and cost overruns
The notion that Text-to-SQL is a solved problem is a dangerous myth, as LLMs can generate non-deterministic SQL queries that pose risks to sensitive data. Approaches like feeding the entire schema to the LLM or using se…
-
Databricks enables unified data access control across engines
Databricks has introduced a beta version of its Cross-Engine ABAC feature, allowing attribute-based access controls to be defined once in Unity Catalog and enforced across various external data engines. This new capabil…
-
AI workloads demand new data architecture layer
The traditional data stack is insufficient for modern AI workloads, which require handling unstructured data, real-time embeddings, and robust lineage tracking. A new 'Platinum' or AI-native layer is proposed, extending…
-
Databricks launches analytics engineer learning pathway for SQL pros
Databricks has launched a new learning pathway designed for SQL practitioners to become analytics engineers. This curriculum focuses on transforming raw data into governed, AI-ready semantic models and metric views with…
-
Databricks launches AI sales tool for prescriptive actions
Databricks has developed PipelineIQ, an AI-powered sales intelligence tool designed to move beyond traditional forecasting. Instead of relying on retrospective data, PipelineIQ analyzes messy CRM information to provide …
-
Databricks enables external engines to write to Unity Catalog tables
Databricks has introduced a beta feature allowing external engines like Apache Spark, Flink, and DuckDB to create, read, and write to Unity Catalog managed Delta tables. This expansion builds on the open APIs for Unity …
-
Unified Revenue Platform architecture tackles sales analytics fragmentation
This article proposes a four-layer architecture for a Unified Revenue Platform (URP) to address the fragmentation of sales analytics in B2B organizations. The proposed system integrates CRM data ingestion, target cascad…
-
Databricks MCP lets AI agents directly query Lakehouse data
Databricks has released an integration called MCP that allows AI agents like Claude and Cursor to directly access and interact with data stored in Databricks Lakehouse. This tool enables AI models to query Delta tables,…
-
Databricks Catalog Commits GA unifies lakehouse data coordination
Databricks has announced the general availability of Catalog Commits for Unity Catalog managed tables, a significant platform upgrade. This feature aims to unify the lakehouse by aligning Delta Lake with Iceberg's catal…
-
SAP acquires Dremio and Prior Labs to bolster its AI-ready data platform
SAP has acquired Dremio, a data lakehouse provider, to enhance its data analytics and AI capabilities. This move aims to unify enterprise data, both internal and external, to power AI agents at scale. The acquisition em…