Apache Iceberg
PulseAugur coverage of Apache Iceberg — every cluster mentioning Apache Iceberg across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
Data lake enables hybrid product search, cutting catalog copies
Researchers have developed a method to enable hybrid product search directly from data lake tables, eliminating the need for separate copies of e-commerce catalogs. This approach leverages Apache Pinot, an OLAP engine, …
-
Databricks evolves Delta Sharing to OpenSharing for AI assets
Databricks has evolved its Delta Sharing protocol into OpenSharing, a new Linux Foundation-governed standard designed for sharing a wider array of AI assets beyond just data tables. This expansion includes agent skills,…
-
Enterprise GenAI Platform Architecture for On-Premises Deployment
This article outlines a reference architecture for building an enterprise-grade Generative AI platform that operates entirely on-premises. It details the necessary soft artifacts, such as MLOps modules, model serving, s…
-
Databricks Unity Catalog advances open lakehouse with enhanced data portability
Databricks has enhanced its Unity Catalog to promote greater openness in data lakehouses. The catalog now supports the Apache Iceberg REST catalog specification for registering and unregistering tables, enabling data po…
-
Databricks enables multi-statement transactions for Delta Lake and Apache Iceberg
Databricks has announced the general availability of multi-statement transactions for both Delta Lake and Apache Iceberg. This feature enables organizations to run complex, multi-step workflows with atomic guarantees di…
-
Databricks simplifies dbt ETL pipeline migration with new guide
Databricks has released a guide detailing how users can migrate their dbt ETL pipelines to the Databricks Lakehouse platform with minimal code changes. The process involves updating the dbt adapter and configuration, ad…
-
Databricks introduces APIs to prevent catalog lock-in for lakehouses
Databricks has introduced new REGISTER and UNREGISTER APIs to enhance data portability within lakehouse architectures. These APIs allow tables to be moved between different catalogs without duplicating data or metadata,…
-
Databricks enhances Apache Iceberg governance with read restrictions
Databricks has introduced new features for Apache Iceberg, focusing on enhancing data governance across various engines. The updates include read restrictions and catalog labels, which aim to make governance policies mo…
-
Databricks enhances Open Lakehouse governance with Apache Iceberg specs
Databricks has introduced new specifications for Apache Iceberg, focusing on read restrictions and catalog labels to enhance governance within the Open Lakehouse architecture. These advancements aim to make data governa…
-
Open Table Formats: Iceberg, Delta Lake, and Hudi Explained
Open table formats like Apache Iceberg, Delta Lake, and Apache Hudi are crucial for data lakes, enabling ACID transactions, schema evolution, and time travel on data stored in object storage. These formats transform col…
-
Data Clustering and Z-Ordering Optimize Table Scans by Enabling Data Skipping
A technical article explains Z-ordering and data clustering, techniques used to optimize data retrieval in large tables, particularly within systems like Delta Lake. It highlights how traditional partitioning can be ins…
-
AI predicts data compaction utility in lakehouse tables
Researchers have developed a method to predict the utility of data compaction in lakehouse tables using metadata. By extracting 17 features from manifest files and training an XGBoost model, they achieved high accuracy …
-
New Lakehouse Method Enhances Vector Search with File Pruning
Researchers have developed a novel method for integrating approximate nearest-neighbor (ANN) search with structured data filtering within an open lakehouse table format, specifically Apache Iceberg over Parquet. This ap…
-
Databricks enables external engine access to Unity Catalog managed tables
Databricks has announced that external engines can now create, read, and write to Unity Catalog managed Delta tables in public preview. This feature, built on open UC Delta APIs, allows engines like Apache Spark, Apache…
-
Couchbase launches AI Data Plane for production AI agents
Couchbase has introduced its AI Data Plane, designed to serve as the operational data layer for AI agents in production environments. This new offering includes features such as persistent Agent Memory, an Agent Catalog…
-
pgEdge launches ColdFront for queryable PostgreSQL data in S3
pgEdge has launched ColdFront, a new tool designed to move PostgreSQL data to S3 while maintaining queryability and updatability. This solution aims to address the challenge of managing large datasets by allowing them t…
-
Databricks launches OpenSharing for AI agents, models, and data
Databricks has introduced OpenSharing, an evolution of its Delta Sharing protocol designed for the agentic AI era. This new open-source protocol, now hosted by the Linux Foundation, expands beyond data sharing to encomp…
-
Databricks enables unified data access control across engines
Databricks has introduced a beta version of its Cross-Engine ABAC feature, allowing attribute-based access controls to be defined once in Unity Catalog and enforced across various external data engines. This new capabil…
-
Databricks Unity Catalog bolsters Apache Iceberg support with new features
Databricks has enhanced its Unity Catalog to provide comprehensive support for Apache Iceberg, a popular open table format. Key updates include General Availability for Managed Iceberg, Iceberg v3, and Foreign Iceberg, …
-
AI workloads demand new data architecture layer
The traditional data stack is insufficient for modern AI workloads, which require handling unstructured data, real-time embeddings, and robust lineage tracking. A new 'Platinum' or AI-native layer is proposed, extending…