@Rocky are there any plans to have the ability to create in starRocks iceberg format tables ?

Yes, this feature is already available. Since StarRocks v3.1, you can create Apache Iceberg tables directly from StarRocks using an Iceberg Catalog. How it works To create Iceberg tables, you first need to set up an Iceberg Catalog that connects to your metastore (Hive Metastore, AWS Glue, Tabular, etc.). Once the catalog is configured, you can perform DDL operations just like you would with native StarRocks tables. Example: Create an Iceberg Table

sql
-- 1. Create the catalog (if not already done)
CREATE EXTERNAL CATALOG iceberg_catalog
PROPERTIES (
    "type" = "iceberg",
    "iceberg.catalog.type" = "hive",
    "iceberg.catalog.hive.metastore.uris" = "thrift://<metastore_ip>:9083"
);

-- 2. Create a table within that catalog
CREATE TABLE iceberg_catalog.iceberg_db.new_table (
    id INT,
    name STRING,
    event_date DATE
)
PARTITION BY (event_date);

Key Features Supported: * CTAS (Create Table As Select): You can create an Iceberg table and populate it with data from StarRocks or other catalogs in one step:

sql
    CREATE TABLE iceberg_catalog.db.target_tbl
    AS SELECT * FROM default_catalog.local_db.source_tbl;

* Partitioning: Supports Iceberg partition transforms like

identity

,

year()

,

month()

,

day()

,

hour()

, and

bucket()

. * Data Sinking: You can use

INSERT INTO

to write data into these Iceberg tables. * Time Travel: From v3.4 onwards, StarRocks supports Iceberg snapshot management (branching and tagging) and time-travel queries. Roadmap & Future Plans While basic creation and insertion are supported, StarRocks is continuously adding more advanced Iceberg features: * v4.0+: Added support for

ORDER BY

(sort keys) in Iceberg tables to improve query performance. * Upcoming (2025-2026): Roadmaps indicate plans for full Iceberg v3 support, including enhanced

MERGE INTO

,

UPDATE

, and

DELETE

operations to make the Iceberg experience even more "native." For more details on syntax and properties, you can refer to the Iceberg DDL documentation. References • data_source/catalog/iceberg/DDL.md • data_source/External_table.md • data_source/catalog/iceberg/iceberg_catalog.md

@Rocky I mean starRocks is my data lake and the iceberg tables are in starRocks

It sounds like you want StarRocks to manage your data lake directly using the Iceberg format, rather than just querying external data. To clarify, StarRocks currently separates tables into two categories: Internal (using StarRocks' own highly optimized segment format) and External (using open formats like Iceberg). However, the "plan" you are looking for is already being realized through the StarRocks Lakehouse architecture. Here is how you can use StarRocks as your "Iceberg Data Lake": 1. The "Managed Iceberg" Approach (Available Now) While the

default_catalog

(Internal) still uses StarRocks' proprietary format for maximum performance, you can use StarRocks to manage the entire lifecycle of Iceberg tables on your own storage (S3, MinIO, HDFS). By creating an Iceberg Catalog, StarRocks acts as the owner of those tables. You can: * Create databases and tables directly from StarRocks. * Write/Ingest data using

INSERT INTO

or

ROUTINE LOAD

. * Manage the data: StarRocks (starting in v3.4 and v4.0) provides background compaction services and snapshot management for Iceberg, which were previously "manual" tasks in a data lake. 2. StarRocks 4.0 & 4.1: Iceberg as a "First-Class Citizen" The most recent updates (v4.0 and the v4.1 roadmap) move StarRocks closer to your goal of having Iceberg tables "in" StarRocks: * Native Support: StarRocks is moving away from "just a compute engine" toward an "Iceberg-native Warehouse." It now integrates Iceberg-specific types (like the Variant type for semi-structured data) directly into its execution engine. * Automatic Maintenance: StarRocks 4.0 introduced an asynchronous compaction service for Iceberg. This means StarRocks automatically cleans up small files and optimizes your Iceberg tables in the background, just like it does for its internal tables. * Unified Catalog: From v3.2+, the Unified Catalog allows you to manage Hive and Iceberg tables within a single namespace, making the distinction between "Internal" and "External" much thinner. Summary: Do you need to wait? * If you want the

default_catalog

to be Iceberg: There is currently no official plan to replace the internal StarRocks storage format with Iceberg for the

default_catalog

, because the StarRocks format is significantly faster for sub-second OLAP. * If you want StarRocks to be your Data Lake Manager: You can do this today. You simply create an Iceberg catalog (pointing to your S3/MinIO bucket). All tables you create there are "in" StarRocks in the sense that StarRocks manages their metadata, writes their data, and optimizes their performance. Recommendation: If you want an "Iceberg-first" experience where StarRocks feels like the primary home for your data, use a StarRocks-managed Iceberg Catalog on top of object storage. This gives you the open format you want with the management features of a warehouse. References • data_source/catalog/iceberg/iceberg_catalog.md • data_source/catalog/unified_catalog.md • quick_start/iceberg.md References • data_source/catalog/iceberg/iceberg_catalog.md • data_source/catalog/unified_catalog.md

@Rocky right now the native tables in starRocks are primary key modal table, duplicate key table and so on. the question is will there be a native iceberg table in starRocks (not s3 or some other external storage)

Yes, this capability is a central part of the StarRocks 4.0 "Native Lakehouse" vision. While StarRocks maintains its own internal storage format (Primary Key, Duplicate Key, etc.) for sub-second OLAP performance, it has evolved to treat Iceberg as a "first-class citizen." 1. Managed Iceberg (Already Available) Since v3.1, you can create, drop, and write to Iceberg tables directly from StarRocks using an Iceberg Catalog. In StarRocks 4.0, this experience has been deepened to make Iceberg tables behave like "native" tables: * Background Maintenance: StarRocks now provides an asynchronous compaction service for Iceberg tables, automatically handling small file issues and optimizing table health (just like it does for internal tables). * Native Writes: You can use

INSERT INTO

or

CTAS

with native write optimizations, hidden partitioning support, and automatic statistics collection. * Catalog-Centric Governance: With integration into catalogs like Apache Polaris (REST), StarRocks acts as the manager of the Iceberg data lake, providing centralized access control and credential vending. 2. Internal vs. Iceberg: The Storage Difference To your point about tables being "in" StarRocks (and not on S3): * Proprietary Format (default_catalog): Tables like Primary Key or Duplicate Key use StarRocks' own optimized segment format. This format is deeply integrated with the execution engine to provide 3–5x better performance than open formats. These can be stored on local disks (Shared-Nothing) or Object Storage (Shared-Data). * Iceberg Format: These tables must reside on a file system that the Iceberg catalog can talk to. While this is usually S3/Object Storage, you can also use HDFS or an S3-compatible local storage (like MinIO). Is there a plan for "Native Iceberg" on local disks? Currently, there is no official plan to replace StarRocks' proprietary internal format with Iceberg for the

default_catalog

. The reason is purely performance: StarRocks' internal format supports fine-grained indexing, delete vectors, and SIMD-optimized layouts that are not yet possible within the current Iceberg specification. The Strategy: Instead of making the internal format Iceberg, StarRocks is making the Iceberg experience "Native." * If you need maximum speed (high concurrency, sub-second latency): Use internal tables. * If you need openness and interoperability (so Spark, Flink, and Trino can also read the data): Use a StarRocks-managed Iceberg catalog. In StarRocks 4.0 and beyond, you can manage both from the same SQL interface, and StarRocks will handle the "heavy lifting" (compaction, indexing, metadata) for both. References • release_notes/release-4.0.md • data_source/catalog/iceberg/iceberg_catalog.md • quick_start/iceberg.md References • data_source/catalog/iceberg/iceberg_catalog.md • data_source/catalog/iceberg/iceberg_rest_s3.md