@Rocky I encounter this error Cannot invoke "com.starrocks.catalog.Database.getId()" because "database" is null when running insert into files using dbt starrocks

The error

Cannot invoke "com.starrocks.catalog.Database.getId()" because "database" is null

typically occurs when a SQL command is executed in a session that does not have a current database context (i.e., no

USE <database>

has been executed). Even though

INSERT INTO FILES

unloads data to an external location (like S3 or HDFS) and might not seem to require a local database, StarRocks still needs a database context to manage metadata such as load labels, transaction IDs, or permissions. Common Causes and Solutions 1. Missing Database Context in the Session If you are running the

INSERT INTO FILES

command via a

dbt

macro,

post_hook

, or a custom materialization, the connection might not have a default database set. Solution: Explicitly set the database at the start of your SQL block:

sql
USE your_database_name;
INSERT INTO FILES(
    "path" = "<s3://your-bucket/path/>",
    "format" = "parquet",
    ...
)
SELECT * FROM your_table;

2. StarRocks Version Compatibility

INSERT INTO FILES

for unloading/exporting data was officially introduced in StarRocks v3.2. • If you are on a version earlier than 3.2.0, this syntax may not be fully supported or may contain bugs related to session context. • If you are on an early 3.2.x version, consider upgrading to a more recent patch release (e.g., 3.2.7+), as there were several fixes for

INSERT INTO FILES

metadata handling. 3. dbt-starrocks Configuration Ensure your

profiles.yml

includes the

schema

(which corresponds to the StarRocks database name). dbt usually attempts to run

USE <schema>

upon connecting, but some execution paths (like

run-operation

) might bypass this if not configured correctly. Check your `profiles.yml`:

yaml
your_profile:
  target: dev
  outputs:
    dev:
      type: starrocks
      host: <fe_host>
      port: 9030
      schema: your_database  # Ensure this is present
      username: <user>
      password: <password>

4. Fully Qualified Table Names Even if you use fully qualified names in your

SELECT

(e.g.,

SELECT * FROM catalog.db.table

), the

INSERT

part of the statement might still look for the session's current database to generate a label. Always ensure a database is selected for the session. Summary Checklist • ☐ Does your SQL script start with

USE <database_name>;

? • ☐ Is your StarRocks version 3.2.0 or higher? • ☐ Is the

schema

defined in your dbt profile? • ☐ If you are using a

WITH LABEL

clause, try removing it or ensuring the database context is set, as labels are often scoped to a database. References • sql-reference/sql-statements/loading_unloading/INSERT.md • loading/InsertInto.md • unloading/unload_using_insert_into_files.md • sql-reference/sql-functions/table-functions/files.md • data_source/file_external_table.md

@Rocky I encounter this error Cannot invoke "com.starrocks.catalog.Database.getId()" because "database" is null when running insert into files using dbt starrocks test with cli client it do success , but when run dbt-starrocks, the flow is: begin; insert into files ... commit; these failed with error Cannot invoke "com.starrocks.catalog.Database.getId()" because "database" is null

The error

Cannot invoke "com.starrocks.catalog.Database.getId()" because "database" is null

occurs because you are attempting to run an

INSERT INTO FILES()

(unload) operation inside an explicit SQL transaction block (

BEGIN; ... COMMIT;

). Why this happens 1. Transaction Scope: StarRocks' SQL transactions (introduced in v3.5) are designed for atomic DML operations (

INSERT INTO <table_name>

,

UPDATE

,

DELETE

) targeting internal database tables. 2. Missing Database Context: The

INSERT INTO FILES()

statement is used to unload data to remote storage (like S3 or HDFS). Unlike a standard

INSERT

, it does not have a target table or database in the StarRocks catalog. 3. The Error: When wrapped in a

BEGIN...COMMIT

block, the StarRocks transaction manager attempts to resolve the "target database" to manage locks and ACID properties. Since

FILES()

does not belong to a database, the internal

database

object is null, leading to the Java

NullPointerException

you see. Solution

INSERT INTO FILES()

must be executed as a single, independent statement outside of an explicit transaction block. If you are using dbt-starrocks:

dbt-starrocks

often wraps operations in transactions by default to ensure atomicity. To fix this, you need to ensure the statement is executed without a transaction. 1. For Hooks (Pre-hook/Post-hook): If you are running this in a hook, ensure the adapter is not forcing a transaction.

yaml
    # In your dbt_project.yml or model config
    post-hook:
      • sql: "INSERT INTO FILES(...)"
        transaction: false

2. For Macros/Operations: If you are calling this via a macro or

dbt run-operation

, ensure you are using a connection that doesn't start a transaction, or execute it as a raw statement. 3. Check dbt Materialization: If you have written a custom materialization that uses

INSERT INTO FILES

, you must modify the template to remove the

BEGIN

and

COMMIT

statements surrounding this specific command. Summary of Restrictions * Transactions (

BEGIN...COMMIT

) are supported for:

INSERT INTO <internal_table>

,

UPDATE

, and

DELETE

. * Transactions are NOT supported for:

INSERT INTO FILES()

(unloading),

INSERT OVERWRITE

, or cross-database DML. Since your CLI test succeeded (likely because it was run as a single statement without

BEGIN

), moving the command out of the transaction block in dbt will resolve the issue. References • loading/SQL_transaction.md • integrations/dbt.md