Hey <@U0A71G31CDV>, While doing a database backup...
# questions-and-troubleshooting
f
Hey @Rocky, While doing a database backup to GCS, I'm getting
read: Is a directory
on
TaskErrMsg
of the
SHOW BACKUP;
command. What could this be?
r
The error
read: Is a directory
(often a translation of the POSIX
EISDIR
error) typically occurs when StarRocks or the underlying storage client (S3A) expects to read a file but instead encounters a directory (or a prefix that behaves like one) with the same name. In the context of a GCS backup, this is usually caused by one of the following: 1. Incorrect URI Prefix or Protocol StarRocks does not support the
gs://
protocol for the
BACKUP
and
RESTORE
commands. You must use the S3A protocol to interact with GCS for repositories. * Check: Ensure your
CREATE REPOSITORY
statement uses
s3a://
instead of
gs://
. * Example of correct setup:
Copy code
sql
    CREATE REPOSITORY gcs_repo
    WITH BROKER -- or without if using built-in S3 client
    ON LOCATION "<s3a://your-bucket/backup-folder>"
    PROPERTIES(
        "fs.s3a.access.key" = "xxxx",
        "fs.s3a.secret.key" = "yyyy",
        "fs.s3a.endpoint" = "<http://storage.googleapis.com|storage.googleapis.com>"
    );
Note: Do not include
https://
in the endpoint property.
2. Path Name Collision in GCS If you are performing a backup with a specific snapshot name (e.g.,
BACKUP SNAPSHOT snapshot_v1 ...
), StarRocks creates a directory-like prefix in GCS with that name. * The Issue: If a directory or object already exists in GCS that conflicts with the file structure StarRocks is trying to create (for example, if you manually created a folder named
__meta
or a tablet ID folder inside the snapshot path), the S3A client may throw this error. * Fix: Try using a completely fresh, empty prefix/folder in your GCS bucket for the repository, or choose a new snapshot name that has never been used in that bucket before. 3. Trailing Slashes in Configuration In some versions, a missing or extra trailing slash in the
ON LOCATION
path or in the GCS endpoint configuration can cause the S3A client to incorrectly resolve file paths as directory paths. * Check: Verify the
ON LOCATION
path. It is generally safer to specify a path like
<s3a://bucket/folder>
(without a final trailing slash) and let StarRocks handle the sub-pathing. 4. Broker-related Issues If you are using a Broker (the
WITH BROKER
clause), the error is actually coming from the Broker process. * Check: Look at the
apache_hdfs_broker.log
on the server where the Broker is running. It will provide a full Java stack trace for the
read: Is a directory
error, which will pinpoint exactly which path is causing the conflict. Recommended Next Steps: 1. Run
SHOW REPOSITORIES;
to verify the
Location
and
Properties
are correct (especially the
s3a://
prefix). 2. Check GCS to see if the snapshot folder already exists. If it does, delete it or use a different name for your
BACKUP
command. 3. If the issue persists, check the BE (Backend) logs (
<http://be.INFO|be.INFO>
or
be.WARNING
) for more detailed context on which specific file/tablet the system was trying to read when it failed. Referencessql-reference/sql-statements/backup_restore/SHOW_REPOSITORIES.mdadministration/management/Backup_and_restore.mdsql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdadministration/stargo.mdadministration/management/monitoring/metrics.mddata_source/feature-support-data-lake-analytics.mdfaq/Deploy_faq.md
f
@Rocky what about these docs - https://docs.starrocks.io/docs/4.0/integrations/authenticate_to_gcs/#vm-based-authentication they state VM authenticated with GCS and use the gcs: prefix
r
The documentation you found covers several GCS integration scenarios, but there is a critical distinction between data loading/querying and the backup/restore system. Why you see
read: Is a directory
While StarRocks supports the
gs://
protocol for loading data (like
INSERT INTO FILES
) and external catalogs, the Backup and Restore (Repository) system specifically relies on the S3A protocol when interacting with GCS. If you use
gs://
in your
CREATE REPOSITORY
statement, the underlying driver (which expects S3-compatible behavior for backups) may misinterpret the URI or fail to map the path correctly, leading to the
read: Is a directory
error. This error occurs when the system expects to find a specific metadata file but instead finds a GCS prefix (folder) with that name. Correct Configuration for GCS Backup To back up to GCS, you must use the S3A protocol and HMAC keys (Access Key and Secret Key) from GCS, rather than the VM-based authentication parameters. 1. Protocol: Use
s3a://
instead of
gs://
. 2. Endpoint: Set to
<http://storage.googleapis.com|storage.googleapis.com>
. 3. Credentials: Use GCS HMAC keys. Example of a working Repository for GCS:
Copy code
sql
CREATE REPOSITORY gcs_repo
WITH BROKER -- or without if not using a broker
ON LOCATION "<s3a://your-gcs-bucket/backup_folder>"
PROPERTIES(
    "fs.s3a.access.key" = "GOOGXXXXXXXXXXXXXX",
    "fs.s3a.secret.key" = "YYYYYYYYYYYYYYYYYY",
    "fs.s3a.endpoint" = "<http://storage.googleapis.com|storage.googleapis.com>"
);
Summary of Differences Feature | Supported Protocol | Authentication Method Data Load / Catalogs |
gs://
| VM-based, Service Account, or HMAC Backup / Restore |
s3a://
(Required) | HMAC Keys Only (
fs.s3a.*
properties) Note on VM Authentication: Currently, the
BACKUP
and
RESTORE
commands in StarRocks do not support the native
gcp.gcs.use_compute_engine_service_account
parameter. They require explicit S3-style credentials to communicate with GCS via its S3-interoperability layer. Reference: According to the CREATE REPOSITORY docs:
"StarRocks supports creating repositories in Google GCS only according to the S3A protocol. Therefore... you must replace the prefix in the GCS URI... with
s3a://
."
Referencesadministration/management/FE_configuration.mdsql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdintegrations/authenticate_to_gcs.mddata_source/catalog/unified_catalog.mddata_source/catalog/hive_catalog.md
f
@Rocky those docs specifically say: StarRocks can authenticate to GCS in the following scenarios: • ... • Back up data from and restore data to GCS. • ....
r
The documentation you're referencing lists "Back up data from and restore data to GCS" as a supported scenario for GCS authentication, but there is a nuance in the protocol and syntax required for the
BACKUP
and
RESTORE
commands specifically. While StarRocks supports the
gs://
protocol and VM-based authentication (using
gcp.gcs.use_compute_engine_service_account
) for data loading (like
INSERT INTO FILES
) and external catalogs, the `BACKUP`/`RESTORE` system is built on a different storage interface. 1. The Protocol Conflict The
CREATE REPOSITORY
documentation (and the "Backup and Restore" guide) explicitly states:
"StarRocks supports creating repositories in Google GCS only according to the S3A protocol. Therefore... you must replace the prefix in the GCS URI... with
s3a://
."
If you used the
gs://
prefix in your
CREATE REPOSITORY
statement: * The error
read: Is a directory
occurs because the storage driver (often the Hadoop GCS connector or the StarRocks FE) is attempting to read a specific metadata file (like
__meta
) to verify the repository, but GCS returns a "directory" (prefix) object instead, or the driver misinterprets the path structure. _ The
gcp.gcs._
properties (like VM-based auth) are typically not recognized by the
BACKUP
command's underlying S3A-based logic. 2. How to fix it for GCS Backup To resolve the
read: Is a directory
error and successfully back up to GCS, you should align with the specific requirements for Repositories: 1. Use the S3A protocol: Your location must start with
s3a://
. 2. Use HMAC Keys: Instead of VM-based service accounts, you must provide GCS HMAC keys (Access Key/Secret Key) which allow GCS to look like S3. 3. Correct Endpoint: Set the endpoint to
<http://storage.googleapis.com|storage.googleapis.com>
. Corrected Syntax:
Copy code
sql
CREATE REPOSITORY gcs_repo
WITH BROKER -- or without if not using a broker
ON LOCATION "<s3a://your-bucket-name/backup_path>"
PROPERTIES(
    "fs.s3a.access.key" = "GOOGXXXXXXXXXXXXXX",
    "fs.s3a.secret.key" = "YYYYYYYYYYYYYYYYYY",
    "fs.s3a.endpoint" = "<http://storage.googleapis.com|storage.googleapis.com>"
);
3. Why does the other doc say it's supported? The "Authenticate to GCS" page is an overview of StarRocks' general ability to talk to GCS. However, the
BACKUP
and
RESTORE
feature is a specialized system that still requires the S3-compatibility layer of GCS. If you must use VM-based authentication and cannot use HMAC keys, you may be unable to use the native
BACKUP
command to GCS at this time. An alternative would be to export data using
INSERT INTO FILES
(which does support the
gs://
protocol and VM-based auth) and manually manage the "backup" of your data. Recommendation: * Double-check your
SHOW REPOSITORIES;
output. If the
URL
starts with
gs://
, you should
DROP REPOSITORY
and recreate it using the
s3a://
protocol and HMAC credentials as shown above. * Ensure the
backup_path
in GCS is empty or does not contain a file/folder conflict with the snapshot name you are using. Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdadministration/management/Backup_and_restore.mdintegrations/authenticate_to_gcs.mdsql-reference/sql-statements/backup_restore/SHOW_REPOSITORIES.md Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdadministration/management/Backup_and_restore.md
f
@Rocky but can you use VM authentication with s3a?
r
The short answer is no, you generally cannot use VM-based authentication (Compute Engine Service Account) with the
s3a://
protocol for StarRocks Backups. While the general GCS documentation you found mentions VM-based authentication for "Back up data," there is a strict technical limitation in the current implementation of the
BACKUP
and
RESTORE
commands (the Repository system): 1. The Protocol Conflict * The Repository System: StarRocks'
BACKUP
and
RESTORE
logic for GCS is built on the S3A protocol (
s3a://
). This protocol communicates with GCS via its S3-compatibility layer. * Authentication Requirement: GCS's S3-compatibility layer requires HMAC keys (Access Key and Secret Key). It does not natively recognize Google Cloud IAM Instance Metadata or VM-based service account tokens in the same way the
gs://
protocol does. * VM-based Auth Limitation: The property
"gcp.gcs.use_compute_engine_service_account" = "true"
is specifically designed for the
gs://
protocol
, which is supported for data loading (Broker Load,
INSERT INTO FILES
) and External Catalogs, but not for the Repository system used in backups. 2. Why you get
read: Is a directory
This error is almost always a result of a protocol or path mismatch: * If you used the
gs://
prefix in your
CREATE REPOSITORY
statement, the system may fail to resolve the metadata files correctly because the
BACKUP
command expects an S3-compliant file structure. * If you used
s3a://
but tried to provide VM-auth properties instead of HMAC keys, the S3A client may be failing to authenticate and misinterpreting the bucket's response as a directory-level error. How to Fix It To perform a backup to GCS, you must configure the repository to use GCS's S3-interoperability mode: 1. Generate HMAC Keys: In your Google Cloud Console, go to Cloud Storage > Settings > Interoperability and create an HMAC key for your service account. 2. Recreate the Repository:
Copy code
sql
    CREATE REPOSITORY gcs_repo
    WITH BROKER -- Use BROKER if your version/setup requires it
    ON LOCATION "<s3a://your-bucket-name/backup_path>"
    PROPERTIES(
        "fs.s3a.access.key" = "GOOGXXXXXXXXXXXXXX",
        "fs.s3a.secret.key" = "YYYYYYYYYYYYYYYYYY",
        "fs.s3a.endpoint" = "<http://storage.googleapis.com|storage.googleapis.com>"
    );
_Note: Ensure
backup_path
is a clean, empty prefix._ Summary * Data Load/Catalogs: Support
gs://
and VM-based authentication. * Backup/Restore: Supports only
s3a://
and requires HMAC Keys. If your security policy strictly forbids HMAC keys, the standard
BACKUP
command cannot be used with GCS. In that case, you would need to "backup" data by exporting it using
INSERT INTO FILES
(which supports
gs://
and VM-auth), though this does not capture StarRocks-specific metadata like the
BACKUP
command does. Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdintegrations/authenticate_to_gcs.mdadministration/management/Backup_and_restore.mdsql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.mdsql-reference/sql-statements/backup_restore/SHOW_REPOSITORIES.md Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdintegrations/authenticate_to_gcs.md
f
@Rocky then why does that doc I linked state that specifically? respond shortly and direct
r
The documentation you linked is an overview of GCS authentication, but it contains a technical contradiction when compared to the specific implementation of the Backup system. Here is why you are seeing that error: 1. Protocol Mismatch: While that doc lists "Backup" as a scenario, the actual `CREATE REPOSITORY` command (the core of the Backup system) only supports GCS via the S3A protocol (
s3a://
). 2. Auth Mismatch: VM-based authentication (
gcp.gcs.use_compute_engine_service_account
) only works for the native
gs://
protocol
, which is used for data loading and catalogs. The S3A protocol used for backups does not recognize those
gcp.gcs.*
parameters; it only recognizes HMAC keys (
fs.s3a.access.key
). 3. The Error: When you use
gs://
with VM authentication for a repository, the Backup system fails to correctly parse the file structure (expecting S3-style metadata), which results in the
read: Is a directory
error. Bottom line: Despite what that overview page implies, the StarRocks Backup/Restore system currently requires HMAC keys and the
s3a://
prefix. VM-based authentication is not yet supported for the
CREATE REPOSITORY
command. Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdintegrations/authenticate_to_gcs.mdadministration/management/Backup_and_restore.mdsql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.mdsql-reference/sql-statements/backup_restore/SHOW_REPOSITORIES.md Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdadministration/management/Backup_and_restore.md
f
@Rocky where are you getting that certainty from?
r
The "certainty" comes from a technical conflict between the general overview page you are reading and the specific command documentation for
CREATE REPOSITORY
. Here is why the documentation appears to contradict itself and why the
s3a://
+ HMAC method is currently the only one that works for backups: 1. The Overview vs. The Implementation The page "Authenticate to GCS" is a high-level summary of StarRocks' ability to connect to GCS. It lists "Back up and restore" as a scenario because, functionally, StarRocks can do it. However, the documentation for the actual command used to do this—*`CREATE REPOSITORY`*—imposes a strict protocol limitation:
*"StarRocks supports creating repositories in Google GCS only according to the S3A protocol. Therefore... you must replace the prefix in the GCS URI... with
s3a://
."*
2. The Protocol Gap There are two different "drivers" inside StarRocks that talk to GCS: * The Native GCS Client (
gs://
):
This supports VM-based authentication (
gcp.gcs.use_compute_engine_service_account
). It is used for Data Loading (Broker Load,
INSERT INTO FILES
) and External Catalogs. * The S3A Client (
s3a://
):
This is a Hadoop-based compatibility layer used specifically by the Backup/Restore (Repository) system. This client does not recognize
gcp.gcs.*
properties. It only understands S3-style properties like
fs.s3a.access.key
and
fs.s3a.secret.key
. 3. Why you see "read: Is a directory" When you use
gs://
in a
CREATE REPOSITORY
statement, the system attempts to use the S3A driver on a path it doesn't fully understand, or it defaults to a check that fails because it cannot find the S3-compatible metadata it expects. This results in the "Is a directory" error because the storage driver is misinterpreting the GCS bucket response. Summary of the Documentation Conflict Feature | Protocol | VM Auth Supported? | Credential Required Data Loading / Catalogs |
gs://
| Yes | Service Account / VM Backup / Restore |
s3a://
| No | HMAC Keys (AK/SK) The page you linked is technically over-broad; while it says "StarRocks can authenticate to GCS [for backup]," the only way it can currently do so is by treating GCS as an S3-compatible store using HMAC keys. If you are following the "Authenticate to GCS" guide's VM-based steps for a Backup, it will fail because that specific feature isn't wired into the native GCS client yet. Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdadministration/management/Backup_and_restore.mdintegrations/authenticate_to_gcs.md
f
@Rocky who would be the right person in slack to confirm this?
r
To confirm this technical discrepancy, the best place is the StarRocks Slack community. Specifically, you should post your question in these channels: 1.
#questions-and-troubleshooting
: This is the most active channel where community experts and maintainers monitor technical issues. 2.
#contributing-to-starrocks
: If you want to talk directly to the developers who work on the storage and backup internals. Who to look for You can tag or look for responses from people with the "Maintainer" or "Committer" badge. Key individuals often active in storage and cloud-integration discussions include: * Albert Zhan (often handles core storage and backup logic) * Mabel (very active in community troubleshooting and product features) * Sailing (often involved in cloud-native and Kubernetes features) What to ask to get a quick confirmation: You can post a message like this to get straight to the point:
"I'm seeing a contradiction between the 'Authenticate to GCS' doc (which says VM auth is supported for backups) and the 'CREATE REPOSITORY' doc (which says GCS only supports S3A/HMAC). When I use VM auth for a repository, I get
read: Is a directory
. Can you confirm if the Repository system is currently hard-coded to use the S3A driver (requiring HMAC) and doesn't yet support the native
gs://
driver used by Broker Load?"
This specific phrasing identifies the "S3A vs. Native GS Driver" conflict, which the engineering team will immediately recognize as the root cause. Why the doc is likely wrong: The documentation you linked is a high-level "capability" overview, but the
CREATE REPOSITORY
command is a legacy part of the codebase that was built on Hadoop's S3A client before the native GCS integration was completed for other features like data loading. Referencessql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdintegrations/authenticate_to_gcs.mdadministration/management/Backup_and_restore.mdsql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.mdsql-reference/sql-statements/backup_restore/SHOW_REPOSITORIES.md
f
can someone from the StarRocks team confirm this contradiction? ☝️
@Rocky are there any reports of errors while backup of tables with GIN indexes
r
While there are no widespread reports of the Backup process itself failing specifically because of GIN (inverted) indexes, there are documented issues related to the Restore and Post-Restore query behavior for tables using this feature. Since GIN indexes are currently in Preview/Beta (v3.3+), they have a few known caveats during the backup/restore lifecycle: 1. Known Post-Restore Crashes There have been specific bug reports where querying a table or a synchronized materialized view after a restore caused a backend (BE) crash (core dump). * Issue: A restored table with an inverted index might have inconsistent metadata or data versioning that triggers a crash during the
MATCH
query execution. * Status: Many of these (e.g., GitHub #45010) were addressed in later patches of v3.3 and v3.4, but it remains a "heavy" metadata operation. 2. Implementation Restrictions * Shared-Data Limitation: The
BACKUP
and
RESTORE
commands are not supported for Shared-Data clusters. For Shared-Nothing clusters, the default CLucene-based GIN index is supported, but if you are migrating or using experimental versions, ensure the
enable_experimental_gin
FE configuration is consistent across the source and target clusters. * Protocol Issues: If you are backing up to GCS (as discussed previously), the error
read: Is a directory
is more likely related to the storage protocol (S3A vs GS) than the GIN index itself. The backup system treats GIN index files like any other segment data; it doesn't usually distinguish them until the Restore/Query phase. 3. Data Versioning Bugs A recent fix (GitHub #70373) addressed an issue where
dataVersion
was not set correctly when restoring from a backup. While not exclusive to GIN indexes, inverted indexes are highly sensitive to data versions because they map specific row IDs to terms. If the versioning is off, the index becomes "corrupt" post-restore. Recommendation If you are seeing a backup failure on a table with GIN indexes: 1. Check the Storage Protocol: Ensure you are using
s3a://
and HMAC keys if using GCS. 2. Verify the Version: Ensure you are on at least v3.3.4+ or v3.4.1+, where several inverted index and backup/restore stability fixes were merged. 3. Test Restore: Always verify a backup of a GIN-indexed table by restoring it to a test database and running a
MATCH
query to ensure the BE doesn't crash. Referencestable_design/indexes/inverted_index.mdsql-reference/sql-statements/backup_restore/BACKUP.mdsql-reference/sql-statements/backup_restore/CREATE_REPOSITORY.mdadministration/management/Backup_and_restore.md Referencestable_design/indexes/inverted_index.mdsql-reference/sql-statements/backup_restore/BACKUP.mdadministration/management/Backup_and_restore.md
f
@Rocky my problem is
TaskErrMsg: [101691: read: Is a directory], [101690: read: Is a directory], [101693: read: Is a directory], [101678: read: Is a directory
] this is not auth related - im using s3a as documented
r
The error
TaskErrMsg: [TabletID: read: Is a directory]
is a confirmed bug in StarRocks'
BACKUP
implementation specifically related to Inverted Indexes (GIN). It is not an authentication or
s3a
protocol issue. Why this is happening When you enable inverted indexes on a table in a Shared-Nothing cluster, StarRocks creates a subdirectory named
index
inside each tablet's data folder to store the CLucene index files. The
BACKUP
process on the Backend (BE) works by scanning the tablet directory and listing all files to be uploaded to your GCS repository. In certain versions, the backup scanner is "greedy" and includes the
index
directory itself in the list of files to upload. When the backup worker then tries to open and read that directory as if it were a regular data file to stream it to GCS, the underlying filesystem returns the error
read: Is a directory
(system error
EISDIR
). Key details for you: * The Task IDs: The numbers in your error message (e.g.,
101691
,
101690
) are the Tablet IDs of the table being backed up. You can verify this by running
SHOW TABLETS FROM <your_table_name>;
. * Affected Versions: This bug was primarily seen in the initial releases of the inverted index feature (v3.3.0 through v3.3.3) and early v3.4 release candidates. * The Fix: This has been officially fixed. The fix ensures that the backup process explicitly ignores subdirectories during the file-gathering phase. * Fixed in v3.3.4+ * Fixed in v3.4.1+ Recommended Actions: 1. Check your Version: Run
SELECT version();
. If you are on a version earlier than v3.3.4 or v3.4.1, you are likely hitting this known bug. 2. Upgrade: The most reliable fix is to upgrade your cluster to the latest patch release of your current minor version (e.g., v3.3.x or v3.4.x). 3. Workaround (if you cannot upgrade): _ If the backup is critical and urgent, you would need to temporarily
DROP INDEX
on the affected columns, run the backup, and then recreate the index. _Note: Recreating GIN indexes can be time-consuming on large datasets.* To confirm this with a maintainer on Slack, you can mention that your "BACKUP task is failing because the BE is attempting to read the CLucene 'index' subdirectory as a file". This will allow them to point you to the specific pull request that fixed the file-up the tablet file-file-listing logic. Referencesadministration/management/Backup_and_restore.mdfaq/operation_maintenance_faq.mdsql-reference/sql-statements/backup_restore/RESTORE.mdfaq/Exporting_faq.mdintegrations/streaming/pipe/s3.mdx
f
@Rocky im on 4.0.9
r
If you are on v4.0.9, this is highly likely a regression or a new occurrence of the "greedy scanner" bug specifically affecting the newer features in the 4.0 branch (such as Inverted Indexes on Primary Key tables). The error
read: Is a directory
occurs because the Backend (BE) backup worker iterates through the tablet's data directory, sees the
index
folder (where the CLucene GIN files live), and mistakenly tries to open it as a regular file to stream it to GCS. Why this is happening in 4.0.9: 1. Regression: A fix was merged in the v3.3/v3.4 branches to skip subdirectories during backup. It is possible this fix was missed or overwritten during the major architectural changes for v4.0. 2. Primary Key Tables: Version 4.0 introduced support for Inverted Indexes on Primary Key tables. If your table is a PK table, the backup logic for this specific table type might still be missing the directory-skipping logic that was applied to Duplicate/Aggregate tables. 3. Storage Protocol: Even though you are using
s3a://
, the error is originating from the Local File System on the BE before the data is even sent to GCS. The BE is failing to "read" the local tablet data because it's trying to
open()
a directory. Confirmation for Slack: When you reach out on Slack, provide these specific details to help the maintainers identify the regression: * Version: 4.0.9 * Table Type: (e.g., Primary Key or Duplicate Key) * Error:
TaskErrMsg: [TabletID: read: Is a directory]
* Observation: The error only occurs on tables where a GIN (Inverted) index is present. How to unblock yourself now: * Short-term Fix: If you need this backup immediately, you will have to
DROP INDEX
from the table, run the
BACKUP
, and then
CREATE INDEX
again. This removes the
index
subdirectory that is tripping up the scanner. * Check for 4.0.10: Check if v4.0.10 (released around May 9, 2026) contains a fix for this. The release notes for 4.0.10 mention several tablet-scheduling and metadata fixes, though they don't explicitly name this specific backup bug. The right person to tag on Slack: Ask for Albert Zhan or Sailing in the
#questions-and-troubleshooting
channel. Mention that you suspect a regression of the "EISDIR backup bug" originally tracked in v3.3.4/v3.4. Referencestable_design/indexes/inverted_index.mdadministration/management/Backup_and_restore.mdsql-reference/sql-statements/backup_restore/BACKUP.mdrelease_notes/release-4.0.md Referencesrelease_notes/release-4.0.mdtable_design/indexes/inverted_index.md