Hi, @Rocky do you have any idea why this happens while I am trying to read some data in shared data cluster? ERROR 1064 (HY000): starlet err meta/0000000000000000_0000000000000018.meta: No such file or directory: BE:583524. When i check this partition files, i see this meta file

This error message indicates that the Starlet component (the storage engine layer for StarRocks shared-data/compute-storage separation) is unable to find a specific tablet metadata file on the remote object storage (like S3, MinIO, or HDFS). Even if you can see the file when browsing your storage manually, the BE might be failing to locate it for several reasons. Here are the most likely causes and how to troubleshoot them: 1. Configuration/Path Mismatch The BE/CN nodes use the configuration provided by the FE to determine the path to the data. If there is a mismatch in the bucket name, endpoint, or prefix between the FE's metadata and the BE's environment, the BE will look in the wrong place. * Check the BE logs (

<http://be.INFO|be.INFO>

): Search for the specific file name (

0000000000000000_0000000000000018.meta

). The logs usually contain the full URI (e.g., `s3://my-bucket/starrocks/meta/...`) that Starlet attempted to access. * Verify the URI: Compare the path in the logs with the actual path where you found the file. Even a small difference in the prefix or bucket name will cause this error. 2. Permissions and Credentials The BE/CN nodes must have the correct credentials (Access Key/Secret Key or IAM Role) to read from the object storage. * If you can see the file via a CLI tool or a browser but the BE cannot, ensure the user/role the BE process is running under has

GetObject

and

ListBucket

permissions for that specific path. * Check if the BE has the correct

aws_sdk

or S3-related configurations in

be.conf

or

cn.conf

(if you are using specific environment variables or configuration overrides). 3. Starlet Cache Inconsistency In shared-data mode, StarRocks caches metadata and data locally under the

storage_root_path/starlet_cache

directory. * If the local cache becomes inconsistent or points to a file that was moved/deleted/re-uploaded, you might see this error. * Troubleshooting: Try clearing the local cache on the specific BE reporting the error (in your case, BE

583524

). You can usually do this by stopping the BE and clearing the contents of the

starlet_cache

directory, then restarting it. 4. FE Metadata Inconsistency If you recently migrated the cluster, restored from a backup, or manually moved files in the object storage, the FE's internal metadata might still be pointing to the "old" location or an incorrect version of the tablet. * Run

SHOW TABLET <tablet_id>

(you can find the tablet ID from the error context or by searching the BE logs) to see where the FE thinks the data is located. 5. Compaction or GC (Garbage Collection) It is possible that a background compaction process merged this metadata into a newer version and deleted the old one, but a running query or a stale FE state is still trying to reference the old file. * If this happened during a heavy load or right after a delete/update operation, try running the query again to see if it was a transient state. Recommended Next Steps: 1. Find the full path: Check the

<http://be.INFO|be.INFO>

log on the BE node with ID

583524

. Look for the "No such file or directory" error to see the full S3/HDFS path. 2. Verify access: From that same BE server, try to

curl

or use a CLI tool (like

aws s3 cp

) to download that specific

.meta

file using the same credentials the BE uses. 3. Check FE state: Run

ADMIN CHECK TABLET (<tablet_id_list>)

to see if the FE detects any inconsistencies for the affected table. References • data_source/catalog/paimon_catalog.md