We are currently using StarRocks version 3.5.6. Can I adjust the FE/BE configurations to prevent this issue from happening?

check what the query looks like, that triggers this error.

The sql：select project_id,indicator , sum（value） as value from （ select project_id, device_id, indicator , sum（value） as value, count（） as nums, Count（distinct device_id） as devNums from （ select project_id, device_id, indicator, num ,date_trunc（'month',fromUnixtimeWithTimezone（cast（`time` as string）,'Europe/Berlin'）） as month, value from tuya_basic.device_indicator_dimension_detail_all where types='open' and project_id='p1684122229458udn5g3' and indicator in （'ele_produce'） and （（fromUnixtimeWithTimezone（time,'Europe/Berlin'） >= '2025-10-26 000000' and fromUnixtimeWithTimezone（time,'Europe/Berlin'） <= '2025-10-31 235959'）） and time_format >= '2025-10-25 000000' and time_format <= '2025-11-01 235959' and tag10 in （'207905469'） and time_zone_id <> '' union all SELECT project_id, device_id, indicator, num ,date_truncate（'month',convert_tz（hour,'Etc/GMT','Europe/Berlin'）） as month, sumsValue as value FROM tuya_basic.dws_device_energy_metering_timezone_hour WHERE types = 'open' AND project_id = 'p1684122229458 udn5g3' and indicator in （'ele_produce'）and （（convert_tz（hour,'Etc/GMT','Europe/Berlin'） >= '1970-01-01 000000' and convert_tz（hour,'Etc/GMT','Europe/Berlin'） <= '2025-10-25 235959'）） and hour >= '2022-12-31 000000' and hour <= '2025-10-26 235959' and tag10 = '207905469' and time_zone_id <> '' ）a group by project_id, device_id, indicator ）a group by project_id,indicator

how long has it been running before err out?

The TTL of the table tuya_basic.device_indicator_dimension_detail_all is set to 3, so it only retains data for 3 days. The abnormal file db22111/35238/25002588/meta/0000000000000000_0000000000000008.meta corresponds to a tablet in the device_indicator_dimension_detail_all table, which belongs to the partitions from the last 2–3 days.

check if the partition which the tablet belongs to get dropped during the query.

How can I check this? The table’s partition expiration relies on the 3-day TTL set during table creation, and there should be no manual drop operations.

search from FE logs or audit logs.

For this partition, the query between 07:42 and 07:46 on 2025-10-24 reported a “The specified key does not exist” error, while the deletion of the partition

<s3://olap-eu/c5/08052b61-b255-4d57-9f18-481558523d7c/db22111/35238/25002588>

occurred at 075106.486 on 2025-10-24.

shall check the begin of the drop partition op on FE log, the "removed s3://...." printed at the end of the deletion, which may take some time, depending on how many files/objects under the path.

This partition has many query failures, but no deletion log was found.

or it is caused by the version expiration, the tablets employs MVCC, multiple versions can be there for the same tablet, and by default the non-latest version will be cleaned in 30 mins. try adjust

lake_autovacuum_grace_period_minutes

in FE configuration, if this helps in your case.

Thanks, I’ll try it

if continuously reported from the same file, it could be a different issue.

The logs continuously print the same key at different times.

1. show partitions from the specific table, check the visible version. 2. list files from remote storage under the .../meta/ directory, what's the meta files there right now.

visible version

it's fine with the long 0 prefix, I assume it is 4.0 cluster, which the file bundling feature is enabled by default.

From the logs and meta information, the version info seems to far away, which looks like a version info inconsistency. Also, I'd like to ask about the composition of the filename 0000000000000000_0000000000001CAF.meta. 0000000000001CAF is the visible version, but what does 0000000000000000 represent?

000000 is the tablet id if file bundling feature is off, when file bundling feature is on, all the tablets of the same partition shares the same meta file, so the tablet id part is just dummy 0s.

StarRocks Version: 3.5.6-3ba6b8d

upgraded to v4.0 and then rollback to v3.5.6?

Upgraded from v3.5.3 to v3.5.6, and v4.0 has not been deployed yet.

cc @Casey Luo

Is there any way to avoid the “The specified key does not exist” error in version 3.5? @Kevin Cai @Casey Luo

These 00000_000xxxx.meta not exist is not related, shall figure out the real tablet id trying to access which is not found.

The reported partition has multiple tablets, and all of them still exist — none have been deleted.