Hi All, I'm getting this error sometimes for hudi ...
# questions-and-troubleshooting
m
Hi All, I'm getting this error sometimes for hudi table(external catalog), shared data architecture of starrocks 4.0.
Copy code
com.starrocks.common.DdlException: hdfsOpenFile failed, backend=starrockscluster-cn-0.starrockscluster-cn-search.starrocks.svc.cluster.local, file=gs://<bucket>/created_on_partitioned=202511/c3d89fc1-115b-4917-9d87-f32c021280f9-0_0-30230-90178_20251108134108762.parquet: file = gs://<bucket>/created_on_partitioned=202511/c3d89fc1-115b-4917-9d87-f32c021280f9-0_0-30230-90178_20251108134108762.parquet: BE:10003
	at com.starrocks.common.ErrorReport.reportDdlException(ErrorReport.java:90)
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2653)
	at com.starrocks.qe.StmtExecutor.handleDMLStmtWithProfile(StmtExecutor.java:2396)
	at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:801)
	at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:434)
	at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:659)
	at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:1017)
	at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:71)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
after rerunning the query, it is getting succeeded, it is trying to look for some file which already removed from hudi table path as it writes to new files and older files are cleaned. Is there any way to not fail the queries, if it fails to find the file, fetch again from bucket like fresh load?
Could someone check this please?
Anyone?
r
This looks kinda similar to the issue I reported today: https://github.com/StarRocks/starrocks/issues/65381. So it seems like not only Iceberg external catalogs are affected. Interesting