#2186 [Ideas] Better parallel scans for AO and AOCS tables New discussion created by roseduan Description Hi all, I'm working on making parallel scan of append-optimized tables (AO and AOCS) more useful, and I'd like your feedback. The problem Today each parallel worker reads a whole segment file. A table with few segment files, often just one after a bulk load, gets little or no parallelism(if GUC

gp_appendonly_insert_files

is not set), and one big file is always read by a single worker. The idea Split each segment file into many small chunks of rows, and let workers take chunks one at a time from a shared counter. Every worker stays busy until the scan ends, however many segment files the table has. No index or extra metadata is needed. Status • AOCS: implemented. Results match serial scans, the related regression tests pass, and large tables scan several times faster in my tests. The planner can now choose a parallel plan for AOCS tables with only one segment file. • AO row: planned next, using the same approach. Questions for the community • Does this direction make sense to you? • Anything else I should cover or test? Thanks! Use case/motivation No response Related issues No response Are you willing to submit a PR? • Yes I am willing to submit a PR! apache/cloudberry