Is there a way to truly ensure that a datalake que...
# questions-and-troubleshooting
k
Is there a way to truly ensure that a datalake query only uses and runs on a single compute node? I have
Copy code
prefer_compute_node=true
use_compute_nodes=1
parallel_exchange_instance_num=1
and most of my fragments I see backendnum:1 However, I still end up with a few fragments where backendnum: 30 which means it is using the whole cluster. On those fragments during heavy load I get killed on shuffle. I'd like to eliminate the shuffle completely by executing the whole query on a single compute node.
Copy code
Fragment 25                                                                                                                                                                                                                                                                                                                           |
| │   BackendNum: 30                                                                                                                                                                                                                                                                                                                  |
| │   InstancePeakMemoryUsage: 1.125 MB, InstanceAllocatedMemoryUsage: 46.853 MB                                                                                                                                                                                                                                                      |
| │   PrepareTime: ?                                                                                                                                                                                                                                                                                                                  |
| └──DATA_STREAM_SINK (id=91)                                                                                                                                                                                                                                                                                                     |
|    │   PartitionType: HASH_PARTITIONED                                                                                                                                                                                                                                                                                              |
|    │   PartitionExprs: [655: owner_id, 654: link_type_id, 659: domain, 656: src_entity_id, ...]                                                                                                                                                                                                                                     |
|    └──ICEBERG_SCAN (id=90)                                                                                                                                                                                                                                                                                                      |
|           Estimates: [row: 12330764, cpu: ?, memory: ?, network: ?, cost: 0.0]                                                                                                                                                                                                                                                        |
|           TotalTime: 17.895ms (0.03%) [CPUTime: 940.783us, ScanTime: 16.954ms]                                                                                                                                                                                                                                                        |
|           OutputRows: 2.810K (2810)                                                                                                                                                                                                                                                                                                   |
|           RuntimeFilter: 2.810K (2810) -> 2.810K (2810) (0.00%)                                                                                                                                                                                                                                                                       |
|           SubordinateOperators:                                                                                                                                                                                                                                                                                                       |
|               CHUNK_ACCUMULATE