Hello :wave: We would like to use fine-grained res...
# troubleshooting
s
Hello đź‘‹ We would like to use fine-grained resource management feature at the TableAPI level. The documentation mentions fine-grained resource management is only supported at the DataStreamAPI level. We are experimenting with the TableAPI-DataStreamAPI bridge feature but we are seeing an extra slot being required when we convert from TableAPI to DataStreamAPI. Do you have some ideas for this?
Copy code
Table res = getTableEnvironment().sqlQuery(query);
                StreamTableEnvironment stEnv = (StreamTableEnvironment) getTableEnvironment();

                DataStream dataStream = stEnv.toDataStream(res);
                dataStream.getTransformation().setSlotSharingGroup(ssgA);

                dataStream.getExecutionEnvironment().registerSlotSharingGroup(ssgA);

                res = stEnv.fromDataStream(dataStream);
w
Do you means this slot request is no need? Could you please share the log of jobmanager?
s
Thanks @weijie guo! I mean that we expected the toDataStream will convert the table to a dataStream without the need of an extra slot: if we do not use toDataStream, then the table can execute with only one slot. Here are the logs and the related line :
Copy code
2023-03-02 12:33:08,082 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager [] - Received resource requirements from job 3f3fe6ede66302a88c1cd9f60e572f7a: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}, ResourceRequirement{resourceProfile=ResourceProfile{cpuCores=0.53, taskHeapMemory=133.000mb (139460608 bytes), taskOffHeapMemory=0 bytes, managedMemory=0 bytes, networkMemory=320.000kb (327680 bytes)}, numberOfRequiredSlots=1}]
Notice how there are two “ResourceRequirements”, each one asking for 1 slots.
Hi @weijie guo, just reminder if you have some clue? many thanks!