Hi Team, I have multiple jobs using the same kafka...
# troubleshooting
h
Hi Team, I have multiple jobs using the same kafka-table so they have the same Group_ID. What happens if kafka consumer fails? Should I create multiple kafka-table queries from the same kafka topic with different Group_ID?
n
What happens if kafka consumer fails?
The partitions assigned to that consumer will be reassigned to others. Multiple jobs with the same group ID is weird to me. If you have multiple jobs with the same group ID doing different things then the results will be unpredictable for each of those jobs. If they are doing the same thing, then why have different jobs? I might be missing something here.
h
@Nicholas Erasmus Suppose I have job A using topic FL_T24_FUNDS_TRANSFER, I create a FLINK table with groupID "*FL_A*". `CREATE TABLE
KAFKA
.
T24_FUNDS_TRANSFER
(...
Copy code
)`
COMMENT ''
WITH (
'properties.group.id' = 'FL_A',
'topic' = 'FL_T24_FUNDS_TRANSFER'
...
)
I have another job B that also wants to read data from topic FL_T24_FUNDS_TRANSFER, should I use the same TABLE
KAFKA
.
T24_FUNDS_TRANSFER
or create another FLINK TABLE with a different groupID, "*FL_B"* for example.
one difference with Apache FLINK is that I am using Ververica so there are catalogs/databases storing FLINK Tables. I want to reuse it for different jobs.
n
If you have 2 different jobs then the table names don't matter. It's just about the group ID.
👍 1