got a weird one... refactored my golang data produ...
# troubleshooting
g
got a weird one... refactored my golang data producer to publish direct avro serialized payloads to my 2 Kafka topics (avro_salesbaskets and avro_salespayments). it's working, i can inspect them on CP cluster, see data flowing in. I now try to run the following in the flink sql client to create a flink virtual tables against which I later want to joins on.
Copy code
CREATE TABLE avro_salesbaskets (
    INVOICENUMBER STRING,
    SALEDATETIME_LTZ STRING,
    SALETIMESTAMP_EPOC STRING,
    TERMINALPOINT STRING,
    NETT DOUBLE,
    VAT DOUBLE,
    TOTAL DOUBLE,
    STORE row<ID STRING, NAME STRING>,
    CLERK row<ID STRING, NAME STRING, SURNAME STRING>,
    BASKETITEMS array<row<ID STRING, NAME STRING, BRAND STRING, CATEGORY STRING, PRICE DOUBLE, QUANTITY INT>>,
    SALESTIMESTAMP_WM AS TO_TIMESTAMP(FROM_UNIXTIME(CAST(SALETIMESTAMP_EPOC AS BIGINT) / 1000)),
    WATERMARK FOR SALESTIMESTAMP_WM AS SALESTIMESTAMP_WM
) WITH (
    'connector' = 'kafka',
    'topic' = 'avro_salesbaskets',
    'properties.bootstrap.servers' = 'broker:29092',
    'properties.group.id' = 'testGroup',
    'scan.startup.mode' = 'earliest-offset',
    'value.format' = 'avro-confluent',
    'value.avro-confluent.schema-registry.url' = 'http://schema-registry:8081',
    'value.fields-include' = 'ALL'
);
and
Copy code
CREATE TABLE avro_salespayments (
    INVOICENUMBER STRING,
    FINTRANSACTIONID STRING,
    PAYDATETIME_LTZ STRING,
    PAYTIMESTAMP_EPOC STRING,
    PAID DOUBLE,
    PAYTIMESTAMP_WM AS TO_TIMESTAMP(FROM_UNIXTIME(CAST(PAYTIMESTAMP_EPOC AS BIGINT) / 1000)),
    WATERMARK FOR PAYTIMESTAMP_WM AS PAYTIMESTAMP_WM
) WITH (
    'connector' = 'kafka',
    'topic' = 'avro_salespayments',
    'properties.bootstrap.servers' = 'broker:29092',
    'properties.group.id' = 'testGroup',
    'scan.startup.mode' = 'earliest-offset',
    'value.format' = 'avro-confluent',
    'value.avro-confluent.url' = 'http://schema-registry:8081', 
    'value.fields-include' = 'ALL'
);
Trying simple select results in the following, both tables err same way.
Copy code
Flink SQL> select * from avro_salesbaskets;
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator
d
are you using event-time? I think this is indicated for working with watermarks.
Double-check that the Avro schemas in Confluent Schema Registry match exactly with the table schema you’ve defined in your Flink SQL statements. Also check field types in your Flink SQL DDL match the actual Avro schema types. SALETIMESTAMP_EPOC is defined as STRING in your SQL but seems to contain an epoch timestamp might be better as BIGINT if that’s supported.
It could also be the way you are recalculating SALESTIMESTAMP_WM & PAYTIMESTAMP_WM doing string epoch back to timestamp. Make sure that this really produces a valid timestamp and string parsing back to timestamp is working.
also if you have not already bump up logging level to DEBUG on all Flink jobs for more precise info
g
problem was case... mismatches i was running camelcase in my avro schema as registered in schema registry, but my create tables did not include backtick, as such it defaulted to uppercase.
the above create tables as you can see was uppercase. that <> to what was in schema registry.
d
yep just a slight variance will do it. I was not aware that case would cause this. Good to know.
g
this comment "Double-check that the Avro schemas in Confluent Schema Registry match exactly" made me go look, think/try...