Hi Flink team, if I want to implement a flink Job ...
# troubleshooting
l
Hi Flink team, if I want to implement a flink Job to read avro files from S3 path, how can I do that? Which class shall I use to do that?
l
Hi @Bhupendra Yadav , thanks for the link. I implemented like this using
AvroInputFormat
:
Copy code
String s3Path =
        "s3://...";
    AvroInputFormat<CustomerSession> avroInputFormat =
        new AvroInputFormat<>(new Path(s3Path), CustomerSession.class);
    TypeInformation<CustomerSession> typeInfo = TypeInformation.of(CustomerSession.class);
    DataStream<CustomerSession> datastream = env.createInput(avroInputFormat, typeInfo);
But encountered this error in the job: org.apache.avro.AvroTypeException: Found KafkaRecord, expecting com.nordstrom.customer.object.analytical.CustomerSession, missing required field channel But I am currently reading CustomerSession avro files from our s3 bucket, and without any modifications, output to a customersession kafka topic, which has already been registered to the schema of CustomerSession.