Hi everyone. I want to use Flink 1.18.1 to write p...
# troubleshooting
w
Hi everyone. I want to use Flink 1.18.1 to write parquet file from pubsub.But i face an error.
Copy code
Caused by: java.lang.NoSuchMethodError: 'org.apache.avro.LogicalType org.apache.avro.Schema.getLogicalType()'

here is my pom
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-parquet</artifactId>
            <version>${flink.version}</version>
            <exclusions>
                <exclusion>
                    <artifactId>protobuf-java</artifactId>
                    <groupId>com.google.protobuf</groupId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-avro</artifactId>
            <version>${flink.version}</version>
            <exclusions>
                <exclusion>
                    <artifactId>avro</artifactId>
                    <groupId>org.apache.avro</groupId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-avro-confluent-registry</artifactId>
            <version>${flink.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-avro</artifactId>
            <version>1.12.2</version>
            <exclusions>
                <exclusion>
                    <artifactId>slf4j-api</artifactId>
                    <groupId>org.slf4j</groupId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-hadoop</artifactId>
            <version>1.12.2</version>
        </dependency>
how can i solve it. I try to print org.apache.avro.Schema methods, it has getLogicalType() method when runtime.
s
can you try updating
Copy code
<dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-avro</artifactId>
            <version>1.12.2</version>
            <scope>runtime</scope>
            <exclusions>
                <exclusion>
                    <artifactId>slf4j-api</artifactId>
                    <groupId>org.slf4j</groupId>
                </exclusion>
            </exclusions>
        </dependency>
👀 1
w
I tried, but it didn’t work
Copy code
1.13.1 
1.14.1
if i set scope to runtime, org.apache.parquet.avro.AvroParquetWriter not found.
s
can you add parquet-java in pom file ? https://github.com/apache/parquet-java/tree/master