Okay here s a better question after resolving my user error Apache Flink #troubleshooting

Okay, here's a better question after resolving my ...

Nathanael England

04/24/2023, 8:27 PM

Okay, here's a better question after resolving my user error above. How can I get flink to respect the

.so

included in the path? I can see from logging my dependencies contents that the so is making it to the job.

Copy code

ERROR:root:/tmp/pyflink/8d6f1590-3459-4e27-b118-2cf001999248/c6e62fed-b841-4a37-be6e-2001db67f477/usrlib/.deps/google/protobuf/pyext/cpp_message.py
ERROR:root:/tmp/pyflink/8d6f1590-3459-4e27-b118-2cf001999248/c6e62fed-b841-4a37-be6e-2001db67f477/usrlib/.deps/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so
ERROR:root:/tmp/pyflink/8d6f1590-3459-4e27-b118-2cf001999248/c6e62fed-b841-4a37-be6e-2001db67f477/usrlib/.deps/google/protobuf/pyext/__init__.py

The error I get is

Copy code

ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/tmp/pyflink/8d6f1590-3459-4e27-b118-2cf001999248/8541a21e-7834-425d-9a53-63350409a2a5/.deps/google/protobuf/pyext/__init__.py)

Nathanael England

04/24/2023, 8:41 PM

This works just fine in other docker images we have without any special handling

Copy code

/app# PYTHONPATH=.deps/protobuf-3.19.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/ python
Python 3.8.16 (default, Mar 23 2023, 17:40:49) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.protobuf.pyext import _message
>>> _message.__file__
'/app/.deps/protobuf-3.19.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so'

3 Views

Open in Slack

Previous Next