Hello, I’m having an issue upgrading DataHub via Helm from

0.9.1

to

0.9.2

(or newer). It seems that

datahub-acryl-datahub-actions

is the problem, it throws:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/site-packages/datahub_actions/pipeline/pipeline_manager.py", line 42, in run_pipeline
    pipeline.run()
  File "/usr/local/lib/python3.10/site-packages/datahub_actions/pipeline/pipeline.py", line 166, in run
    for enveloped_event in enveloped_events:
  File "/usr/local/lib/python3.10/site-packages/datahub_actions/plugin/source/kafka/kafka_event_source.py", line 154, in events
    msg = self.consumer.poll(timeout=2.0)
  File "/usr/local/lib/python3.10/site-packages/confluent_kafka/deserializing_consumer.py", line 139, in poll
    raise ValueDeserializationError(exception=se, kafka_message=msg)
confluent_kafka.error.ValueDeserializationError: KafkaError{code=_VALUE_DESERIALIZATION,val=-159,str="HTTPConnectionPool(host='prerequisites-cp-schema-registry', port=8081): Max retries exceeded with url: /schemas/ids/2 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8c1ad1e8f0>: Failed to establish a new connection: [Errno 111] Connection refused'))"}
%4|1675236299.041|MAXPOLL|rdkafka#consumer-1| [thrd:main]: Application maximum poll interval (10000ms) exceeded by 170ms (adjust <http://max.poll.interval.ms|max.poll.interval.ms> for long-running message processing): leaving group

Ingestions seems to go into pending state and nothing happens. I haven’t changed anything apart from app version.

We haven't seen this error before. Which version of datahub-actions image is used here ?

0.0.10

and which version was used before upgrade ?

0.0.8

Just to confirm, these are acryldata/datahub-actions image versions right ? Can you try using the same actions image as before ?

I haven’t checked that, doing it right now

awesome. You can try upgrading datahub-actions image as a separate activity, if required.