Hi team :slightly_smiling_face: I’m facing ingesti...
# troubleshoot
p
Hi team 🙂 I’m facing ingestion issue with Redshift cluster Can someone help? I can share logs 10x
📖 1
🔍 1
l
This content can't be displayed.
p
DH version - 0.9.6.1 Deployment method - k8s
Regarding sharing logs - I prefer a private chat with your Engineer 🙂
a
Hi, we have some fixes to redshift ingestion coming in our next release, @hundreds-photographer-13496 may be able to help in the meantime
a
Hi @powerful-cat-68806 could you share the error you are facing ?
p
@hundreds-photographer-13496 hi 🙂 From the logs, I see the following error frequently:
Copy code
Error was (psycopg2.errors.FeatureNotSupported) Specified types or functions (one per INFO '
           'message) not supported on Redshift tables



   '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 168, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}
My ingest config is:
Copy code
source:
    type: redshift
    config:
        host_port: 'xxx:xxx'
        database: xxxxx
        username: xxxxx
        password: xxxxxxxx
        include_tables: true
        include_views: true
        options:
            connect_args:
                sslmode: prefer
                sslrootcert: null
        profiling:
            enabled: true
            max_workers: 3
            profile_table_level_only: false
            include_field_sample_values: false
h
p
Ok I didn't managed to upgrade ...can you assist?
h
I believe redshift server versions got autoupgraded, that's why this had to be fixed. If you are on managed ingestion, you should be able to use newer datahub version using this - https://datahubproject.io/docs/ui-ingestion/#advanced-running-with-a-specific-cli-version
p
Managed ingestion?
h
UI based ingestion
p
I'm running in k8s I want to upgrade to latest..
h
I can't help upgrading entire DataHub deployment , sorry 🙂 can you check in #all-things-deployment?
a
If you want to only move redshift ingestion the latest cli vesion and you are using UI based ingestion, you can specify cli version as mentioned above to confirm it works fine.
p
Ok I'll check and ping this thread 😁
h
Hey @powerful-cat-68806 were you able to resolve this?
p
HI @hundreds-photographer-13496 I’m trying now, from the UI, with version
0.10.2.2
but my main goal is to upgrade my namespace(k8s) Also - using secrets not working for me & I need to use explicit passwords
@hundreds-photographer-13496 It fails with version
0.10.2.2
Repeatedly errors:
Copy code
"TypeError: 'str' object is not callable\n"

"PipelineInitError: Failed to find a registered source for type redshift: 'str' object is not callable\n"
h
Have you specified ingestion version 0.10.2.2 explicitly as specified here - https://datahubspace.slack.com/archives/C029A3M079U/p1682075356943389?thread_ts=1681998473.815389&cid=C029A3M079U? I am afraid the default cli version UI picks by default is 0.10.2 and not 0.10.2.2
As per this, I would alter my suggestion to use 0.10.1.2 instead. https://datahubspace.slack.com/archives/CUMV92XRQ/p1682702423740919
p
@hundreds-photographer-13496 hi Still failing with version
0.10.1.2
a
same error?
p
Yes @hundreds-photographer-13496
See this @hundreds-photographer-13496