powerful-cat-68806
05/16/2023, 8:58 AM'/usr/local/bin/run_ingest.sh: line 40: 2722 Killed ( datahub ${debug_option} ingest run -c "${recipe_file}" '
several times. Each time, I’ve extended the memory & CPU for acryl-datahub-actions
pod which came out finally to this enormous config:
acryl-datahub-actions:
enabled: true
image:
repository: acryldata/datahub-actions
tag: "v0.0.7"
resources:
limits:
memory: 120Gi
requests:
cpu: 32Gi
memory: 100Gi
But still failing with similar error
I’m using, from the UI, the CLI version 0.10.1.2
I’ve checked also the memory usage for this pod & I don’t see any issue
Any idea?
Cc: @modern-garden-35830 @icy-controller-68116astonishing-answer-96712
05/16/2023, 8:32 PMpowerful-cat-68806
05/17/2023, 9:39 AMbrainy-tent-14503
05/17/2023, 5:31 PMdescribe
on the failed pod and any logs from the actions container?brainy-tent-14503
05/17/2023, 5:34 PMfamous-waitress-64616
05/17/2023, 5:43 PMinclude_tables: false
include_views: false
include_table_lineage: false
include_copy_lineage: false
include_usage_statistics: false
profiling:
enabled: false
powerful-cat-68806
05/17/2023, 5:58 PMpowerful-cat-68806
05/17/2023, 6:04 PM0.10.2.3
My current ingestion recipe looks as follows:
source:
type: redshift
config:
host_port: 'xxxxxx:xxxx'
database: xxxxx
username: xxx
password: 'xxxxxx'
include_tables: true
include_views: true
options:
connect_args:
sslmode: prefer
sslrootcert: null
profiling:
enabled: true
max_workers: 3
profile_table_level_only: false
include_field_sample_values: false
Another issue I’m facing is the password field. When trying to use the secret service the UI provides, it’s failing with an authentication error. Is any special template needed here?aloof-gpu-11378
05/18/2023, 3:14 AM\n
, and don't include any quotes.powerful-cat-68806
05/18/2023, 11:02 AMpowerful-cat-68806
05/18/2023, 11:03 AMpowerful-cat-68806
05/18/2023, 11:52 AMpassword: 'A!R2jyg543mjy$vd'
Any reason that DH secret service won’t work?famous-waitress-64616
05/18/2023, 1:53 PMvalue
field as A!R2jyg543mjy$vd
?powerful-cat-68806
05/18/2023, 2:08 PMpowerful-cat-68806
05/21/2023, 6:47 AMERROR {datahub.entrypoints:195} - Command failed: {'S': 'FATAL', 'C': '28000', 'M': 'password "
'authentication failed for user
When putting the password explicitly in the YAML recipe, it not failing with this error
What’s the issue here?powerful-cat-68806
05/21/2023, 6:48 AMpowerful-cat-68806
05/22/2023, 2:43 PMaloof-gpu-11378
05/22/2023, 4:26 PMbrainy-tent-14503
05/22/2023, 4:29 PMpassword: "${REDSHIFT_PASSWORD}"
in the recipe.powerful-cat-68806
05/22/2023, 8:53 PM