rapid-room-3050
09/08/2021, 8:42 PMaloof-gigabyte-305
09/09/2021, 5:21 PMcurved-daybreak-29035
09/10/2021, 10:35 AMbrave-market-65632
09/13/2021, 6:43 AMcalm-river-44367
09/13/2021, 8:49 AMfaint-hair-91313
09/13/2021, 9:36 AMdatahub ingest list-runs
, so cannot roll back if needed. Does it happen on your side, too? The other recipes show up.red-smartphone-15526
09/13/2021, 11:32 AMfaint-hair-91313
09/13/2021, 1:47 PMcurved-jordan-15657
09/13/2021, 2:08 PM{
"sink": {
"config": {
"connection": {
"schema_registry_url": "<my-schema-registry>:8081"
}
}
}
}
But it gives me an error like:
sink
extra fields not permitted (type=value_error.extra)
Also above lines in log, i see
Host: <my-broker>:9092, Port: None, Schema: , Login: , Password: None, extra: {'sink': {'config': {'connection': {'schema_registry_url': '<my-schema-registry>:8081'}}}}
So, it gets my host but can not get port? I didn’t get it. Also, how can i resolve the extra fields problem. Thanks!calm-river-44367
09/14/2021, 8:31 AMblue-megabyte-68048
09/15/2021, 3:20 PMthousands-tailor-5575
09/15/2021, 3:31 PMlively-laptop-26966
09/15/2021, 11:06 PMmicroscopic-musician-99632
09/16/2021, 1:28 PMcurved-sandwich-81699
09/16/2021, 2:45 PMbest-toddler-40650
09/16/2021, 3:37 PMsource:
type: oracle
config:
username: user
password: password
host_port: myserver:1522
service_name: servicename
which translate in the following DSN passed to sqlalchmy connect dialect
'dsn': '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=myhost)(PORT=1522))(CONNECT_DATA=(SERVICE_NAME=myservicename)))'
however, the oracle server requires further parameters in the DSN in order to authenticate, as below:
'dsn': '(description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=myhost))(connect_data=(service_name=myservicename))(security=(ssl_server_cert_dn="CN=certsever, OU=Oracle, O=Oracle Corporation, L=City, ST=State, C=Country")))
Any idea on how to insert these extra parameters in the oracle recipealoof-gigabyte-305
09/17/2021, 1:09 AMeager-answer-71364
09/17/2021, 4:52 AMhappy-orange-34018
09/20/2021, 9:19 AMstocky-noon-61140
09/20/2021, 9:47 AMaverage-bear-318
09/20/2021, 10:33 AMquiet-kilobyte-82304
09/20/2021, 8:02 PMKeyValueSchema
? Looking into ingesting key/value pairs from couchbase into datahub. I’m more interested in seeing how fields that have isPartOfKey
set to true
are exposed on the UI https://github.com/linkedin/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/schema/SchemaField.pdl#L84polite-flower-25924
09/20/2021, 8:56 PM(97bed71)
It works with older versions like c9c1ba4
. I guess some libraries are changed and this needs to be fixed.microscopic-musician-99632
09/21/2021, 5:45 AMhigh-hospital-85984
09/21/2021, 10:00 AMdatahub ingest list-runs
and got presented with something unexpected. Most IDs are random GUIDs, and one suspicioulsy large run wrt row count is called no-run-id-provided
. We primarily use the kafka sink, is there a way of providing some human-readable name to the runs for easier rollback?calm-river-44367
09/22/2021, 7:01 AMcalm-sunset-28996
09/22/2021, 3:20 PMextra_headers
or something which would be k,v to add to the emitter? Then for future usecases people could add whatever they want if they would need it.loud-vase-59377
09/23/2021, 12:43 AMsource:
type: bigquery
config:
project_id: "su-project1"
schema_pattern:
allow:
- "db1"
table_pattern:
allow:
- "db1.table1"
profiling: {
enabled: true,
}
profile_pattern:
allow:
- "db1.table1"
sink:
type: "datahub-rest"
config:
server: "<http://localhost:8080>"
I tried removing the schema pattern and retain the table pattern but it ingest all database in the project. Are there other options I need to set?ripe-furniture-93265
09/23/2021, 8:33 AMeager-answer-71364
09/23/2021, 11:03 AM