quick-pizza-8906
01/10/2023, 12:37 AMquick-pizza-8906
01/10/2023, 1:33 AMmetrics so far RestoreIndicesResult(ignored=0, rowsMigrated=1000, timeSqlQueryMs=2, timeGetRowMs=0, timeUrnMs=6, timeEntityRegistryCheckMs=0, aspectCheckMs=0, createRecordMs=92, sendMessageMs=854)
Successfully sent MAEs for 1000/1651106 rows (0.06% of total). 0 rows ignored (0.00% of total)
0.02 mins taken. 33.25 est. mins to completion. Total mins est. = 33.27.
While time increases with each batch to reach 20 mins per batch at the end:
Reading rows 1650000 through 1651000 from the aspects table completed.
Successfully sent MAEs for 1649942/1651106 rows (99.93% of total). 58 rows ignored (0.00% of total)
2113.96 mins taken. 1.49 est. mins to completion. Total mins est. = 2115.45.
metrics so far RestoreIndicesResult(ignored=58, rowsMigrated=1650942, timeSqlQueryMs=40, timeGetRowMs=0, timeUrnMs=1821, timeEntityRegistryCheckMs=376, aspectCheckMs=296, createRecordMs=31830, sendMessageMs=80009)
Args are RestoreIndicesArgs(start=1651000, batchSize=1000, numThreads=1, batchDelayMs=100, aspectName=null, urn=null, urnLike=null)
Successfully sent MAEs for 1650942/1651106 rows (99.99% of total). 58 rows ignored (0.00% of total)
Reading rows 1651000 through 1652000 from the aspects table started.
2117.09 mins taken. 0.21 est. mins to completion. Total mins est. = 2117.30.
Did somebody experience similar problems with restore job? Can we improve performance somehow?average-dinner-25106
01/10/2023, 2:29 AMbest-wire-59738
01/10/2023, 8:18 AMlate-book-30206
01/10/2023, 10:36 AM2023/01/10 10:19:06 Problem with request: Get <http://elasticsearch:9200>: dial tcp: lookup elasticsearch on 10.3.0.10:53: no such host. Sleeping 1s
2023/01/10 10:19:06 Timeout after 2m0s waiting on dependencies to become available: [<http://elasticsearch:9200>]
late-book-30206
01/10/2023, 10:37 AM2023/01/10 10:19:06 Problem with request: Get <http://elasticsearch:9200>: dial tcp: lookup elasticsearch on 10.3.0.10:53: no such host. Sleeping 1s
2023/01/10 10:19:06 Timeout after 2m0s waiting on dependencies to become available: [<http://elasticsearch:9200>]
It seems, elasticsearch is not reachable or not resolved by his IP address.
For information :
prerequisistes 0.0.9
datahub 0.2.89
Does anyone know what could be the problem here please ?acceptable-terabyte-34789
01/10/2023, 12:26 PMfaint-actor-78390
01/10/2023, 2:57 PMlate-shampoo-30424
01/10/2023, 5:54 PMDatasource mssql_datasource is not present in platform_instance_map
('Unable to emit metadata to DataHub GMS', {'message': '401 Client Error: Unauthorized for url: <http://localhost:9002/openapi/entities/v1/aspects?action=ingestProposal'}>)
Any idea on what is going on? I’m a bit lostkind-policeman-5342
01/10/2023, 7:47 PMsalmon-jackal-36326
01/10/2023, 10:53 PMsalmon-jackal-36326
01/10/2023, 10:54 PMsalmon-jackal-36326
01/10/2023, 11:35 PMpython3 -m datahub docker quickstart --restore --restore-file ../datahub/docker/quickstart/backup_2023_01_10.sql
but despite the data being already inserted in the database, I still don't see anything through the ui.
What should I do?
Because when I use the command above I receive [2023-01-10 233638,311] ERROR {datahub.cli.docker_cli:353} - Failed to run MySQL restoreaverage-dinner-25106
01/11/2023, 7:50 AMmelodic-dress-7431
01/11/2023, 11:24 AMmicroscopic-mechanic-13766
01/11/2023, 3:25 PMfreezing-father-76422
01/11/2023, 4:54 PMmetadata_aspect_v2
table)
• drop the chart completely and reinstall it back
• restore MySQL data
After that, I expect datasets and users to re-appear in the UI, but UI remains completely clean.faint-painting-38451
01/11/2023, 6:11 PMfreezing-father-76422
01/11/2023, 7:28 PMripe-electrician-13049
01/12/2023, 1:44 AMred-waitress-53338
01/12/2023, 2:25 AMmagnificent-lock-58916
01/12/2023, 7:22 AMdazzling-shampoo-79134
01/12/2023, 8:03 AMlemon-cat-72045
01/12/2023, 9:11 AMValidation error (UnknownType) : Unknown type 'BatchGetStepStatesInput' (code undefined)
steep-fountain-54482
01/12/2023, 9:43 AMsteep-fountain-54482
01/12/2023, 9:44 AMquery {
search(input: {
type: DATA_PLATFORM,
query: "*",
count: 100,
start: 0,
}) {
total
searchResults {
entity {
urn
}
}
}
steep-fountain-54482
01/12/2023, 9:44 AMsteep-fountain-54482
01/12/2023, 9:44 AMchilly-postman-99763
01/12/2023, 2:49 PMHello,
I'm trying to install bigquery plugin in composer,
modern-garden-35830
01/12/2023, 3:01 PMsource:
type: "csv-enricher"
config:
filename: "/datahub/datahub.csv"
should_overwrite: false
delimiter: ","
array_delimiter: "|"
sink:
type: "datahub-rest"
config:
server: "<http://serveraddress.com>"