https://datahubproject.io logo
Join Slack
Powered by
I performed a superset ingestion, the yml code is ...
# ingestion
a
I performed a superset ingestion, the yml code is shown below
Copy code
source:
  type: superset
  config:
    # Coordinates
    connect_uri: <http://localhost:8088>

    # Credentials
    username: xxx
    password: xxx
    provider: db

sink:
  # sink configs
  type: "datahub-rest"
  config:
    server: "<http://192.168.229.4:8080>"
then, I carried out the ingestion command,
Copy code
datahub ingest -c superset.yml
I get this hint But in my datahub homepage, I can't see the charts. Can someone tell me why? Thank you
b
Some things to check, are all the datahub containers running normally? Any interesting messages in datahub-gms container?
a
@better-orange-49102 The container I'm currently running has these • I didn't notice the change in the gms log, I'll do one more ingestion and see
@better-orange-49102 I take the ingestion command again, and get this log hint.
Copy code
14:54:04.380 [I/O dispatcher 1] INFO  c.l.m.s.e.update.BulkListener:25 - Failed to feed bulk request. Number of events: 1 Took time ms: -1 Message: failure in bulk execution:
[0]: index [chartindex_v2_1639755291895], type [_doc], id [urn%3Ali%3Achart%3A%28superset%2C11%29], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=index [chartindex_v2_1639755291895] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];]]]
14:54:07.401 [I/O dispatcher 1] INFO  c.l.m.s.e.update.BulkListener:25 - Failed to feed bulk request. Number of events: 1 Took time ms: -1 Message: failure in bulk execution:
[0]: index [system_metadata_service_v1], type [_doc], id [xZq5bCeBoq6LZao5f07yMg==], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=index [system_metadata_service_v1] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];]]]
14:54:10.412 [I/O dispatcher 1] INFO  c.l.m.s.e.update.BulkListener:25 - Failed to feed bulk request. Number of events: 1 Took time ms: -1 Message: failure in bulk execution:
[0]: index [graph_service_v1], type [_doc], id [0nd6ah//pferG1suVO05Sg==], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=index [graph_service_v1] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];]]]
b
Seems like your ES service needs more resources. But I'm not very familiar with how to overcome this. Possibly, the supersets are not yet recorded inside ES indices and hence they do not return when u browse.
a
Ok, I'll check again. Thank you😄
e
If you are running docker locally, you may want to run 
docker system prune
to prune unused resources. I’ve seen that this error comes up when you are running out of disk space.
👍 1
o
It's almost certainly this ^ If you are building images locally you will run out of disk space very quickly using the default docker settings. ES bases its watermark amounts on the total space Docker has allocated and cuts off at high percentages. You may need to include the -a flag to clear some dangling images, but note that this will require a re-pull/rebuild. You can get your current disk usage by running 
docker system df
(with optional -v for more info) and can increase how much disk you have allocated to docker in the Docker Desktop UI settings
a
@early-lamp-41924 thanks
@orange-night-91387 thank you
👍 1
Open in Slack
PreviousNext
Powered by