Hi, I'm trying to run a recipe for ingesting metadata from Feast:

source:
  type: feast
  config:
    core_url: <s3://featurestore/dev/feature-store-dev.db> 
    env: "DEV"
    use_local_build: False
sink:
  type: "datahub-rest"
  config:
    server: "<http://datahub-datahub-gms.datahub.svc.cluster.local:8080>"

When I run the recipe, I get the following "ConnectionError":

docker.errors.ContainerError: Command 'python3 ingest.py --core_url=<s3://featurestore/dev/feature-store-dev.db> --output_path=/out.json' in image 'acryldata/datahub-ingestion-feast-wrapper' returned non-zero exit status 1: b'Traceback (most recent call last):\n  File "/usr/local/lib/python3.8/site-packages/feast/grpc/grpc.py", line 48, in create_grpc_channel\n    grpc.channel_ready_future(channel).result(timeout=timeout)\n  File "/usr/local/lib/python3.8/site-packages/grpc/_utilities.py", line 140, in result\n    self._block(timeout)\n  File "/usr/local/lib/python3.8/site-packages/grpc/_utilities.py", line 86, in _block\n    raise grpc.FutureTimeoutError()\ngrpc.FutureTimeoutError\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "ingest.py", line 138, in <module>\n    cli()\n  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "ingest.py", line 26, in cli\n    tables = client.list_feature_tables()\n  File "/usr/local/lib/python3.8/site-packages/feast/client.py", line 683, in list_feature_tables\n    feature_table_protos = self._core_service.ListFeatureTables(\n  File "/usr/local/lib/python3.8/site-packages/feast/client.py", line 134, in _core_service\n    channel = create_grpc_channel(\n  File "/usr/local/lib/python3.8/site-packages/feast/grpc/grpc.py", line 51, in create_grpc_channel\n    raise ConnectionError(\nConnectionError: Connection timed out while attempting to connect to <s3://featurestore/dev/feature-store-dev.db>\n'

I'm using datahub 0.8.17 and feast 0.14.1. In "core_url" I configured the feast featurestore .db file. I've seen in the Feast documentation that from version +0.10 there have been changes in Feast core:

"Feast Core was replaced by a file-based (S3, GCS) registry: Feast Core is a metadata server that maintains and exposes an API of feature definitions. With Feast 0.10, we've moved this entire service into a single flat file that can be stored on either the local disk or in a central object store like S3 or GCS. The benefit of this change is that users don't need to maintain a database and a registry service, yet they can still access all the metadata they had before."

I don't know if I am setting core_url correctly or will I need a Feast version lower than 0.10 with Feast core. Does anyone have any ideas?

@chilly-holiday-80781 Any ideas?

Yep – Feast 0.10+ doesn't use Feast Core anymore, so we need to update our connector to this new db-based format.