Hi all :wave::skin-tone-5: can anyone please help ...
# ingestion
c
Hi all 👋🏾 can anyone please help with the following error when ingesting LookerML data? I have been following this documentation
My yml file:
I’m able to get to the right folder from a browser
with the same client I was also able to ingest looker dashboards
v
I have the same issue. I guess the question is whether the
base_folder
is supposed to be a Looker url path to projects or a local folder cloned from your LookML Git repo?
m
@victorious-garage-66835:
base_folder
is supposed to be a local folder cloned from your LookML Git repo
👀 1
1
the LookML connector parses files in your git repo directly
👀 1
1
c
Thanks @mammoth-bear-12532 for your suggestion haven’t had the chance to try it but will feedback once I do!
v
@mammoth-bear-12532 LookML ingestion works fine after cloning the projects to local folders. It is also great to see that Looker lineage now connects automatically to source Postgres and MySQL tables as well 👍 Thanks for your help!
🎉 1
c
Haven’t had the chance to test this week but glad to hear that its working for you Amit! A thought I had which you might be able to clarify @mammoth-bear-12532 is why does the project needs to be available locally if we using the API client?
b
@shy-island-99768
s
Hi there, I'm having issues with the lookml ingestion. We have lookml code in codeCommit (not in gitHub). Could this be causing the problem? In the recipe you specify the looker credentials but not the ones for accessing the git repo. Should it be a public repo? Thanks for the help
m
Hi Adriana, integration with Looker is broken up into two segments. Looker -> ingests Dashboards, Charts, Explores (hits the looker API directly) Lookml -> ingests Views and lineage from Views to upstream tables (needs access to your lookml files on the local file system)
So it doesn't need to be github or public... you just need to be able to run the recipe on a machine that has the repo checked out
s
Hi Shirshanka, thanks for the answer. I was able to ingest the first segment with the looker recipe and now I have Dashboards, Charts, Explores metadata in datahub. My problem is with the Lookml module. From the documentation it seems you need to specify the github repo but for you answer it seems that only the location of the repo is important. Could you please explain me then what is the use of the repo in the recipe? Thanks
m
the github repo config is only used to render the
View in Looker
button for the views .. it is not a mandatory field
s
Hi there, I have a weird problem with the lookml ingestion. After Shirshanka suggestion I was able to ingest lookml metadata and connect a snowflake table with the view defined in looker. After a few day I reviewed the link and it seems lost. I've tried to reingest the lookml again with the same recipe but no result. The only thing I have change is that i've upgraded datahub to version 0.8.38 (before it was 0.8.36) . Any suggestions? Thanks
m
that is very strange .. what options are you using in the recipe for
connection_to_platform_map
? Are you using that or using the api directly
s
yep, very strange. i'm using the API
my recipe is something like this