https://datahubproject.io logo
Join Slack
Powered by
Hello, can I set the location of the `policies.jso...
# troubleshoot
s
Hello, can I set the location of the 
policies.json
with a variable, i.e. in the docker-compose.yml?
b
You cannot - it will always be mounted in the resources directory, as that's where we do the classpath scan from the Java application
Thus you'd need to change the file mounts themselves to provide a custom policies file mounted at the expected location
Which should be....
Copy code
/datahub/datahub-gms/resources/policies.json
šŸ‘ 1
t
@bitter-dusk-52400 FYI
šŸ‘ 1
@big-carpet-38439 cc: @bitter-dusk-52400 Is this still the correct location? We have added a custom file in this location for our GKE deployment, but even after recreating all pods (we understood that policies.json will only be used on first boot) the initial policies are unmodified and the file does not seem to have been used. As for the actual changes in our custom policies.json, there is only one. 1. Change ā€œstateā€ of ā€œAll Users - All Platform Privilegesā€ from ā€œACTIVEā€ to ā€œINACTIVEā€ FYI We have been able to make use of custom user.props file successfully.
b
Ah got it
So this is marked as an editable policy
Meaning it wonā€™t be overwritten each time the service boots
So that users can edit in the UI
t
How should we alter the configuration to make sure this is disabled? Should we also set ā€œeditableā€ to ā€œfalseā€?
b
yeah that should be it
t
Got it. Weā€™ll try that. Thanks John.
@big-carpet-38439 Sorry to keep bothering you. We have tried the suggestion but it did not apply the custom policies. This is the only think holding us back from releasing internally. Are there any ways we can debug or any other options we can try?
FYI we redeployed the whole of datahub on GKE to make sure that there were no existing state to prevent intake of the custom policy.
b
Still no custom policies got in?
Seems that way yes
t
Yeah, it seems like the file was ignored.
Just changed the all-users, but in future we will add others.
b
Hm. Seems the thing got ignored. Can you control policy through the UI?
I mean - why do you need to change policies.json?
t
It is possible to do so.
b
You can deactivate the policy via the UI, right? Is that the goal?
(Also note that if you upgrade 0.8.38 you donā€™t need user.props anymore)
t
Itā€™s better for us to control policies via configuration and have this updated when necessary.
b
I see. So where are you mounting the custom file?
You can still manage using an ingestion recipe btw
Via the ā€˜fileā€™ source
t
As you wrote above
Copy code
/datahub/datahub-gms/resources/policies.json
Ingestion recipe could be another option! Hopefully I can solve why the file is not being taken in.
b
Got it. We donā€™t strongly recommend this because we update this file when new privileges become available in upgrades
And if you want the root user to get those automatically itā€™s better to keep this one separate
t
Ah ok. What is the recommended solution?
Yeah root defaults are fine as is
b
Recommended is either do this via the UI, GraphQL api, or File source for DataHub ingest CLI
if you use the file source to ingest, you can simply produce MCP events against the dataHubPolicyInfo aspect!
t
I see. Where could I find more about the ingest recipe? Is there an example on the site?
We found the necessary information. THanks for your help John.
b
Wonderful
e
Hello @big-carpet-38439 I am trying to ingest a custom policies.json. The error I'm getting is that the ingestion cron can't find the path of 
/datahub/datahub-gms/resources/policies.json
My recipe is below
Copy code
source:
  type: file
  config:
    # Coordinates
    filename: ../policies.json

sink:
  type: file
  config:
    filename: /datahub/datahub-gms/resources/policies.json
I want to disable the UI's ingestion so no users can use it. I'm trying to do so with a custom policies.json. Is this the best approach?
g
Hi @elegant-nightfall-29115 You can create platform policy from DataHub Portal and allow all privileges except managed ingestion and apply that policy to all user
Please find the attached demo for the same
datahub-policy-creation-demo.webm
e
Thank you for the demo. I followed along exactly and my admin role still has access to "Ingestion" after signing back in. What role did your user have in that demo?
g
The role is editor. You can set role to 
No Role
and manage the privileges from policy
e
Okay well I need to disable UI Ingestion for all roles. I'm not sure that possible through changing policies through the UI. What would you recommend to accomplish that?
g
Hi @elegant-nightfall-29115 Those roles are locked, non editable, However you can set GMS environment variable 
UI_INGESTION_ENABLED=false
to disable managed ingestion for all users.
e
Thank you for the suggestion @gentle-hamburger-31302 but I've already tried that and commented here about how it didn't work. @big-carpet-38439 were you able to reproduce this? My configuration is below
Copy code
datahub-gms:
  enabled: true
  image:
    repository: linkedin/datahub-gms
    tag: "v0.9.5"
  extraEnvs:
    - name: UI_INGESTION_ENABLED
      value: "false"
g
Hi @elegant-nightfall-29115 I tried this flag on 
docker-compose.quickstart.yml
with quickstart command and it is working fine. Please find the screenshots In you case 
UI_INGESTION_ENABLED
is recommended way. please raise your issue related to 
UI_INGESTION_ENABLED
on #all-things-deployment channel. The deployment team will help you to find out root cause.
Open in Slack
PreviousNext
Powered by