https://linen.dev logo
Join Slack
Powered by
Hello, what is the task/process that generates `ai...
# contributing-to-airbyte
c
Hello, what is the task/process that generates 
airbyte-integrations/bases/base-python/airbyte_protocol/models/airbyte_message.py
? For some reason, when i run my 
./gradlew build
it ends up with a new ownership by 
root:root
user and makes my command fails afterward because of permission access... ⁉️
u
from 
airbyte-integrations/bases/base-python/build.gradle
u
generateProtocolClassFiles
u
Does that mean your gradle is running as root somehow?
u
no
u
can you add 
whoami
in 
generate-protocol-files.sh
and run 
./gradlew generateProtocolClassFiles
u
(maybe it's the docker daemon ran by root though?)
u
docker isn't creating this file though
u
oh nvm
u
completely forgot
u
u
maybe we need to add 
--user "$(id -u):$(id -g)"
to the docker call?
u
are you also getting everything created as root (logs, etc) if you run an integration through the airbyte app?
u
the 
whoami
returns the user not root in the script shell
u
i guess yes?
Copy code
ls -l /tmp/airbyte_local/output_data/exchange_rate.csv 
-rw-r--r-- 1 root root 13447 Oct 23 19:11 /tmp/airbyte_local/output_data/exchange_rate.csv
u
if you add 
--user "$(id -u):$(id -g)"
to just the docker call in 
generate-protocol-files.sh
does it fix your problem?
u
yes that's much better
u
@charles do you think we should add this user scoping everywhere?
u
in my opinion, definitely (for dev users not on mac os...) and the Dockerfiles should probably include the recommended snippet from your linked post too then:
Copy code
ARG USER_ID
ARG GROUP_ID

RUN addgroup --gid $GROUP_ID user
RUN adduser --disabled-password --gecos '' --uid $USER_ID --gid $GROUP_ID user
USER user
u
BTW this is also why your build failed here: https://github.com/airbytehq/airbyte/pull/684/checks?check_run_id=1299333311
Copy code
Task :airbyte-integrations:bases:base-python:blackFormat FAILED
[python] .venv/bin/python -m black . --line-length 140
	 error: cannot format /home/runner/work/airbyte/airbyte/airbyte-integrations/bases/base-python/airbyte_protocol/models/airbyte_message.py: [Errno 13] Permission denied: '/home/runner/work/airbyte/airbyte/airbyte-integrations/bases/base-python/airbyte_protocol/models/airbyte_message.py'
u
why did that just start happening though?
u
because it never caused problems until blackFormat needs to rewrite the generated python file and can't because it's owned by root
u
ahh
u
good point
u
I'm going to try adding this to the generation
u
does that snippet replace haing to do --user everwhere?
u
no you should do both
u
no it forces you to set user and group everywherre
u
^^
u
I guess maybe we don't care as much for everything in-app?
u
code tooling seems highest priority
u
but it does seem reasonable that people want to run airbyte as a user, just not as high priority
u
actually this is specifying at build time
u
which is a bit strange
u
I don't think we actually want that since people will often be pulling these images
u
ah 
generateProtocolClassFiles
is working the first time but not the second
u
it wouldn't be too bad to add these fixes at least to the docker build/run for 
generate-protocol-files.sh
, right?
u
I got the build passing by explicitly excluding regeneration in the later gradle runs
u
So it's merged now
u
What I'm most worried about the fixes is the fact that it ties UIDs/GIDs to the artifact (the image) which seems strange.
u
It's fine for something like code generation but it doesn't feel right for anything we'd publish?
u
for the code generation code, we provide the actual ID when calling the 
docker run
not building the image
u
true. for code generation especially it probably doesn't need more than the --user flag
17 Views
Open in Slack
PreviousNext
Powered by