https://linen.dev logo
Join Slack
Powered by
Hello <#C02CTC89LAX|>, I am reaching out to see if...
# community-support
j
Hello #C02CTC89LAX, I am reaching out to see if the official docker image has support for AWS Lambda. I am encountering an issue when running the image as a containerized lambda. The full error is in the thread, returning a Read-only file system error when creating /usr/app/edr_target. I've tested locally and able to generate a report and send to S3. My image is stored in ECR and using CodePipeline to orchestrate.
👀 1
Copy code
________                          __                  
/ ____/ /__  ____ ___  ___  ____  / /_____ ________  __
/ __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
/ /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
/____/   
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/elementary/cli/cli.py", line 27, in get_log_path
target_path = ctx_args[ctx_args.index(target_path_flag) + 1]
ValueError: '--target-path' is not in list
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/edr", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/elementary/cli/cli.py", line 60, in invoke
files_target_path = get_log_path(ctx)
File "/usr/local/lib/python3.10/site-packages/elementary/cli/cli.py", line 29, in get_log_path
os.makedirs(os.path.abspath(target_path), exist_ok=True)
File "/usr/local/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/usr/app/edr_target'
________                          __                  
/ ____/ /__  ____ ___  ___  ____  / /_____ ________  __
/ __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
/ /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
/____/   
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/elementary/cli/cli.py", line 27, in get_log_path
target_path = ctx_args[ctx_args.index(target_path_flag) + 1]
ValueError: '--target-path' is not in list
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/edr", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/elementary/cli/cli.py", line 60, in invoke
files_target_path = get_log_path(ctx)
File "/usr/local/lib/python3.10/site-packages/elementary/cli/cli.py", line 29, in get_log_path
os.makedirs(os.path.abspath(target_path), exist_ok=True)
File "/usr/local/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/usr/app/edr_target'
START RequestId: a6b154a4-4217-4b2a-8aac-1be29a0c40fa Version: $LATEST
RequestId: a6b154a4-4217-4b2a-8aac-1be29a0c40fa Error: Runtime exited with error: exit status 1
Runtime.ExitError
END RequestId: a6b154a4-4217-4b2a-8aac-1be29a0c40fa
REPORT RequestId: a6b154a4-4217-4b2a-8aac-1be29a0c40fa	Duration: 16501.26 ms	Billed Duration: 16502 ms	Memory Size: 128 MB	Max Memory Used: 67 MB
i
Hi @Jason Hamilton! We haven't tested the docker image against lambda. Based on the error above - I believe it's because the only writable path for lambda dockers is 
/tmp
, where we write by default to other locations. I think a possible workaround may be to override the following environment variables (I think this should be possible to set in the Lambda function):
Copy code
DBT_LOG_PATH=/tmp/logs
DBT_TARGET_PATH=/tmp/target
If still relevant, will be great if you can try and write here if it works!
j
Hi @Itamar Hartstein, Thank you for the suggestions! I've attempted to add the DBT paths to my environment variables in the lambda, ECS, and in the dockerfile as ARGs and I'm still seeing the same error trying to write to /usr/app/edr_target. I'm thinking this may run better as an ECS task rather than a containerized lambda?
Update on this: I have the container running as an ECS task using Fargate, but I'm running into issues running the actual commands in the container. It seems no matter what I run, the container doesn't recognize it. I can run this commands locally and get the expected outcome.
Copy code
________                          __
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ /
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /
                                                 /____/

Any feedback and suggestions are welcomed! join our community here - <https://bit.ly/slack-elementary>

2023-10-30 14:59:21 — INFO — Running with edr=0.12.0
Usage: edr [OPTIONS] COMMAND [ARGS]...
Try 'edr --help' for help.

Error: No such command 'send-report --aws-access-key-id MY_ACCESS_KEY --aws-secret-access-key MY_SECRET_ACCESS_KEY --s3-bucket-name MY_S3_BUCKET_NAME'.
It will recognize send-report as a command, but as soon as I add any arguments, I get no such command. I'm not sure what I'm missing. Is there any insight you can provide?
29 Views
Open in Slack
PreviousNext
Powered by