Hello Everyone, Anyone worked with AWS Python SDK Boto3? I am using that in my project and want to upload files to s3 in threads, the application is hosted as AWS Lambda Function. But the issue is, I am unable to upload files once I host that function, works fine in local. Please have a look at this question on SO: https://stackoverflow.com/questions/65223145/multi-threading-upload-in-aws-lambda-not-working-with-s3-and-boto3

Can you paste the Cloudwatch logs you got while executing this lambda function? And what's the memory limit provided for that lambda function?

Can you also try uploading the same objects without the Thread?

@swift-printer-82590 Memory limit is set to 512MB and execution time is 30 Sec.

That's what I suspected. So, according to the code you posted, you are not waiting for the thread to finish and returning right after starting the started. There should be a

thread.join()

after you start the thread. After you return, AWS Lambda has finished execution of the main routine and will halt the process. The reason it is working for you locally maybe because you are running a Flask server using

python app.py

. QQ: Why is the need here for a single thread? Ideally, all of the code inside the for loop should be in their own thread to leverage concurrency.

If I use

thread.join()

then I have to wait till I get the thread gets executed. Which I dont want, I want to start the thread and return the response and let the thread running in background and upload csv file to s3

I don't think that is going to work with Lambda. How will AWS Lambda's execution engine know that you are running a background thread and how will it know to kill the thread after the Lambda request-response cycle has been fulfilled?