https://serverless-stack.com/ logo
Join Slack
Powered by
Any thoughts on what would cause an S3 PutObject c...
# general
r
Any thoughts on what would cause an S3 PutObject call using the JavaScript SDK to an SST defined bucket to return success but write a zero byte file? The code is pretty simple
Copy code
const putObjectParams: PutObjectRequest = {
  Bucket: bucket,
  Key: filename,
  Body: 'arse',
};
const result = await s3.putObject(putObjectParams).promise();
I see the object in S3 via the console but it's zero bytes
f
Try 
Body: Buffer.from('arse', 'utf8')
Not sure if that’s the issue
Copy code
await s3.putObject({
  Bucket: process.env.MY_BUCKET,
  Body: Buffer.from(JSON.stringify(data, null, 2), 'utf8'),
  Key: "path/to/file",
  ContentType: 'application/json',
}).promise();
This is what i have in my code
r
Thanks Frank, still same result. I've actually stripped this back to debug the problem. I was originally creating and writing a zip file, and so was sending a buffer. Decided to make it simpler and just write some text but this is always the result
How should I be providing privileges to write to the bucket in the SST stack? I currently have:
(I'm wondering if it could be allowing creation of an object but not write to it - clutching at straws though)
f
This should’ve given the necessary permission
Copy code
permissions: [statsCsvBucket],
r
Ok, that's what I thought
Hmm
How are you creating your s3 client?
f
Copy code
AWS.config.update({region:'us-east-1'});
const s3 = new AWS.S3();
Maybe create a bucket manually on S3, and see if u can upload to there?
Just to make sure SST isn’t creating the bucket in any weird way. I’m sure its not lol
r
No streams, I'll try a manual one
stepping through the aws-sdk code I can see the body has content
So...I've created it manually and the object gets put correctly
So could actually be a bug in the SST S3 bucket creation?
f
Can you manually upload a file to the SST S3 bucket through the S3 console?
r
yeah, that works
I'm just trying to set permissions in a different way via a Policy Document instead
f
yeah.. seems like permission related
r
I don't seem to be able grant permissions to PutObject to the SST created bucket, whereas this works when it's a manually created bucket via the console.
Copy code
const testS3Arn1 = 'arn:aws:s3:::test-stats-bucket';
const testS3Arn2 = 'arn:aws:s3:::test-stats-bucket/*';
const s3Policy = new PolicyStatement({
  effect: Effect.ALLOW,
  actions: ['s3:*'],
  resources: [testS3Arn1, testS3Arn2],
});
Then:
Copy code
const saveStatsAsCsvHandler = new Function(this, 'saveStatsAsCsvHandler', {
      handler: 'src/main/handlers/api.handleSaveStatsAsCsv',
      timeout: scope.local ? 30 : undefined,
      memorySize: 1024,
      environment: {
        ...environment,
        ...rebookingEnvironment,
      },
      permissions: [s3Policy],
      logRetention,
    });
with or without
Copy code
saveStatsAsCsvHandler.addToRolePolicy(s3Policy);
Always results in Access Denied
f
hmm.. if u look at the CFN template in 
.build/cdk.out
, do you see the IAM role has the added policy?
r
If I do the same but the two arns contain that of the manually created bucket, it works
In which scenario?
f
Can you compare the CFN template of granting permissions to the manually created bucket:
Copy code
permissions: [s3Policy],
vs granting permission to the SST S3 bucket
Copy code
permissions: [s3Bucket],
while keeping everything else the same.
r
In the last scenario with manually setting the permission using the ARN, I see
Copy code
{
              "Action": "s3:*",
              "Effect": "Allow",
              "Resource": [
                "arn:aws:s3:::test-stats-bucket",
                "arn:aws:s3:::test-stats-bucket/*"
              ]
            },
ok, 2 mins
Right with
Copy code
permissions: [s3Policy]
I see
Copy code
{
              "Action": "s3:*",
              "Effect": "Allow",
              "Resource": [
                "arn:aws:s3:::test-stats-bucket",
                "arn:aws:s3:::test-stats-bucket/*"
              ]
            },
With
Copy code
permissions: [s3Bucket]
I see
Copy code
"Resources": {
    "statsCsvBucket9F7B4665": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketName": "test-stats-bucket"
      },
      "UpdateReplacePolicy": "Retain",
      "DeletionPolicy": "Retain",
      "Metadata": {
        "aws:cdk:path": "test-omw-ofsc-be-OmwOfscBeStack-test/statsCsvBucket/Bucket/Resource"
      }
    },

...

{
              "Action": "s3:*",
              "Effect": "Allow",
              "Resource": [
                {
                  "Fn::GetAtt": [
                    "statsCsvBucket9F7B4665",
                    "Arn"
                  ]
                },
                {
                  "Fn::Join": [
                    "",
                    [
                      {
                        "Fn::GetAtt": [
                          "statsCsvBucket9F7B4665",
                          "Arn"
                        ]
                      },
                      "/*"
                    ]
                  ]
                }
              ]
            },
f
Hmm that looks right. CFN will resolve the arn at deploy time.
I need to step away now, i can give it a try tonite and see if I can replicate the issue
r
no problem, i need to get some sleep too, thanks Frank
f
Hey @Ross Coundon I created a test SST app with 1 Api and 1 Bucket https://github.com/fwang/sst-s3-test
The Api write a file to S3, similar to the code u shared above.
This worked for me… maybe give it a try and see if any luck.
r
I'll give it another go, thank you
This is still happening for us but one thing I've noticed is that with bucket versioning turned on the file is originally written with contents but then something changes it to be zero bytes. The only other thing that's happening to the file is the generation of a presigned URL which should touch it but I'll carry on digging
It turns out that this was, in fact, a coding error on our part, The code that created the signed URL also created an empty file which in effect overwrote what was there. Although I still can't understand why it seemed to work when outside of SST and not within but I'm pretty certain this isn't an SST problem. Thanks for looking into it, sorry it was a wild goose chase
f
Oh nice! Glad u caught it.
2 Views
Open in Slack
PreviousNext
Powered by