hi! i faced with an error

Caused by: com.facebook.presto.hive.s3.PrestoS3FileSystem$UnrecoverableS3OperationException: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: ; S3 Extended Request ID: ; Proxy: null), S3 Extended Request ID:  (Path: <s3p://myapp/flink-checkpoints/98d9c76cf8bb4fae909c4bcd0a45c271/chk-27/b89524c6-bf57-425d-82c7-846861f085c3>)
	at com.facebook.presto.hive.s3.PrestoS3FileSystem.lambda$getS3ObjectMetadata$5(PrestoS3FileSystem.java:677)
	at com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:139)
	at com.facebook.presto.hive.s3.PrestoS3FileSystem.getS3ObjectMetadata(PrestoS3FileSystem.java:664)
	at com.facebook.presto.hive.s3.PrestoS3FileSystem.getS3ObjectMetadata(PrestoS3FileSystem.java:648)
	at com.facebook.presto.hive.s3.PrestoS3FileSystem.getFileStatus(PrestoS3FileSystem.java:353)
	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1760)
	at com.facebook.presto.hive.s3.PrestoS3FileSystem.create(PrestoS3FileSystem.java:398)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1195)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1175)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1064)
	at org.apache.flink.fs.s3presto.common.HadoopFileSystem.create(HadoopFileSystem.java:154)
	at org.apache.flink.fs.s3presto.common.HadoopFileSystem.create(HadoopFileSystem.java:37)
	at org.apache.flink.core.fs.PluginFileSystemFactory$ClassLoaderFixingFileSystem.create(PluginFileSystemFactory.java:170)
	at org.apache.flink.core.fs.SafetyNetWrapperFileSystem.create(SafetyNetWrapperFileSystem.java:130)
	at org.apache.flink.core.fs.EntropyInjector.createEntropyAware(EntropyInjector.java:76)
	at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:453)

i am working with aws s3 and here is list of allowed actions

"GetObject",
"GetObjectAcl",
"GetObjectVersion",
"DeleteObject",
"DeleteObjects",
"HeadObject",
"ListObjects",
"ListObjectsV2",
"PutObject",
"PutObjectAcl",
"PutObjectRetention",
"UploadPart",
"ListBucket"

Flink successfully created a recovery in S3 but cannot create a checkpoint. It seems that I need to extend the list of allowed actions to be able to store checkpoints, but I cannot determine which actions to allow. Does anybody have any familiarity with this problem or any ideas on how to fix it?

What bucket

arn

did you use?

i just copied it from aws s3 arn is

arn:aws:s3:::anomaly-analyzer-066381753496-eu-central-1

uri is

<s3://anomaly-analyzer-066381753496-eu-central-1>

The Resource need to contain 2

arn

one for the bucket and another for the content of the bucket: ex:

{
            "Sid": "AllowFlinkToExampleBucket",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
//All your other action permissions
            ],
            "Resource": [
                "arn:aws:s3:::example-bucket/*",
                "arn:aws:s3:::example-bucket"
            ]
        }

oh, i see) yes i have configured it exactly as in your example bucket and bucket/* with actions listed above

you might need th multipart upload one, if I recall presto uses it

but looks like it works for example for recovery, cause flink have created flink-recovery/default/submittedJobGraph4105772250e0 file

Can you confirm that it does work when using

s3:*

as an action?

Yes, I will check it. I think it will be the most optimal solution, and if the problem is resolved, we will only need to understand what permissions were missing. I will check it tomorrow because we need one approval 💩)

Hi Dima, were you able to fix this?

hi! maybe i am late, but yes, it was worked ok, but after several weeks i faced the same issue and at that moment did not investigate what the problem was