I am trying to plug in an AWS Kinesis data stream, but I am unable to get authenticated because we use temporary credentials, which requires an AWS_SESSION_TOKEN to be given. Is there a way to give a session token along with the key and secret? Here is part of my table config.

"streamConfigs": {
          "streamType": "kinesis",
          "stream.kinesis.topic.name": "stream-name",
          "region": "us-east-1",
          "accessKey": "sdfsdf",
          "secretKey": "sdfdsf",

will this AWS_SESSION_TOKEN keep changing?

Yes, looking into it

so set

AWS_SESSION_TOKEN

this environment variable and remove fields

accessKey

and

secretKey

from tableConfig? Would you please add this to the docs on the various ways to access https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion/amazon-kinesis @User

added.

Thank you! I have given my keys to the container and am getting this error now:

"shaded.software.amazon.awssdk.core.exception.SdkClientException: Unable to load an HTTP implementation from any provider in the chain. You must declare a dependency on an appropriate HTTP implementation or pass in an SdkHttpClient explicitly to the client builder."

Given that this deployment is on EKS, I'm confused why it wouldn't automatically have credentials.

I think this is due the shaded jar issue that amazon-kisnesis package doesn’t load the default HTTP implementation

Yes, seems like that. Without shading though, it might cause issues with maven enforcer.