I am trying to make a kinesis autoscaler lambda ba...
# aws
t
I am trying to make a kinesis autoscaler lambda based on existing code basically update shard count based on incoming records alarm metric. During testing I notice something odd in using aws cli commands to get the number shards shown above. Basically describe-stream-summary says the OpenShardCount is one this seems like the right answer but describe-stream and list-shards report there are 4 shards. Which is correct ? Why are they not consistent ? Hope there is a kinesis expert here who can explain what is going on thanks
I think I understand the shards are not yet expired and are still readable but not writable because of retention period
a
Right. Not all shards are open. Anyway, there are off-the-shelf solutions for auto-scaling Kinesis streams, I would highly recommend using them instead of writing your own: https://aws.amazon.com/blogs/big-data/scaling-amazon-kinesis-data-streams-with-aws-application-auto-scaling/
t
@Alex Jurkiewicz Thanks I based my solution on that code https://github.com/aws-samples/aws-application-auto-scaling-kinesis from the article you linked to but found issues with both the cloudformation and the python lambda code. So I am improving it to make it more production ready.
👍 1