Hi, I am trying to use the Flink dynamodb connecto...
# troubleshooting
c
Hi, I am trying to use the Flink dynamodb connector to use dynamodb as a sink, however I noticed that it only supports PUT and DELETE operations and I am wondering if there is a plan to support UPDATE operations too?
@Pedro Mázala have you encountered this before? The alternative is to do a getItem and putItem , I think
p
Hey there! Based on the docs tags I'd say update should be available seeing these tags (
upsert mode
) Are you sinking it through SQL or StreamAPI? https://nightlies.apache.org/flink/flink-docs-stable/docs/connectors/table/dynamodb/
👀 1
c
I am using StreamAPI
p
IIRC PUT on Dynamo also works to update based on primary key, no? Talking more about Dynamo than Flink
Update works like updating a subset of fields. In the other hand, put will just override as a upsert.
From AWS docs
PutItem
Creates a new item, or replaces an old item with a new item.
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html
c
So in the docs it says "completely replaces the item"
So if put item is missing the other field in the value, those will get overwritten
That's how I understand it
Let me do a quick experiment and get back to you
Yeah I used put-item and it overwrites and does not update the target field
p
Got it. then the update is what you need indeed
c
Yeah 😀 Any ideas/recommendations? I was thinking of doing a get and put, but I think implementing update in the connector would be helpful to anyone else too
p
Indeed. Doing the get/put or even holding some state on flink would be almost the same effort than creating a update I think. Perhaps it's good to speak with whomever manages the dynamodb connector
c
Awesome, you're right. Thank you for your time and effort
Do you know who manages that?
Just look at the git contributors page?
actually I found a better solution, I could implement a custom
SinkFunction
and I could write the update logic in it, it was actually easy
🙌 1