https://serverless-stack.com/ logo
Join Slack
Powered by
Any recommendations for higher level TS interfaces...
# random
r
Any recommendations for higher level TS interfaces to dynamodb than DocumentClient?
t
ElectroDB!
I tried em all and electroDB is the only one that's truly TS native
r
Nice 😎
t
just an fyi the learning curve seems a bit high at first but know you don't have to really understand the services concept to get started
just start defining your schemas and go
r
Awesome, I'll check it out
j
@thdxr have u done any pagination on ElectroDB?
t
haven't gotten to it yet but I did see it supported it
j
yeah ive been playing with onetable, ElectroDB looks to have a similar api but yet to find a “production” way of sending the pager over http
t
I'm asking him
t
Heya @Jono Allen! I can definitely answer any question about ElectroDB. Can you a bit deep in detail what you envision for "a 'production' way of sending the pager over http"?
I will check back in this thread tomorrow morning, but here is the start of an answer: There are three different options for the structure of a "pager": 
raw
, 
item
, and 
named
. The 
raw
option is the easiest to grep; it is the 
LastEvaluatedKey
as it was returned in the DocumentClient results. The 
item
and 
named
options are pretty much the same concept: ElectroDB will use its knowledge of key structures to disassemble the 
LastEvaluatedKey
into its constituent parts. For example, if 
LastEvaluatedKey
looks like this:
Copy code
{ 
  pk: '$taskapp#country_united states of america#state_oregon', 
  sk: '$offices_1#city_portland#zip_34706#office_mobile branch'
}
ElectroDB knows how the key is formed un-parse the key into its individual parts:
Copy code
{
  "city": "portland",
  "country": "united states of america",
  "state": "oregon",
  "zip": "34706",
  "office": "mobile branch"
}
When I first started building ElectroDB I was designing a Transaction Warehouse for a Credit Union platform. Two challenges from that project when using DynamoDB was the following: 1. The key expressions applied to a query had no bearing on the 
LastEvaluatedKey
returned from DynamoDB. This had the unfortunate consequence of leaking keys to the frontend which was likely to include the accountIds and transactionIds of other members. 2. Some attributes of a key, for example a 
UserId
, should never come from the user themselves. Those types of values should come from a session or some other authorized source. In both cases breaking out the individual attributes from a 
LastEvaluatedKey
was helpful in mitigating these issues. It allowed me to add additional logic around what was exposed to the user, and what was required to be supplied by the user to paginate. I can go into detail about any of these concepts further if you have any questions. There are some caveats to the above worth discussing. I am interested in hearing more about your needs, and possibly extending ElectroDB to fit those as well 👍 On the roadmap I have felt the addition of a fourth option (
cursor
) that returns a single string would be very helpful for many users. Lastly, I also have a WIP branch that adds (non-breaking) AsyncIteration to the existing pagination API for a nicer DX.
t
that's a really good point, I never considered the vector of sending a falsified paging token to the backend to read other data
j
Thanks for your in-depth reply! I haven’t used electroDB yet but given this answer I’m going to migrate a project over and i have been struggling/not happy with pagination using one-table. Currently I am base64 encodeing the 
pk
and 
sk
and then encrypting it. My main issue is if the key changes the pagination breaks. When i say “production” what i was meaning was something that can exist in url, that is safe and can’t pose any risks but is also reliable/backwards compatible and safe to future changes. (ie key changing etc) I really like the idea of adding a 
cursor
option as it solves the issue of not having a single reference which is supported by the data layer and the application code doesn’t need to add any abstractions to provide a reference to the frontend.
t
To be clear, are you saying you base64 and encrypt the LastEvaluatedKey? This is great input into the range of needs you'd need from a cursor, it's making me wonder if allowing users to define some sort of serialize/deserialize functions on the model to allow users implement their own. Could you expand on your issue with “keys changing”?
j
Correct base64 encoding the LastEvaluatedKey to be supplied to the ExclusiveStartKey. I think being able to override the cursor serialize/deserialize would be a great feature. My main reason for encrypting is so no schema data is leaked and can be tampered with. If the cursor was able to hold the LastEvaluatedKey without containing any schema information that would be an ideal situation. Not sure how possible that is with single table design though? Sorry that was poor wording from me on 
(ie key changing etc)
I was referring to the secret changing on the encrypted key would break the pagination. Like when working with distributed applications like offline mobile apps. I guess to summaries what I’m saying is I’m after a stable, non sensitive LastEvaluatedKey that can’t be used to comprise the system.
t
@Jono Allen I'll look to add the override here soon then 👍
If anything else comes up while using the library definitely feel free to hit me up anytime
j
@Tyler Walch really appreciate it, will do!
Open in Slack
PreviousNext
Powered by