If a request is timing out you probably want to as...
# generalt
Thomas Ankcorn
11/19/2021, 7:19 PMIf a request is timing out you probably want to assume it's a dud after a few seconds as dynamodb is no slouch. You can set that with the aws-sdk. 150 writes is not much unless the objects are massive though
r
Ross Coundon
11/19/2021, 7:20 PMWe run them all in parallel, they don't time out themselves, the overall lambda timeout is what's getting exceeded
Ross Coundon
11/19/2021, 7:20 PMCloudwatch metrics suggest they're taking 5ms
t
Thomas Ankcorn
11/19/2021, 7:21 PMWhat percentage of the items are successfully written to the table?
r
Ross Coundon
11/19/2021, 7:22 PMAbout 80%
t
Thomas Ankcorn
11/19/2021, 7:24 PMAnd do you have a timeout value set when using the aws-sdk
Thomas Ankcorn
11/19/2021, 7:24 PMWait which language are you using?
r
Ross Coundon
11/19/2021, 7:24 PMNot explicitly, no
Ross Coundon
11/19/2021, 7:24 PMNodejs
Ross Coundon
11/19/2021, 7:25 PMWe have connection reuse turned on and maxSockets effectively set to 50 as per the docs
Ross Coundon
11/19/2021, 7:25 PMV2 sdk
t
Thomas Ankcorn
11/19/2021, 7:26 PMI will try to recreate because I'm interesed. We write quite a lot of data we pull from 30rd party api's to dynamo but we put it into S3 first as it's timeseries but low frequency and that gives us a cheaper historic record
r
Ross Coundon
11/19/2021, 7:27 PMThank you, that's really appreciated. We're using on demand capacity too
Ross Coundon
11/19/2021, 7:28 PMOur lambda runs every 5 mins with a 4.5 min timeout
Ross Coundon
11/19/2021, 7:44 PMLooks something like this:
Copy code
async someFunction(){
const savePlanSetPromises = someArray.map((dataset) => {
return SomeDataLayer.save(dataset).catch((err) => {
log.error(`Some error': ${(err as Error).message}`);
});
});
// Save each activity
let saveCount = 0;
const saveActivityPromises = activities.map((activity) => {
return SomeOtherDataLayer.save(activity)
.then(() => {
saveCount++;
})
.catch((err) => {
log.error(
`Some Error ${(err as Error).message}`,
);
});
});
await Promise.all(savePlanSetPromises);
const saveActivityResults = await Promise.all(saveActivityPromises);
return saveCount;
}