<@U6ZNGL2NM> Yeah i know about this but I am tryin...
# orm-helpj
Jscott388
04/16/2018, 2:57 AM@glen Yeah i know about this but I am trying to figure out the best way to fetch my data and insert the data. I am doing a real estate site, so I am not sure if the lambda function will handle all that data.
m
Mike Clymer
04/16/2018, 11:51 AMI am currently using async Django tasks in Lambda to collect, process, and store regulation data. You need to be aware of the payload limits for things like request bodies. https://docs.aws.amazon.com/lambda/latest/dg/limits.html
Mike Clymer
04/16/2018, 12:00 PMMy general strategy has been to break up the data fetching between async function invocations, dump it to S3, then signal intermediate data processing and storage task functions through S3 object creation events and SNS. I make sure to base64 encode the data, and compress it, as I am moving it around in S3 and passing to SNS.