❓ Opinions wanted: We’re building a serverless user activity notifications platform. We currently have the following setup:

Webhook -> Lambda -> EventBridge -> Kinesis Stream -> Kinesis Firehose -> S3 Bucket

It’s tracking a stream of user activity across a site and saving it to S3. It’s missing 1-2 pieces but I’m not 100% sure how to work with firehose data in S3. Essentially, each user “might” have an associated “owner” (we need to call out to check) and IF they do… we want to send that owner a daily digest of that user’s activity on the platform for proactive customer support. Which services would be good to tac on the remaining functionality of parsing s3 data to get that day’s digest and send it out to the owners?

Checkout AWS Athena, not sure if it'll make sense for exactly what you're trying to do

Yeah, maybe also checkout adding a Firehose Lambda processor. https://docs.aws.amazon.com/firehose/latest/dev/data-transformation.html

Also curious what's going on in the middle steps vs doing webhook -> lambda -> s3bucket

@thdxr Its basically whatever we want it to be. The challenges are that our data is in two separate services with their own APIs. (ex Service1, Service2) Service1 • knows what activity our users are up to • webhook that is called per user activity (page view, file DL etc) Event data is small but could get approx 2-5/s events per user and we have over 1000 users Service2 • hosts a list of users that are VIP and their internal “owner” *Dedup/match on user email Goals: 1. Send a daily digest of user activity to owners, ONLY for VIP users. 2. Eventually having #1 be a realtime app would be even better. Hence the steams.