Last night I was doing some deploys and the build ...
# seedr
Ross Coundon
10/29/2021, 8:02 AMLast night I was doing some deploys and the build log pane was updating very slowly, i.e. I got the notification that the build had completed but the build log was still only showing it as half way through.
Today, it's entirely empty!
Ross Coundon
10/29/2021, 8:05 AMNothing happening in Cloudformation console either
Ross Coundon
10/29/2021, 8:14 AMRetried, still the same.
Ross Coundon
10/29/2021, 8:19 AMRetried again, this time it started after a delay of about 6 minutes
j
Joe Kendal
10/29/2021, 9:07 AMHi would you say SEED is stable / safe for prod at this stage? We are moving to prod finally and would rather avoid any disasters. What resources / SST constructs are you using?
r
Ross Coundon
10/29/2021, 9:10 AMI've never had any issues before, it's always worked well and reliably. We use it to deploy a number of production workloads with APIs, lambdas, Cognito User Pools, Eventbridge crons, DynamoDB tables and streams, SQS queues etc
j
Joe Kendal
10/29/2021, 9:18 AMOkay, I feel as if they have raised not enough to support the growth in demand from users. I know a pair of engineers that recently raised $3m pre-seed for a similar startup with zero customers, zero previous startups, zero revenue, no launched product, few months into it. SEED/SST have a great community who want to use their solutions but have only raised $1m and it seems they are inundated with support issues, bugs, feature requests. It all sounds quite stressful for the founders who are working around the clock. But I think it will pay off, because community-led product development is key.
r
Ross Coundon
10/29/2021, 9:24 AMMaybe, but bear in mind there was an existing revenue stream from the Seed subscriptions. The more cash you take on, the more of the business you give away, not everyone goes for the biggest amount they can.
j
Joe Kendal
10/29/2021, 9:26 AMYes, I think the guys who raised $3m will regret it / higher chance of failure.
Joe Kendal
10/29/2021, 9:27 AMWe can tolerate some imperfections but the response time is what matters. Given that the error you’re experiencing here is fatal / production system down, it is concerning for business critical applications
r
Ross Coundon
10/29/2021, 9:30 AMIn this particular case, the issue went away. It's mostly a timezone thing, we're not paying for a particular SLA or follow-the-sun support, maybe that's something the team could offer at some point
j
Joe Kendal
10/29/2021, 9:30 AMFor example, AWS provides a level of customer service and system reliability unmatched by any other business. The main advantage of SEED as far as I can tell, is the shorter pipeline deployments. So why not just use CodePipelines with the easy @aws-cdk/aws-pipelines module
j
Jay
10/29/2021, 4:43 PM
Hey Ross we’ll have a look at your account for this today. It seems like the builds were getting pushed to our backup systems and that was taking long to respond.
Jay
10/29/2021, 5:19 PM
@Joe Kendal just to add some context around Seed. You might know this but it predates SST, it’s been around since 2018, growing and generating revenue, with hundreds of companies using it prod. We recently announced the $1m round but we had raised around a $1M before that as well and some more fundraise plans to share in a couple of months.
We don’t usually provide real-time support through Slack. We do have paid support SLAs but we definitely need to grow the team to keep up with all the activity that you are noticing. That’s going to be our focus for the coming months.
And yeah it is stressful as founders! But that’s part of the job!
r
Ross Coundon
10/29/2021, 6:50 PMThanks Jay, it sorted itself out after a while. I guess, from a feedback perspective, if possible, it'd be good to show some kind of feedback in the build log to say things are failing over or something's happening. The blank screen just freaked me out a bit!
j
Jay
10/29/2021, 6:53 PM
Hmm yeah that’s a good point /cc @Frank
f
Frank
10/30/2021, 6:58 AM
Yeah agreed. This was likely the case of deployed log propagation from ECS to CloudWatch. We should show at least show some sort of status.
Frank
10/30/2021, 6:59 AM
@Ross Coundon just to clarify, was this a case where it took the build long to start up, or has the build completed but the log was empty?
Frank
10/30/2021, 7:00 AM
Can you also DM me a link to the build?
r
Ross Coundon
10/30/2021, 7:04 AMHI Frank. The first time when it was blank, nothing happened at all and nothing was happening in Cloudformation either looking via the console and I cancelled it.
The next time it sat blank for 6 minutes-ish and then started building.
The night before the build logs had been very slow to print i.e. I got a Slack notification that the build had finished a minute or two before it showed in the build logs
f
Frank
11/02/2021, 4:19 AM
Got it! Thanks for the details @Ross Coundon. There are two problems here:
1. Builds are experiencing a delay in starting up: we are seeing an increase amount of SST builds on SEED. We are in the middle to expanding the build server cluster. We should have this resolved very soon.
2. Build logs are delayed: occasionally logs from AWS ECS are slow to show up on CloudWatch. This doesn’t happen too often.
In either case, I totally agree with you that we should be transparent on what is happening.
r
Ross Coundon
11/02/2021, 7:11 AMMakes sense, thanks Frank. Thanks for being transparent here too