https://serverless-stack.com/ logo
Join Slack
Powered by
We are having serious memory issues with esbuild w...
# sst
j
We are having serious memory issues with esbuild when running our stack (its a decent size). How can we make this better? are there some config options to look out for? It causing vscode debug etc to crash. @thdxr / @Frank
we are running on node 12.x, is it worth going to 14.x?
t
Wow that's a lot. Hm I'm not sure what could be causing this
If this is on 
sst start
that esbuild process is what rebuilds your files on change. You can try upgrading node but I don't think that'll help
Does it immediately use a lot of memory? Or does it build up over time
j
a lot to start with then adds on each file save @thdxr
t
How much does it start with on sst start?
j
yes seen that issue, looks still to be open. What size stack are you guys testing on and what is your memory usage? @thdxr
t
let me see
f
@Jack Fraser which version of SST version are you using?
We fixed an issue in v0.35.2 a while back where 
aws-sdk
was being transpiled by esbuild, and since we are keeping the esbuild process around, the memory doesn’t get released.
@thdxr we start up 1 esbuild process for each Function handler, and we keep the esbuild process around for rebuild. When I was looking into the issue above a while back, these esbuild processes could take a log of memories, predominately b/c 
aws-sdk
is HUGE, so we are now force excluding it in the esbuild config. More details about my findings in the comment here https://github.com/serverless-stack/serverless-stack/pull/610/files
j
@Frank we are on the latest (0.46) build of SST
f
I see. A few follow on questions: • is the SST app (ie. sst.json) at the repo root? • is 
srcPath
configured for the functions?
j
@Frank Yes we use 
srcPath
when using 
addRoutes
api.addRoutes(this, {
"GET /settings/v1/settings": {
function: {
srcPath: "src/apis/settings",
handler: "get-settings.handler",
},
},....
we have multi sst.json in the project but we run debug as if one of the sst.json is the root. • root ◦ backend ▪︎ lib ▪︎ src ▪︎ sst.json ◦ another-sst-stack ▪︎ lib ▪︎ src ▪︎ sst.json our launch.json looks like ... 
{
"name": "SST (Start Backend Debug)",
"type": "node",
"request": "launch",
"cwd": "${workspaceRoot}/backend",
"runtimeExecutable": "${workspaceRoot}/*backend*/node_modules/.bin/sst",
"runtimeArgs": ["start", "--increase-timeout"],
"console": "integratedTerminal",
"skipFiles": ["<node_internals>/**"]
},
f
Thanks @Jack Fraser. I think I have a rough idea of what’s causing the memory usage. Just to double check 1 more thing, can you show me what’s in 
/backend/package.json
and what’s in 
/backend/src/apis/settings/package.json
?
Also are there 
package.json
files at 
/backend/src
and 
/backend/src/apis
?
j
Package.json just at /backend level
f
is there one at 
/backend/src/apis/settings/package.json
?
j
no package.json at 
/backend/src/apis/settings/package.json
or at 
/backend/src/apis/
our package at 
/backend/
f
Ah I see. Can you try changing ur code to this:
Copy code
api.addRoutes(this, {
      "GET /settings/v1/settings": {
        function: {
          handler: "src/apis/settings/get-settings.handler",
        },
      },....
So 
srcPath
is meant to point to the directory where 
package.json
sits.
I think this might fix the memory issue
j
oh
will try it
@Frank have made the change and no great luck. First it VScode pop up with two errors, first "it may take a while" on then "...terminated". Looking at moemory it got to around 4.8gb then 💥
f
can you try running 
sst start
directly in your terminal?
Let’s just to rule out VSCode for the moment
j
memory of esbuild is down to 8mb! 🥳 Saying that cant get a npm start to run as AWS is throwing 
Throttling: Rate exceeded
🙀
@Frank could that error have been caused by we have 20 stacks all trying to be deployed at once?
Have got it to deploy via npm start 🥳
using only 568mb of memory for esbuild now 🎉
and went down after a couple min of running
f
Oh nice! Did you get the 
Throttling: Rate exceeded
while running 
sst deploy
?
Are you still able to grab a screenshot lol?
j
Throttling: Rate
exceeded error was during 
npm start
Open in Slack
PreviousNext
Powered by