https://serverless-stack.com/ logo
#sst
Title
# sst
a

Adrián Mouly

06/06/2022, 7:31 PM
Hey guys, not sure what I changed, but during migration to SST v1, have this issue:
Copy code
ReferenceError: __dirname is not defined in ES module scope
This file is being treated as an ES module because it has a '.js' file extension and '/Users/amouly/Projects/yabble/yabble-serverless/.build/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.
t

thdxr

06/06/2022, 7:31 PM
are you using
__dirname
somewhere
particularly in your stacks code
a

Adrián Mouly

06/06/2022, 7:33 PM
Let me see.
No, I’m not using it.
What could be the problem?
t

thdxr

06/06/2022, 7:36 PM
ESM doesn't support
__dirname
. Can you send me
.build/lib/index.js
and the error doesn't give more info about the stacktrace?
a

Adrián Mouly

06/06/2022, 7:36 PM
It gives me the stack trace yes.
t

thdxr

06/06/2022, 7:36 PM
can you show me
a

Adrián Mouly

06/06/2022, 7:37 PM
Copy code
ReferenceError: __dirname is not defined in ES module scope
This file is being treated as an ES module because it has a '.js' file extension and '/Users/amouly/Projects/yabble/yabble-serverless/.build/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.
    at node_modules/app-root-path/index.js (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:100493:27)
    at __require2 (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:18:50)
    at node_modules/typeorm/connection/ConnectionOptionsReader.js (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:112285:51)
    at __require2 (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:18:50)
    at node_modules/typeorm/globals.js (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:161892:37)
    at __require2 (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:18:50)
    at node_modules/typeorm/index.js (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:163944:26)
    at __require2 (file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:18:50)
    at file:///Users/amouly/Projects/yabble/yabble-serverless/.build/lib/index.js:796582:28
    at ModuleJob.run (node:internal/modules/esm/module_job:185:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:281:24)
    at async file:///Users/amouly/Projects/yabble/yabble-serverless/.build/run.mjs:85:17

Reading cached notices from /Users/amouly/.cdk/cache/notices.json
t

thdxr

06/06/2022, 7:37 PM
hm why does typeorm get pulled into your stacks code?
a

Adrián Mouly

06/06/2022, 7:37 PM
Good question.
t

thdxr

06/06/2022, 7:37 PM
.build/lib/index.js
should be pretty small
a

Adrián Mouly

06/06/2022, 7:38 PM
It’s 25 mb.
😐
t

thdxr

06/06/2022, 7:40 PM
can analyze with this
Copy code
esbuild --minify --external:"@serverless-stack/*" --bundle --format=esm --target=esnext --platform=node ./stacks/index.ts --analyze=verbose
a

Adrián Mouly

06/06/2022, 7:41 PM
This is looking for my index for stacks, right?
t

thdxr

06/06/2022, 7:41 PM
yeah
a

Adrián Mouly

06/06/2022, 7:43 PM
What this output should have?
I see calls to typeorm from here.
Example…
Copy code
├ node_modules/typeorm/decorator/options/JoinTableOptions.js ───────────────────────────────────────────────────── 82b ──── 0.0%
   │  └ node_modules/typeorm/index.js
   │     └ node_modules/typeorm/index.mjs
   │        └ packages/src/data/mySqlDatabaseConnection.ts
   │           └ packages/src/data/index.ts
   │              └ packages/src/common/middleware/databaseMiddy.ts
   │                 └ packages/src/common/index.ts
   │                    └ infrastructure/src/yabble/surveyResponses/YabbleSurveyResponsesIngestionMetadataStack.ts
   │                       └ infrastructure/src/yabble/Yabble.ts
   │                          └ infrastructure/src/index.ts
t

thdxr

06/06/2022, 7:43 PM
can you trace to see where it comes from
a

Adrián Mouly

06/06/2022, 7:43 PM
Tis doesn’t looks right.
Yeah I think I found it.
t

thdxr

06/06/2022, 7:44 PM
ah yeah so this is a quirk that a lot of people get tricked by, esbuild does not treeshake re-exports
a

Adrián Mouly

06/06/2022, 7:44 PM
I have a middy for DB, in a shared package.
t

thdxr

06/06/2022, 7:44 PM
so if you have an index file, eg
common/index.ts
it'll include everything in there even if you're just importing 1 thing
a

Adrián Mouly

06/06/2022, 7:44 PM
Insane.
t

thdxr

06/06/2022, 7:44 PM
You have to import more directly from the file
a

Adrián Mouly

06/06/2022, 7:44 PM
Example?
Not using index?
t

thdxr

06/06/2022, 7:45 PM
yeah
import { Thing } from "common/thing.js"
instead of
import { Thing } from "common"
a

Adrián Mouly

06/06/2022, 7:45 PM
Crap.
I hate it.
😂
t

thdxr

06/06/2022, 7:45 PM
yeah this is the direction nodejs is going in though so we all have to get used to it
a

Adrián Mouly

06/06/2022, 7:46 PM
So there is no way back?
I liked the other way.
I mean, it’s cleaner.
t

thdxr

06/06/2022, 7:49 PM
no there isn't and this is also the case for your function code, they likely are bigger than need to be if you'r eimporting from index files
a

Adrián Mouly

06/06/2022, 7:50 PM
Yeah for sure.
I have to fix this then.
Is importing a bunch of shit.
t

thdxr

06/06/2022, 7:50 PM
yeah I guess I'm mixing up 2 issues
a

Adrián Mouly

06/06/2022, 7:50 PM
I thought the importing will be more intelligent.
t

thdxr

06/06/2022, 7:50 PM
index files with re-exports are file with ESM. It's just the esbuild specifically has trouble treeshaking them
a

Adrián Mouly

06/06/2022, 7:50 PM
I fixed the import for Typeorm and worked!
What you mean with
re-exports
? having them exported again from the
index.ts
in a sub-folder?
So, should I refactor all my imports and kill the
index.ts
in my shared-packages?
Also, when this started to happen? did SST switched to ESBuild recently?
What was used before? previous builder had tree-shaking?
Also, if currently I have this import, I have to convert it into 4 imports?
Copy code
import { BaseError, ErrorType, Logger, COMMON } from '@yabble/packages-common';
r

Ross Coundon

06/06/2022, 8:15 PM
Yeah, I got bitten by this recently. Esbuild seemingly isn’t smart enough to tree-shake when imports are via the re-export from an index file for convenience. I was accidentally importing way more than needed and some unnecessary initialisation was happening which caused an error and led me and Dax to work out why.
a

Adrián Mouly

06/06/2022, 8:16 PM
I see.
And how you fixed it?
You refactored all the imports?
r

Ross Coundon

06/06/2022, 8:16 PM
Yep
a

Adrián Mouly

06/06/2022, 8:17 PM
So sometimes you have to refactor 1 import into 4?
Or more.
r

Ross Coundon

06/06/2022, 8:17 PM
Yep
a

Adrián Mouly

06/06/2022, 8:17 PM
That’s sad 😞
r

Ross Coundon

06/06/2022, 8:17 PM
Yep
a

Adrián Mouly

06/06/2022, 8:17 PM
But is this temporary or esbuild will eventually become better?
r

Ross Coundon

06/06/2022, 8:18 PM
I’d imagine it’d get better but that’s just a guess
a

Adrián Mouly

06/06/2022, 8:20 PM
Yeah, but why nobody is complaining about this?
t

thdxr

06/06/2022, 8:23 PM
the reason this is popping up now for you is even before this was happening, but there were no issues with executing your stacks code
the esm switch made it obvious because it was bundling incompatible typeorm code
so not directly related
a

Adrián Mouly

06/06/2022, 8:23 PM
Ok, makes sense.
So we were using esbuild all the time?
t

thdxr

06/06/2022, 8:24 PM
yeah
and people are complaining about it there's an issue in esbuild for it but the reality is 1. on the frontend people only use esbuild for development - they use rollup for production builds (we may consider this as well) 2. on the backend no one cares about bundle sizes besides lambda users which is a small population
a

Adrián Mouly

06/06/2022, 8:24 PM
Yeah.
I see.
Sad that is not getting prioritized.
What’s the alternative to esbuild?
Vite?
t

thdxr

06/06/2022, 8:43 PM
Vite uses esbuild
a

Adrián Mouly

06/06/2022, 8:43 PM
Ah, good.
So I might refactor my imports and compare the bundle size.
I might get better sizes now.
I thought it was too high already.
Is this the issue?
t

thdxr

06/06/2022, 8:59 PM
yep
a

Adrián Mouly

06/06/2022, 9:01 PM
I’m starting to hate ESBuild, but then I see the comparison and …
Maybe that’s why it’s so fast…
Because they don’t do proper tree shaking 😂 .
m

Mischa Spiegelmock

06/07/2022, 2:19 AM
i'm getting this error too
t

thdxr

06/07/2022, 2:20 AM
are you using __dirname? If so there's a new way to get it in esm
m

Mischa Spiegelmock

06/07/2022, 2:20 AM
just upgraded my project to SST 1.x and I switched bundling from NodejsFunction to sst.Function previously I had this magic fix for esbuild:
Copy code
const ESM_REQUIRE_SHIM = `await(async()=>{let{dirname:e}=await import("path"),{fileURLToPath:i}=await import("url");if(typeof globalThis.__filename>"u"&&(globalThis.__filename=i(import.meta.url)),typeof globalThis.__dirname>"u"&&(globalThis.__dirname=e(globalThis.__filename)),typeof globalThis.require>"u"){let{default:a}=await import("module");globalThis.require=a.createRequire(import.meta.url)}})();`

  // use this to build lambdas as ESM
  const esmBundlingDefaults: Partial<FunctionBundleProp> = {
    format: "esm",
    mainFields: ["module", "main"], // prefer ESM versions of libraries
    banner: ESM_REQUIRE_SHIM,
}
which worked pretty great but i dunno how to do it with SST bundle options
t

thdxr

06/07/2022, 2:20 AM
Is it happening in a function or when building stacks code?
m

Mischa Spiegelmock

06/07/2022, 2:21 AM
when invoking a function
t

thdxr

06/07/2022, 2:21 AM
Hm I could add a __dirname shim to the banner
m

Mischa Spiegelmock

06/07/2022, 2:21 AM
t

thdxr

06/07/2022, 2:22 AM
Curious where the __dirname code is for you
Is it in a dependency?
m

Mischa Spiegelmock

06/07/2022, 2:22 AM
it must be
my code's been ESM longer than SST 😜
t

thdxr

06/07/2022, 2:22 AM
Ok I'll likely add the shim if dependencies are doing this
Sad state of things
m

Mischa Spiegelmock

06/07/2022, 2:23 AM
aye
t

thdxr

06/07/2022, 2:24 AM
We set the banner option internally so it might conflict
but I guess we just skip ours if it's set
a

Adrián Mouly

06/07/2022, 2:37 AM
What is banner and shim?
m

Mischa Spiegelmock

06/07/2022, 3:20 AM
it's a chunk of JS you can inject to bundled functions see the esbuild comment i linked for more info
3 Views