https://serverless-stack.com/ logo
Join Slack
Powered by
ok now I’m running into another issue. It appears ...
# sst
t
ok now I’m running into another issue. It appears that setting 
bundle: false
doesn’t actually stop esbuild from building even for deploy.
I have 4 functions:
Copy code
const lambda = {
      graphql: new sst.Function(this, "graphql", {
        handler: 'index.main',
        bundle: false,
        srcPath: 'dist/graphql',
        environment: env,
        memorySize: 256,
        timeout: 30,
      }),
      
      web: new sst.Function(this, "web", {
        handler: 'index.main',
        bundle: false,
        srcPath: 'dist/web',
        environment: env,
        memorySize: 256,
        timeout: 30,
      }),
      
      worker: new sst.Function(this, "worker", {
        handler: 'index.main',
        bundle: false,
        srcPath: 'dist/worker',
        environment: env,
        memorySize: 256,
        timeout: 500,
      }),
      
      migrate: new sst.Function(this, "migrate", {
        handler: 'index.main',
        bundle: false,
        srcPath: 'dist/migrate',
        environment: env,
        memorySize: 512,
        timeout: 500,
      })
    };
and I would assume that 
bundle: false
would just zip/upload the content of 
srcPath
and inform lambda of the file/handler to call.
however I then noticed that the lambda sizes are much bigger than I would have thought, so I started inspecting the contents of the lambda function and noticed a 
.build
directory in addition to my source
which is an esbuild duplicate of my 
index.js
but with the sourcemap screwed up.
am I missing something here?
bummer. Alright, that means 
bundle: false
isn’t a simple “package up my source and tell lambda to use the handler specified”.
but it’s a esbuild bundle attribute: https://esbuild.github.io/api/#bundle
not at all what I was hoping for, but I guess that’s the system so now to think about how to work within…
hmmm. I suppose it means I will need to feed all of my source through esbuild. So in my case, a legacy project that already has a build pipeline, I’ll either have to do a major rewrite of the build pipeline (definitely no time for that) or I’ll just have to ensure that my current build pipeline doesn’t do any optimizations or minifications that will get screwed up once piped through esbuild.
my lambda functions will be at least 10x bigger, but maybe that’s ok for now.
maybe I can get esbuild to minify
(the rationale behind minification is to keep the lambda zip as small as possible. Source maps have allowed me to minify and still get original stack traces)
ah, looks like the builder would need to be expanded to support a minify attribute: https://github.com/serverless-stack/serverless-stack/blob/master/packages/resources/src/util/builder.ts#L209
which would have to be added in the FunctionBundleProps: https://docs.serverless-stack.com/constructs/Function#functionbundleprops
alright, unfortunately I’m passed my allotted time limit for this experiment right now. Maybe when I come back and have a bit more time I’ll submit a few PRs: 1 - Support minify as a build option 2 - Introduce a raw builder, which doesn’t run through esbuild at all, thus allowing other build pipelines to be used (webpack, rollup, etc).
a quick thought on #2, I don’t think sst should even try to support other loaders. I love that sst is opinionated and is clearly planted in the future with esbuild. It’s a great product and no defense is required for the choice. Rather, I would love to see sst support a raw or non-build option. Legacy build pipelines can generate output which can be zipped up, and for 
sst start
could end-run 90% of the functionality and just map a request to a file/handler. A legacy builder like webpack could be run in watch mode and achieve the same result.
For instance, if using 
sst start
with an external builder, just run 
sst start
and 
webpack --watch
concurrently, with the 
concurrently
package: https://www.npmjs.com/package/concurrently
like this: 
concurrently "webpack --watch" "sst start"
and then sst doesn’t have to support the whole world
which, fwiw is probably how I would think about supporting other languages. In theory as long as “insert language here” knows how to build the source and can be setup to watch -> compile then sst only needs to be a simple forklift for deploys and a simple pipe for development.
you could even simplify the workflow with a 
buildCmd
and 
devCmd
in the sst.json, and substitute the current esbuild process for those if they exist. With this simple addition you could support anything that lambda currently supports.
because the genius of sst (which it is genius) is 1 - The cdk constructs and building on top of cdk. 2 - The development stack.
the packaging is a mostly solved problem for each language. If sst can provide a first-class experience where possible, but then an escape hatch for everything else… wow.
f
Hey @Tyler Flint Thanks for looking into and appreciate the thoughts! Yeah, the esbuild’s bundle flag is always enabled currently. Can you share a snippet of how you are building ur functions right now?
@Jay can you take a look at this thread and share your thoughts? I think supporting a 
buildCmd
as Tyler suggested makes sense. Even with the Go runtime I’m working on right now I can see a usecase of overriding the default 
go build
command.
t
@Frank for the time being, I’m using my existing webpack build process to generate a non-optimized javascript file, then I inform sst to start there. It then gets pumped through esbuild which doesn’t do much since it’s already bundled from webpack. It works, but it’s not ideal because I have to turn off all of the optimizations in webpack and currently I don’t have any way to enable optimizations in esbuild.
here’s my function definition:
Copy code
graphql: new sst.Function(this, "graphql", {
        handler: 'dist/graphql.main',
        environment: env,
        memorySize: 256,
        timeout: 30,
      }),
and 
dist/graphql.main
is a generated file. Here’s a shot of the top of the file:
f
Got it. Can I see the webpack script/command u r using?
t
Copy code
const fs                     = require('fs');
const path                   = require('path');
const webpack                = require('webpack');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');

const entries = (dir) => {
  return fs.readdirSync(path.resolve(__dirname, dir)).reduce((res, file) => {
    // split the file at the extension
    const [name, extension] = file.split('.');
    
    // only add the entry if it's a coffeescript or javascript file
    if (extension.match(/coffee|js/))
      res[name] = `${path.resolve(__dirname, dir)}/${file}`;
      
    return res;
  }, {});
};

module.exports = {
  entry: entries('src/lambda'),
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: '[name].js',
    libraryTarget: "umd"
  },
  mode: 'development',
  target: 'node',
  externals: ['aws-sdk', 'bufferutil', 'utf-8-validate'],
  node: false,
  optimization: {
    usedExports: true,
  },
  plugins: [
    new webpack.ContextReplacementPlugin(/any-promise/),
    new CleanWebpackPlugin()
  ],
  devtool: 'inline-cheap-module-source-map',
  module: {
    rules: [
      {
        test: /\.coffee$/,
        loader: 'coffee-loader',
        options: {
          inlineMap: true,
          transpile: {
            presets: ['@babel/env']
          }
        }
      }
    ]
  },
  resolve: {
    alias: {
      '~'                         : path.resolve(__dirname, 'src'),
      'busboy'                    : path.resolve(__dirname, 'src/app/null.coffee'),
      'apollo-reporting-protobuf' : path.resolve(__dirname, 'src/app/null.coffee'),
      'apollo-tools'              : path.resolve(__dirname, 'src/app/null.coffee'),
    },
    extensions: [ '.wasm', '.mjs', '.js', '.coffee', '.json' ]
  }
};
we still use coffeescript extensively, which isn’t supported through esbuild
and we also gut the garbage out of the apollo project that isn’t needed for serverless deploys
to cut literally MBs out of the resulting bundle
f
I see.. yeah that makes sense
t
at any rate, there are probably ways to get it to work with esbuild by first compiling coffeescript to javascript, but then I’d have to figure out how to ensure the sourcemaps make it all the way through.
then I’d have to figure out how to strip cruft out apollo. Unfortunately the tree-shaking stuff doesn’t work there because of the way they’ve tied all of their types together. Even one function import that has nothing to do with file uploads will end up including a huge library that isn’t needed.
anyways, for me the solution right now is to just keep using webpack for compiling the source, then piping that through esbuild. Ideally, I could just dump the result directly, as was described above.
f
right.. definitely not ideal.. you probably would have to start ur own webpack watcher to output to 
/dist
and 
sst start
watcher gets then triggered
t
yeah, here’s what I’m currently doing:
in my package.json:
Copy code
"scripts": {
    "dev": "concurrently 'webpack --watch' 'sst start'",
it works great
f
ah i see… interesting!
t
I also have a 
compile
script that is just 
webpack
so for my seed configuration I’ll tell it to run 
compile
first before build
that’s what I’m about to do right now
so looking through the source for 
sst start
, you could support a ton of languages right off just by conditionally looking for a 
devCmd
or similar, and if it exists run that in a pty process instead of starting the watcher/builder.
as far as I can tell, the logic to wire up a request from a wss message into a local handler is not specific to javascript.
and for packaging a function for deployment, the existence of a 
buildCmd
or similar could just end-run the build process and zip the contents.
you could literally shove a rails app into it haha. I mean, the cold-start time would probably be horrible but the point is that sst doesn’t have to care.
Open in Slack
PreviousNext
Powered by