Hi everyone, I’m trying to set up middy with SST a...
# help
m
Hi everyone, I’m trying to set up middy with SST and I’m running into some problems. If I forego using middy, SST doesn’t have an issue. However, after adding the middy.js middleware in for the lambda handlers, the lambdas seem to not be getting the request and I’m seeing this error on the console. Does anyone have experience with this kind of problem? (My best guess is this might be failing somewhere in a request/response logger middleware, but it doesn’t specify anything)
Copy code
Uncaught Exception 	{"errorType":"Error","errorMessage":"The worker thread exited","stack":["Error: The worker thread exited","    at Worker.onWorkerExit (/Users/Michael/workspace/theater-profiles/.sst/artifacts/mrooney-theatrum-profiles--core-stack-CreateProfileLambda/src/lambda/create-profile/handler.js:1785:36)","    at Worker.emit (node:events:394:28)","    at Worker.emit (node:domain:475:12)","    at Worker.[kOnExit] (node:internal/worker:281:10)","    at Worker.<computed>.onexit (node:internal/worker:201:20)"]}
/Users/Michael/workspace/theater-profiles/.sst/artifacts/mrooney-theatrum-profiles--core-stack-CreateProfileLambda/src/lambda/create-profile/handler.js:1871
          throw new Error("the worker has exited");
          ^

Error: the worker has exited
    at ThreadStream.flushSync (/Users/Michael/workspace/theater-profiles/.sst/artifacts/mrooney-theatrum-profiles--core-stack-CreateProfileLambda/src/lambda/create-profile/handler.js:1871:17)
    at process.onExit2 (/Users/Michael/workspace/theater-profiles/.sst/artifacts/mrooney-theatrum-profiles--core-stack-CreateProfileLambda/src/lambda/create-profile/handler.js:2084:16)
    at process.emit (node:events:406:35)
    at process.emit (node:domain:475:12)
    at process.exit (node:internal/process/per_thread:184:15)
    at process.<anonymous> (file:///Users/Michael/workspace/theater-profiles/node_modules/@serverless-stack/aws-lambda-ric/lib/index.js:31:17)
    at process.emit (node:events:394:28)
    at process.emit (node:domain:475:12)
    at process._fatalException (node:internal/process/execution:170:25)
Here are the Lambda logs in Cloudwatch:
Copy code
2022-02-27T23:36:05.338-08:00	START RequestId: f2edb37d-916b-47c5-90da-67fba5fed4f5 Version: $LATEST

2022-02-27T23:36:05.347-08:00	2022-02-28T07:36:05.340Z f2edb37d-916b-47c5-90da-67fba5fed4f5 INFO connectAndSendMessage()

2022-02-27T23:36:05.432-08:00	2022-02-28T07:36:05.432Z f2edb37d-916b-47c5-90da-67fba5fed4f5 INFO ws.onopen

2022-02-27T23:36:05.432-08:00	2022-02-28T07:36:05.432Z f2edb37d-916b-47c5-90da-67fba5fed4f5 INFO sendMessage() - send request

2022-02-27T23:36:05.433-08:00	2022-02-28T07:36:05.433Z f2edb37d-916b-47c5-90da-67fba5fed4f5 INFO sendMessage() - sending request via WebSocket

2022-02-27T23:36:05.434-08:00	2022-02-28T07:36:05.434Z f2edb37d-916b-47c5-90da-67fba5fed4f5 INFO sendMessage() - start keep alive timer

2022-02-27T23:36:15.343-08:00	END RequestId: f2edb37d-916b-47c5-90da-67fba5fed4f5
m
I have, yes. I’m using middy in a similar way but not sure what’s causing this strange “worker has exited” error.
m
I guess it’s related to internal nodejs threads.
Could you share how you’re using the middy with the handler? here or DM is fine.
f
@Michael Rooney just checking in to see if you had a chance to get middy to work?
m
No, I still haven’t been able to get it to work. I’m not sure what’s causing these internal node thread problems.
f
hmm.. which middy plugin are u using? Would you able to create a simple repo with this issue? And I can give it a try on my end.
m
I’m using multiple plugins. Here’s the helper function I wrap all the lambdas with:
Copy code
import middy from "@middy/core";
import { Handler as LambdaHandler } from "aws-lambda";
import httpEventNormalizer from "@middy/http-event-normalizer";
import httpHeaderNormalizer from "@middy/http-header-normalizer";
import httpSecurityHeaders from "@middy/http-security-headers";
import jsonBodyParser from "@middy/http-json-body-parser";
import inputOutputLogger from "@middy/input-output-logger";
import httpErrorHandler from "@middy/http-error-handler";
import { log } from "../utils/logger";
import cors from "@middy/http-cors";
import validator from "@middy/validator";

export function with_middleware({ validators, logOptions }: TMiddlewareOptions = {}): (
  baseHandler: LambdaHandler
) => middy.MiddyfiedHandler {
  return (baseHandler) => {
    const lambda = middy(baseHandler);

    lambda
      .before(async (request) => {
        // Prevents jsonBodyParser from complaining on GET requests
        if (!request.event?.body) {
          request.event.body = "{}";
        }
      })
      .use(loggerMiddleware(logOptions)) // my own middleware that sets up a pino logger
      .use(inputOutputLogger({
        logger: (request) => {
            const child = log.child(request.context)
            <http://child.info|child.info>(request.event ?? request.response)
        },
        awsContext: true
      }))
      .use(httpSecurityHeaders()) // Applies best practice security headers to responses.
      .use(httpHeaderNormalizer({ canonical: true })) // Normalizes HTTP header names to their canonical format
      // Normalizes HTTP events by adding an empty object for queryStringParameters, multiValueQueryStringParameters or pathParameters if they are missing.
      .use(httpEventNormalizer({ payloadFormatVersion: 2 }))
      // Automatically parses HTTP requests with JSON body and converts the body into an object.
      // Also handles gracefully broken JSON if used in combination of httpErrorHandler.
      .use(jsonBodyParser())
      .use(httpErrorHandler())
      .use(cors())

    if (validators?.inputSchema) {
      lambda.use(
        validator({
          inputSchema: validators.inputSchema,
          ajvOptions: {
            strict: false,
            useDefaults: false,
          },
        })
      );
    }

    if (validators?.outputSchema) {
      lambda.use(
        validator({
          outputSchema: validators.outputSchema,
          ajvOptions: {
            strict: false,
            useDefaults: false,
          },
        })
      );
    }

    // lambda
    //   .use(identityProvider())
    //   .use(authenticate())
    //   .use(authorize())
    //   .use(cleanup());

    return lambda;
  };
}
I don’t have a simple repo with this issue yet, but I can work on seeing if I can reproduce it with a simple app later today.
f
I see. Let me give it a try.
Btw, does it fail with these commented out?
Copy code
// lambda
    //   .use(identityProvider())
    //   .use(authenticate())
    //   .use(authorize())
    //   .use(cleanup());
m
Yes, those were always commented out since they're unused for now.
f
I’m getting a couple of type errors.
TMiddlewareOptions
and
loggerMiddleware
are not found.
It might be better if you could provide a simple app.
m
Hmm. Interesting. I’m seeing a 10 second timeout error for “_flushSync took too long (10s)” now with this simple app I set up. I will put it into a git repo so you can pull it down and take a look.
Thanks again for your help! Apologies for the long delay between check-ins. Here is the simple app on git: https://github.com/kungpaomichael/my-sst-app
It looks like it is an issue with the pino logger and pretty print. Apologies for leading you on a rabbit trail here! Thank you for your help! 🙂
f
Ah glad u figure it out!