Hi < Frank> Wondering if this might be a new feature for Ser Serverless Stack #sst

Hi <@U01JVDKASAC> - Wondering if this might be a n...

Luke Wyman

09/17/2021, 4:27 AM

Hi @Frank - Wondering if this might be a new feature for Serverless Stack: I think it was a couple of weeks ago that I had been struggling with developing unit testable Python code that could be both reachable from the unit tests covering it, while at the same time, being deployable as a package(s) that the Lambdas can access in their cloud environment. I kinda sorta have that working well enough in my karaoke project (tests run locally and Lambda run in the cloud), but it's clunky and not clean. I built this little spike (raw Lambda/Docker, no sst) to demonstrate packaging unit-testable, non-Lambda Python as a wheel file and copying/installing that into a Docker image with pip. Full description/details in the README. Next, I'm going to see if I can use the custom

installCommands

to get those same steps to happen with sst. The thing is, I'm wondering if some take on this could become a feature of sst. Typescript sst projects have their nice, clean way of running tests in

test

using Jest, and the

tsconfig

et al working the deployment. As it currently stands, coordinating testing and deployment with Python in sst seems a little under-developed by comparison.

Frank

09/18/2021, 12:06 AM

Hey @Luke Wyman, thanks for putting this together! I feel the same about bundling Python functions in SST does not have the out-of-the-box experience as Node.

Frank

09/18/2021, 12:16 AM

I haven’t worked much with Python and I didn’t have a good sense of different bundling options ppl use in the Python world. So I looked up how CDK, Serverless Framework, and SAM package Python functions but it seems they were merely installing the requirement.txt and zipping it up with the Lambda code.

Frank

09/18/2021, 12:22 AM

Imagine this folder structure,

Copy code

/
  libs/
    my-lib.py
  handlers/
    my-handler.py

One issue that often pop up is if

my-handler

tries to import

my-lib

, it can’t be done because Lambda doesn’t support relative import, ie.

from ..libs

. But that works when running locally in

sst start

Frank

09/18/2021, 12:22 AM

I feel what you are suggesting also solves this issue?

Frank

09/18/2021, 12:22 AM

Not sure if I’m on the right track 😂

Luke Wyman

09/19/2021, 1:33 AM

I think you're more or less on the right track (best I can say, since I'm learning myself). I keep vascillating between whether there is a feature to be had here that abstracts Python dependencies and packaging as part of

sst deploy

, or whether this should just be left as a space for Python engineers to use their usual build and packaging tooling. I'm sitting down tonight to play with

installCommands

some more and make another little spike out of that. I think what I'll do, is summarize some steps from the results of my spikes that may or not be abstractable, addressing: 1. The do-ability of relative imports like

from ..libs

- I believe it's doable, from what I've seen so far. I'll be sure to note what my discoveries are around that. 2. The steps a Python dev may expect to happen during deployment, either as independent, scripted commands that they manage, or as steps abstracted away in the cdk constructs and in an

sst deploy

(kinda thinking about the scripts construct you guys recently added to sst). 3. Either publishing the

.whl

as a package to a private Python package server, or just going direct from the

.whl

into the zip (making relative imports like

from ..libs

doable). Will let you know! :)

Open in Slack

Previous Next