https://serverless-stack.com/ logo
Join Slack
Powered by
I'm trying to get an SST python REST API running i...
# help
s
I'm trying to get an SST python REST API running inside a Docker container (to fit within the development setup of my peers). The 
npm run start
works well and the live lambda environment spins up just fine. However, when I hit an endpoint, I get a JSON parsing error
The error is
Copy code
54b9eb2d-7fbe-45db-81fb-89f896e57099 REQUEST sgeoghegan-kas-sst-kas-st-ApiLambdaGETrecordsource-LZHr312naXqt [functions/record_source/retrieve.handler] invoked by API GET /record-source
Traceback (most recent call last):
  File "/code/node_modules/@serverless-stack/core/src/runtime/shells/bootstrap.py", line 59, in <module>
    event = json.loads(r.read())
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
This works fine outside of Docker, so I'm wondering wtf I did here 😕
This Stackoverflow answer suggests the byte stream needs to be decoded:
Copy code
buf = response.read()
result = json.loads(buf.decode('utf-8'))
m
Maybe docker is installing another version of SST. If I'm not wrong 0.47.2 fixed this issue.
s
from my package.json:
Copy code
"dependencies": {
    "@aws-cdk/core": "1.126.0",
    "@serverless-stack/cli": "0.47.2",
    "@serverless-stack/resources": "0.47.2"
  },
m
Don't know about docker that much, but maybe docker is installing a version below 0.47.2 maybe? Can we ask docker to install a specific version of a package explicitly?
s
Hmm, I just noticed that the error message says python 3.5
Copy code
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
Although my SST app is specifying a runtime environment of Python 3.7
Copy code
// Set default runtime for all functions
  app.setDefaultFunctionProps({
    runtime: "python3.7",
    srcPath: "functions",
    environment:{          
      ...dbCredentials
    }
  });
Yeah, I think this issue is related to converting a byte stream to a string before parsing the JSON
OK, I modified bootstrap.py from
Copy code
event = json.loads(r.read())
to
Copy code
event = json.loads(r.read().decode('utf-8'))
which seemed to do the trick.
I am fairly new to Python and Docker, so I'm not sure if this is the correct path forward or not. 🤷
f
@Seth Geoghegan do all Lambda functions fail with the error above inside docker? And what does the Python function return?
s
The API only has 4 endpoints and it happened on each endpoint I tested. The lambda response is a serialized Python dictionary
Copy code
from lib.dbi import DBConfig
from lib.dbi.record_source import RecordSourceDBI

from lib.helper import json_serialize
from lib.helper.connection import ConnectionHelper


def handler(event, _context):
    try:
        response = handler_safe(event, _context)
        status_code = 200

    except Exception as e:
        response = dict(error=str(e))
        status_code = 500

    finally:
        ConnectionHelper.close()
        return dict(body=json_serialize(response), statusCode=status_code)


def handler_safe(event, _context):

    db_connection = ConnectionHelper.open(**DBConfig.get_attribution_db())

    dbi = RecordSourceDBI(db_connection)

    request_params = event.get('queryStringParameters')

    db = request_params.get('record_db')
    table = request_params.get('record_table')
    table_id = str(request_params.get('record_table_id'))

    record_source = dbi.get_record_source(db, table, table_id)

    return dict(record_source=record_source)
f
Thanks for the details @Seth Geoghegan. Do you mind giving this a try real quick and see if this dummy handler works inside and outside docker?
Copy code
def handler(event, context):
  response = {
    "statusCode": 200,
    "body": "Hello World"
  }

  return response
s
This is strange, but I can't seem to replicate the error now. I've reverted back to the prior version of boostrap.py and re-ran my docker-compose commands. It works 🤷
but I'll try replacing the handler code as you've outlined if it somehow pops up again
Open in Slack
PreviousNext
Powered by