https://www.prisma.io/ logo
Join Slack
Powered by
I want to test my changes to the prisma engine whe...
# prisma-client
m
I want to test my changes to the prisma engine when generating the nodejs client. I ran 
cargo build
in 
prisma-engines
- do I need to specify my locally built library in 
PRISMA_QUERY_ENGINE_LIBRARY
when running 
npx prisma generate
? If I pass in the query engine dylib that got built:
Copy code
[db:generate] Error: Unable to require(`/Users/cyber/dev/prisma-engines/target/debug/libquery_engine.dylib`)
[db:generate]  Invalid or unexpected token
[db:generate] npm ERR! Lifecycle script `generate` failed with error:
I want to test my changes to DMMF generation locally and am not clear on how to do this
j
You need to rename the file with the same file ending, Node security mechanism that it can not execute random files.
(Yes, it is weird that the build output itself has a differnet file ending)
m
ah thanks
seems like my changes didn't shrink it that much sadly
I'm happy to work on this more but I would need some direction from someone who knows the setup better. I think we should have a goal of shrinking the DMMF
j
The problem here is that DMMF is an API - we can not just easily mess with it without taking other consumers into account. Also, optimizing a bit of space might not even be the right thing to spend time on.
But we are definitely aware of the problem and just yesterday did a huge deep dive into that area to figure out areas where we can actually do something about it.
m
I'm testing these changes right now
I was able to shrink my generated client by ~1.5-2MB or so
it's not dramatic but it's definitely an improvement
do the clients assume that the absence of a value in DMMF is "false" or do they explode?
and is there any downside to skipping the JSON.parse(dmmfString) step? (separate optimization)
also wondering... maybe it would shrink the engine size if debug symbols were stripped?
Copy code
dwarfdump  /Users/cyber/dev/platform/packages/infra/.build/cdk.out/asset.9644c9f988f239064766cfec2a8895a7bca252c383690784fa5ea2231da01b8d/nodejs/node_modules/@prisma/engines/libquery_engine-rhel-openssl-1.0.x.so.node|wc -l
  505815
j
do the clients assume that the absence of a value in DMMF is "false" or do they explode?
Not sure, we could figure that out. But that might not be true for community generators that also depend on the dmmf value they get during the generation step.
and is there any downside to skipping the JSON.parse(dmmfString) step? (separate optimization)
The Client needs the parsed JSON and not a textual representation, so that is important.
also wondering... maybe it would shrink the engine size if debug symbols were stripped?
Yes - but at the cost of making all the stack traces pretty useless unfortunately. We rely on these to figure out where things go wrong and why. We have not come up with an in between solution that would not have that disadvantage unfortunately.
m
The Client needs the parsed JSON and not a textual representation, so that is important.
so is the change to this file kosher? https://github.com/prisma/prisma/pull/12454/files#diff-c932f51b29bd2edde3fcd294d529cba543d23e89c3b7a1403b19ec91f3c59803 I could make a PR with just that change
j
OK that is funny. No idea why that might be needed. Buuuut: That is code for the 
dataproxy
only. That code path s not used with the normal engine.
m
i thought 
buildCompressedDMMF
was used for 
dataproxy
this is 
buildUncompressedDMMF
j
Ah you are right. I was confused by this being the on-generate-time code.
The end result of this looks like this:
Copy code
const dmmfString = "{\"datamodel\":{\"enums\":[{\"name\":\..."read\":[],\"write\":[\"runCommandRaw\"]}}}"
const dmmf = JSON.parse(dmmfString)
exports.Prisma.dmmf = JSON.parse(dmmfString)
I don't think you can just write the actual dmmf into the string that will be put into 
index.js
m
i've been testing my change, it works fine and results in a smaller bundle
bundle is smaller and you skip calling JSON.parse() on 7MB of JSON... and the example you have there calls it twice?
j
What does the non stringified result in the written 
index.js
look like?
The calling 
parse
twice thing baffled me as well, but is explained in a code comment somewhere. Something how internal object is copied or something.
m
👍 1
continues on for about five million characters
j
Did you measure if it is actually faster than 
Json.parse
? I remember some benchmarking of huge JS objects vs. parsing Json strings (parsing might be faster as it has far less options than the actual object and it is clear that the outcome only will be data from the outset).
m
i didn't test it separately but the smaller DMMF + skipping parsing had a small improvement in cold start times for my app
not earth-shattering but a clear improvement
j
Splitting those apart would be interesting to see the effect on its own each.
m
yeah you're right, doing this now
👍 1
I believe you're right that parsing the dmmfString actually is faster 🤯 testing cold starts of the changes indepdendently:
Stock 3.11.0:
Skipping dmmfString:
Removing 
false
values from dmmf:
it's not the most scientific test in the world but i've tried these a few times and gotten consistent results
conclusion: making the dmmf structure smaller is effective and seems to not break anything for my use case (node)
👍 1
j
We will obviously do all this in a more "scientific" way before we make any changes 😄 But this is great input.
m
okay, I am happy to beta-test any changes you come up with. I am also happy to do some coding if there is something I can help with. my main priority is reducing the cold start time of my app.
j
Do we have access to the app and a description of your deployment to reproduce yet? (I hope I was not the one you gave it to and forgot about it and just delegated 😛 )
m
I can share my schema if you like, the app isn't really open-source, we're using Serverless Stack, CDK, NodejsLambda, esbuild
aurora serverless
i can give you access to the app if you want
j
It might be a good reproduction case for us to look into that.
Should we do a quick NDA to cover both sides?
m
sure
j
I need name, company name, title, email and can start it. Afterwards you can just share things with my GH Account or via email.
m
Mischa Spiegelmock, JetBridge Software Inc., mischa@jetbridge.com
👍 1
Open in Slack
PreviousNext
Powered by