This message was deleted.

Thanks, Kam!

Yo @alex thanks for joining

hello

i am human

o/ Saw the BF post. Our stack’s primarily PHP with a little bit of Golang and a bunch of Next.js for front-end stuff. The PHP bits I expect won’t be supported by y’all’s tooling, but happy to have another set of eyes on dependency security and such, as another test case (I think we’re in a decent spot, with slimmed-down Alpine images and ECR container scanning and such).

hey @Ian Littman! If you are using PHP composer for your dependency management, then our tool will collect and analyze those dependencies. And for sure we will be able to validate your go and js packages. 👍

Thanks for joining in @Ian Littman. We're definitely not PHP wizards but like Chris said we are able to scan Composer manifests and show all known CVEs

Definitely using composer. Was basically a question of “hey do y’all support composer or is it not a priority yet”

So I've heard from @Free Wortley and @Forrest Allison that you guys are working on streaming changes from NPM’s couch DB. I'm also trying to work on that, and I have a few questions. Right now, I'm able to stream changes (roughly like this: https://github.com/npm/registry-follower-tutorial) from replicate.npmjs.com, and then could potentially setup downloading of tarballs, etc. However, one big problem is that replicate.npmjs.com only includes non-scoped packages, whereas registry.npmjs.com includes both (https://stackoverflow.com/questions/64491597/difference-between-npmjs-replicate-and-npmjs-registry), so with this method I'm missing out on a whole bunch of data. However, there doesn't appear to be a way to stream changes from registry.npmjs.com. Though periodically scraping registry.npmjs.com is a possibility. I'm wondering how you guys have been thinking about this, or if I'm missing anything. Thanks!

@Donald Pinckney I think I used slim.npmjs.com as the upstream

Also I had to use a weird bastardized combination of Couch and PouchDB to make it actually go. And a round robin of HTTP proxies to make it bypass CloudFlare rate limits 😅

I'll go throw the DB blob in S3 for you and send that over. That'll save you a few days or weeks lol

Can we also share the scripts themselves @Free Wortley?

Yes but they're ugly AF 😅

Still might be useful

Ah, interesting. So what is the difference between slim, and the other 2?

And yeah, I unfortunately don't have any prior experience with PouchDB

Ah, interesting. So what is the difference between slim, and the other 2?

@Donald Pinckney I believe Slim doesn't include the package zip but it does include scoped packages. I definitely was able to match the same number of packages as what's listed on the main NPM site haha

Ah ok cool

@Donald Pinckney It's

<http://skimdb.npmjs.org|skimdb.npmjs.org>

sorry haha

I was just pulling up my code to go start cleaning it for you

Ah great! Don't stress yourself tho, it's Friday

Interesting discovery: I'm currently streaming changes from replicate.npmjs.com, and interestingly I do actually see scoped packages appearing in the changes

e.g.

but, more interestingly, this @fractures/ui package is indeed missing if you try to look it up on replicate, e.g. https://replicate.npmjs.com/@fractures/ui

This message was deleted.

Hi cameron!

This message was deleted.

you could just use cameron, im 90% sure