https://www.lunasec.io logo
#general
Title
# general
d

Donald Pinckney

05/06/2022, 3:20 AM
So I've heard from @Free Wortley and @Forrest Allison that you guys are working on streaming changes from NPM’s couch DB. I'm also trying to work on that, and I have a few questions. Right now, I'm able to stream changes (roughly like this: https://github.com/npm/registry-follower-tutorial) from replicate.npmjs.com, and then could potentially setup downloading of tarballs, etc. However, one big problem is that replicate.npmjs.com only includes non-scoped packages, whereas registry.npmjs.com includes both (https://stackoverflow.com/questions/64491597/difference-between-npmjs-replicate-and-npmjs-registry), so with this method I'm missing out on a whole bunch of data. However, there doesn't appear to be a way to stream changes from registry.npmjs.com. Though periodically scraping registry.npmjs.com is a possibility. I'm wondering how you guys have been thinking about this, or if I'm missing anything. Thanks!
3 Views