I’m working on a node script to import some data from a project that we’re moving to Graphcool and I’m having some issues with batching and queuing all the mutations that have to happen to import the data. For each JSON record in the incoming data, I need to create a couple of files, and fire off two other mutations to create related nodes. I’m pretty sure it’s just an issue with queuing or batching up the mutations properly. Has anyone gone through this? I’ve been looking at this: https://www.graph.cool/forum/t/import-data-into-node/359

if you can, I'd split it up in several simpler steps. this will likely require to to populate additional data into the nodes you create first, to be able to connect them later on. or you could create a local map from oldId -> newId

@nilan - that makes sense. I’m going to go down that route. Create file nodes, and associated supporting nodes first, then create the final main nodes

yep. it really is trickier when file uploading is involved! I have no good feeling for the most efficient way to reach high throughput for file uploads so would love to hear your feedback on that. here's a script that shows how to stream files though, this should be a good starting point: https://github.com/graphcool-examples/node-remote-file-upload/blob/master/request-streaming.js

I used that example code, it works well!

nice! what throughout are you achieving there?

I’m breaking the file upload into it’s own step now. What’s the best way to measure that?

how many nodes are we talking about here?

A few thousand total

well you can time how long it takes to upload the 3000 files 😄

Will do 😉

ah that's awesome!