Dear group, we have a customer with an Illumina sequencer who's looking to point their data to a Wasabi bucket (their API is S3-compatible). The Illumina config doesn't seem to allow this. I think I have a workaround but I thought I'd probe the group first, anyone ever looked into this?

Do you mean the sequencer output? AFAIK the appliance only allows writing to basespace over S3, otherwise you’re stuck with local mounts

Got it — and by "local mounts" you mean like writing to a local drive on a computer attached to the machine?

The illumina machines can mount external network drives. If you can mount the S3 bucket, you should be able to point the illumina software to the mounted drive.

If your machine is using Linux, I can recommend

rclone

-- it can be used to

sync

with Wasabi buckets, and you can also `mount`Wasabi buckets, so they are accessible as a directory on your local machine. It is also available for Windows and macOS.

Does anyone have any guide to setting that up for a NovaSeq I can pass on? 😉

for plain S3 I’d suggest mounting through a JuiceFS mount

Rclone supports every storage provider I've ever heard of and a hundred more. What I don't know is how long a mounted connection holds up in practice without going stale, and performance is another question.

Yeah, the main issues is auth against GCP, which only stays good for 24h in our case

Yes, that's annoying. Better than Microsoft OneDrive, though, which is only an hour or two. The most practical solution might be to store a local copy of each run and then sync that to the loud whenever it's done.

Update on this, I've been trying to get in touch with Illumina to no avail. The local storage thing is definitely a strong option for power users, but the users in question are not big on tech, very big on biology, they want to beam it straight to Wasabi. In the Illumina ICA video (picture below) there is no option for endpoint URL, only for credentials. @Matthias De Smet First I've heard of JuiceFS but looks way cool. We just used s3fs to automatically mount an s3 bucket to all of our customer instances so your s3 bucket is right there when you connect. I think this is what they want, they want to run the sequencer and then log straight in and align it right there. PS if we wanted to go totally psycho, we could add a DNS record to their router pointing east-us-1.amazonaws.com to east-us-1.wasabisys.com,e but if they were that technical they probably would have no problem with going through local storage +

rsync

first 🙂