Community for the Daft project and all things distributed data

Distributed Data Community

when running a UDF with native runner and `use_process=True` everything works fine locally (mac), but seeing `/usr/bin/bash: line 1:    58 Bus error               (core dumped)` when run on k8s (argo workflows). any thoughts?

the job is also running significantly slower using native or ray runner in k8s compared to local

I believe I figured it out. `/dev/shm` is very small by default `64mb` for tempdir, bumping this up fixed the issue

Thanks for posting your solution. I'm curious if I've been hitting a similar issue in my k8s cluster