This message was deleted.

Does this happen with multiple tasks you’re trying to parallelize or a single one? Ideally each module should have its own task, that way you can reuse it and enjoy the parallelization.

I think Jan wants to parallelize inside a task. I'm guessing you're using functions as tasks. By default, ploomber runs them in a subprocess to ensure memory is cleared up but you can change this setting: https://docs.ploomber.io/en/latest/api/_modules/executors/ploomber.executors.Serial.html#ploomber-executors-serial

executor:
  dotted_path: ploomber.executors.Serial
  build_in_subprocess: false

Thanks! That did the trick :)