If I use Ploomber, how to collaborate with others who do not use Ploomber? Maybe they just refused to use Ploomber, any suggestions for the collaboration?

I think you could ask them to develop their code in a notebook or script and then include those as part of the pipeline. any feedback on why they refused to use it?

Ploomber seems pretty good for me. But we are not in CS related research field, switching from traditional .py files or jupyter notebooks may set a deep learning curve. I would highly recommend it, but I can not push my colleague or partners to change their workflow. If I were the only person using Ploomber in the team, maybe they need to run all my .py or .ipnb files sequentially according to the pipeline.yaml? Is there any way for them who do not install Ploomber to run all tasks, like in a main.py file?

Actually, Ploomber supports .Py files as well. And you can mix ipynb and py files in the same pipeline. Check out this example: https://github.com/ploomber/projects/tree/master/templates/ml-basic

Thanks. I have run the ml-basic demo. Maybe I did not make it clear. After I develop the project with Ploomber. Is there any way to run export like a main.py, like:

# run this script to get all the tasks done

# script 1
os.system("python get.py)

# script 2
os.system("python feature-engineering.py)

# script 3
os.system("python fit.py)

# script 4
os.system("python plot.py)

So my colleague can just run this main.py to get all the outputs, if he do not want to learn how to use ploomber.

Ah. Currently. There isn't a way to export Ploomber pipelines that way. But you could create a bash script that has the “ploomber build” command and have your colleagues execute it that way. Alternatively, you could create a main.py and execute the pipeline from Python. Here's an example: https://github.com/ploomber/projects/blob/master/cookbook/python-load/pipeline.py You can use the “make” function. Define the typical

if __name__ == "__main__"

at the bottom and do:

dag = make()

dag.build()

Thanks, Eduardo! I will try it. Ploomber seems great for me by far. Thanks for your brilliant work. By the way, templates/ml-basic report an error after `ploomber build`:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\nbformat\reader.py", line 18, in parse_json
    nb_dict = json.loads(s, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 192 column 411 (char 5665)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\ploomber\tasks\abc.py", line 554, in _build
    res = self._run()
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\ploomber\tasks\abc.py", line 663, in _run
    self.run()
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\ploomber\tasks\notebook.py", line 833, in run
    self._converter.convert()
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\ploomber\tasks\notebook.py", line 199, in convert
    self._from_ipynb(self._path_to_output, self._exporter,
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\ploomber\tasks\notebook.py", line 254, in _from_ipynb
    nb = nbformat.reads(path.read_text(), as_version=nbformat.NO_CONVERT)
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\nbformat\__init__.py", line 88, in reads
    nb = reader.reads(s, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\nbformat\reader.py", line 72, in reads
    nb_dict = parse_json(s, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\ag10\lib\site-packages\nbformat\reader.py", line 21, in parse_json
    raise NotJSONError(("Notebook does not appear to be JSON: %r" % s)[:77] + "...") from e
nbformat.reader.NotJSONError: Notebook does not appear to be JSON: '{\n "cells": [\n  {\n   "cell_type": "m...

ploomber.exceptions.TaskBuildError: Error building task "fit"

Found that there are related issues at github. Changing

nb: output/nb.html

to

nb: output/nb.ipynb

, it worked fine for me.

Thanks for using Ploomber! And thanks for reporting this issue, we'll get to it