This message was deleted.

Did you check out our streamlit example? Might be useful here. It's definitely possible with Ploomber-engine as well

wow!! Where can i find the streamlit example? 🙂 Thanks! I'm using docker containers for running the streamlit frontend with the useful libraries

thanks very much for the precious help 🙂

Sure thing! Feel free to send feedback and reach out if you need more help 😊

the streamlit example feeds data to a streamlit dashboard, but based on your description it sounds like you want to trigger a run from streamlit. if that's the case, also check this ploomber example that uses the Python API. Using ploomber's Python API will give you more flexibility (caching, parallelization), ploomber-engine is simpler and you could just wrap

execute_notebook

in a for loop. if you are ok running your notebooks serially, I'd recommend ploomber-engine, if running in parallel is important, you can go with ploomber. regardless, feel free to ask for help

Hi Eduardo, thanks very much! yes, I would like to run the 4 notebooks that are inside a folder at the root of the main python script... 1. Click run button 2. start notebook 1 (which has only one input file, and 2 outputs) 3. show in streamlit a sort of "log" messages 4. start notebook 2 (which has as inputs the previous 2 file and one output) 5. and so on... maybe ploomber-engine can easily do this without transform the notebooks to python (even if it's not a big deal to transform in python)

ah. are the notebook in a different language? because ploomber-engine only supports Python, ploomber supports Python + other languages

Hi Eduardo! 🙂 thanks! They are just simple notebook with python script that are doing a simple ml pipeline: data_preparation.ipynb and training.ipynb

ok. yeah, either ploomber or ploomber-engine will work! do the log messages you want to display are static? or do you want to customized based on the notebook results?

in the notebooks i've added simple print statement that print some output after finish execution of the different cells.. I'm thinking to print it in the streamlit app with the proper streamlit command. examples: • print("Adversarial validation is ok" • print("These are the features importance: ", features_importances) --> it's a list/dataframe/dictionary

interesting use case. i think start with ploomber-engine, you'll probably need some processing for getting the log messages but that's fine. I'd say get version 1 running and then we can help you polish the details

thanks eduardo, i'll let you update when i'll have some results

awesome!

@daniele d'avino I think it'll be good to add this example/similar dummy example to our projects repo if you're up for it!

absolutely yes, i hope to be able to do it 😄

Awesome! We can support you once you open a PR with review/guidance etc 🙂

Hi guys, i give you an update. I finally manage to use ploomber engine to easily re-run some notebooks and pass some parameters into them. Locally works perfect, on a deploy in Cloud Run with Streamlit I'm only missing the check of saving the output into a dedicated Cloud Storage, but even in this "toy" version of a deployed service i think that it works well! Thanks very much! 🙂 Here the link to the python file in the repo for the streamlit deployment Here the "deployed" version in cloud run (under fraud analysis/retrain model) Thanks!

Amazing, thanks for sharing Daniele! @Eduardo