This message was deleted.

yep. scroll down a bit here: https://docs.ploomber.io/en/latest/api/_modules/tasks/ploomber.tasks.NotebookRunner.html

i have a huge pipeline running on production , when running the command report i only see 4 files yet to run, however when plotting the same it shows more than 4 to be executed

what happens if you change both commands to do

DAGSpec('path/to/pipeline.yaml').to_dag().status()

and

DAGSpec('path/to/pipeline.yaml').to_dag().plot()

? does it still show the inconsistency?

yes they are still inconsistent

ok, and what happens when you build the pipeline? does it run the ones that the status or the plot say?

Remaining: 'BONUS_LIFT_CALCULATOR-BETBOOTR', 'CUSTOMER_LIFECYCLE_MANAGEMENT-PREMIUM-BETBOOTR', 'PROMOTION&CAMPAIGN ANALYTICS-BETBOOTR', and 'TOP CUSTOMER IDENTIFICATION CARD-BETBOOTR'

above are the remaining notebooks to be executed , however the report and plot shows completely different notebooks

ok, can you share the full output of status? I see it only has four tasks but it should print one line per task

do you have any stable version so that i can put back my production loads

it's hard to tell what's the problem, and if downgrading will solve. if you recorded the version before upgrading, I'd say revert back to that version and see if the problem goes away

ok.. i think 19.9 should work

also, you might want to restart the session, and run status right after plot. ploomber looks at metadata files so if the metadata changed, the results won't look the same

ya , am doing that

Cool let me know how It goes

My Observations on version 20 1. Its overburdening the system and starts killing the kernal despite having enough RAM

Can you point me to the errors you're seeing in pipeline-2.html? I counted the "outated=False", and the others. and they match. I went through them individually and i couldn't find a mismatch

Has not been run. =9 , however in the graph i see more that 9

In the html you shared, the ones that say “False” are up to date. So they appear on green. Any other value is “outdated” hence in red. I don't see anything that hasn't been run in the html.

why do i get outdated when am using --force to run the pipeline

no worries. this is really helpful. it means our plot and reports are not clear enough, and that's on us 🙂. if you do

ploomber build --force

that will run all tasks regardless of status. however; when creating the report/status/plot, it'll report the status (since those commands don't take the

--force

arg

how do i know if a task failed ? from the graph and from the table

the run status is printed after you run

ploomber build

for any task that fails, you'll see the traceback of each one