https://ploomber.io/ logo
Join Slack
Powered by
This message was deleted.
# ask-anything
s
This message was deleted.
e
ok, can you edit the 
pipeline.yaml
and change all 
.html
files for 
.ipynb
?
then try again
m
that worked
Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [0002<0000, 1.41cell/s] Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [0002<0000, 1.93cell/s] Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [0001<0000, 3.23cell/s] Building task 'plot': 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [0005<0000, 1.84s/it] name Ran? Elapsed (s) Percentage ------ ------ ------------- ------------ raw True 2.15235 39.0844 clean True 2.09616 38.0642 plot True 1.25841 22.8514
thank you!!! that was just the example untouched so was confused why there was an error
e
you're good. it's an error in one of the dependencies that ploomber uses. you're using windows right? we've seen this error before
m
yup windows 10
cool now I can get back to building 🙂
e
nice, feel free to post any other questions you may have
m
posting another in separate post ty
e
sure
was the error you posted the full text? cause i was expecting to see something 
'charmap' codec can't encode character
m
when I run my code I do get that
I'll change mine now and see if it works
once I started getting the error I ran the example to see where the problem was
e
ok cool. i was worrying that ploomber might be hiding some of the error traceback, but if you are able to see the 
'charmap' codec can't encode character
, then we're good
m
Error building task 'downloadData': the task ran successfully but the product is missing. Ensure the task is generating the declared products
ok so once I changed .html to .ipynb I get this
e
ok, so that means that in your 
pipeline.yaml
your 
downloadData
task is supposed to generate some file, but after executing the code, ploomber cannot find it
m
only one of the two csv's in the product get completd
e
you need to ensure that your code is actually creating both files, can you share your 
pipeline.yaml
?
👍 1
m
tasks: - source: 00-data_collection/downloadData.R product: nb: 00-data_collection/downloadData.ipynb data: 00-data_collection/metadata/counterparts.csv data: 00-data_collection/metadata/countries.csv
need to ensure my code is working
e
yep, ensure is generating both counterparts.csv and countries.csv
most likely, one of those is missing
m
the R code works
dots.zip
might be easier to share
e
so what's the problem here? it doesn't run?
m
yea it doesnt make counterparts.csv, though I ran the R code line by line and it works, and then error's out saying it can't find the file
it makes countries.csv and the ipynb
e
oh i see what the error is. so you're saying in your pipeline.yaml that you'll save the output in the 00-data_collection folder but in your code, you're saving them in the current directory (the one that executes ploomber)
make this changes
Copy code
# pipeline.yaml
tasks:
  - source: 00-data_collection/downloadData.R
    product:
      nb: 00-data_collection/downloadData.ipynb
      counterparts: 00-data_collection/metadata/counterparts.csv
      countries: 00-data_collection/metadata/countries.csv
then in your script:
Copy code
write.csv(countries, product$countries)
write.csv(counterpart, product$counterparts)
m
ahhhhhhh
thank you so much
that is a dumb mistake
e
no worries, this is very helpful feedback. ploomber should be easier to use 🙂
if you want ploomber to extract the values from pipeline.yaml to your scripts, you can run 
ploomber nb -i
(it does it when you run 
ploomber build
automatically but with this command you can see how it'd look like)
m
ok I made the changes and getting this error
Exception encountered at "In [6]": Error in if (file == "") file <- stdout() else if (is.character(file)) {: argument is of length zero Traceback: 1. write.csv(countries, product$countries) 2. eval.parent(Call) 3. eval(expr, p) 4. eval(expr, p) 5. write.table(countries, product$countries, col.names = NA, sep = ",", . dec = ".", qmethod = "double")
the R code is failing
# + tags=["parameters"] upstream = NULL product = NULL # -
these are the tags
e
is this the output of 
ploomber build
?
m
ploomber build Executing: 86%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 6/7 [0002&lt;0000, 2.35cell/s] Building task 'downloadData': 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [0002&lt;0000, 2.60s/it] =========================================================================================== DAG build failed =========================================================================================== ------------------------------------- NotebookRunner: downloadData -> MetaProduct({'data': File('00-data...ountries.csv'), 'nb': File('00-data...adData.ipynb')}) -------------------------------------- -------------------------------------------- C:\Users\yosty\Desktop\Desktop_Folder\14 - git\timeSeriesDOTS\ploomber\dots\00-data_collection\downloadData.R --------------------------------------------- --------------------------------------------------------------------------- Exception encountered at "In [6]": Error in if (file == "") file <- stdout() else if (is.character(file)) {: argument is of length zero Traceback: 1. write.csv(countries, product$countries) 2. eval.parent(Call) 3. eval(expr, p) 4. eval(expr, p) 5. write.table(countries, product$countries, col.names = NA, sep = ",", . dec = ".", qmethod = "double") ploomber.exceptions.TaskBuildError: Error when executing task 'downloadData'. Partially executed notebook available at C:\Users\yosty\Desktop\Desktop_Folder\14 - git\timeSeriesDOTS\ploomber\dots\00-data_collection\downloadData.ipynb ploomber.exceptions.TaskBuildError: Error building task "downloadData" =========================================================================================== Summary (1 task) =========================================================================================== NotebookRunner: downloadData -> MetaProduct({'data': File('00-data...ountries.csv'), 'nb': File('00-data...adData.ipynb')}) =========================================================================================== DAG build failed ===========================================================================================
this is the ipynb
e
can you share your pipeline.yaml?
m
# pipeline.yaml tasks: - source: 00-data_collection/downloadData.R product: nb: 00-data_collection/downloadData.ipynb counterparts: 00-data_collection/metadata/counterparts.csv countries: 00-data_collection/metadata/countries.csv
e
please share the broken notebook file
m
hmmm it worked
# + tags=["parameters"] upstream = NULL product = NULL # -
I changed these tags
but it was like this before, so not sure
ploomber build Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [0002&lt;0000, 2.75cell/s] Building task 'downloadData': 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [0002&lt;0000, 2.95s/it] name Ran? Elapsed (s) Percentage ------------ ------ ------------- ------------ downloadData True 2.94519 100
e
ok so it's working now
m
yes!
e
yeah, my guess is that there was some error with the tags
👍 1
great to see it working, what are you building?
m
I've been playing around with trade networks for fun https://rcyost.github.io/xgboost-trade-forecast
but its expanded into several data collect files and analysis files which is getting unmanageable
looking to string it all together with ploomber
unfortunately the dbnomics API offers metadata in R client and better data downloading in python client, so the downloading is in R +python haha
eventually I'd like this to be hosted on a cloud and have a front end, will look to leverage ploomber as much as possible!
e
sounds great! please keep us posted with progress and send any questions you may have!
l
folder
3 Views
Open in Slack
PreviousNext
Powered by