https://ploomber.io/ logo
Join Slack
Powered by
This message was deleted.
# ask-anything
s
This message was deleted.
i
Is this for committing code? If yes, it’s better you start with good first issues. There’s no plan atm to have a cli only interface for the telemetry.
e
please check this out @Vinay Narayana 🙂 feel free to post comment on any of them so we assign it to you https://github.com/ploomber/ploomber/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22
v
I would like to try this if this is unassigned https://github.com/ploomber/ploomber/issues/524
e
Any other that you are interested? That one has been taken
v
I can try https://github.com/ploomber/ploomber/issues/429 I guess it is unassigned.
👍 1
Another one deepcopy dag error message improvement https://github.com/ploomber/ploomber/issues/571
👍 1
e
Sure. Please comment on them so I can assign them to you
👍 1
i
You can also go over the contributing guide and start a pr, doesn’t look like these are assigned 🙂
👍 1
v
Hi, added test case for issue#429, let me know if there are any comments
Hi @Eduardo, I'm working with ploomber on windows machine now and I'm getting following error 
------------------------------------------------------------------------------- C:\dev\python\contrib\templates\ml-basic\fit.py --------------------------------------------------------------------------------
Copy code
Traceback (most recent call last):
  File "c:\dev\python\contrib\ploomber\src\ploomber\tasks\abc.py", line 561, in _build
    res = self._run()
  File "c:\dev\python\contrib\ploomber\src\ploomber\tasks\abc.py", line 670, in _run
    self.run()
  File "c:\dev\python\contrib\ploomber\src\ploomber\tasks\notebook.py", line 573, in run
    self._converter.convert()
  File "c:\dev\python\contrib\ploomber\src\ploomber\tasks\notebook.py", line 139, in convert
    self._from_ipynb(self._path_to_output, self._exporter,
  File "c:\dev\python\contrib\ploomber\src\ploomber\tasks\notebook.py", line 195, in _from_ipynb
    path.write_text(content)
  File "c:\program files\python38\lib\pathlib.py", line 1256, in write_text
    return f.write(data)
  File "c:\program files\python38\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\ue6c6' in position 237531: character maps to <undefined>
e
ah, yeah. I've seen it before. can you open the 
pipeline.yaml
? you'll see a path that has a 
.html
extension. change it for ipynb
there's some incompatibility with one dependency on windows
👍 1
v
Yes, it's working now. Another one `FileExistsError`: (expected behavior overwrite right?)
Copy code
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\dev\\python\\contrib\\templates\\ml-basic\\output\\nb.ipynb' -> 'C:\\dev\\python\\contrib\\templates\\ml-basic\\output\\nb.html'
Steps to reproduce: 1. ploomber build example 2. update 
pipeline.yaml
product nb file extension This error popped up when I updated 
pipeline.yaml
file
🙌 1
e
thanks for reporting this, can you open an issue?
v
sure
e
tahnks!
v
Hi, please ignore commit messages mentioned in Issue #571 I removed auto mention of issue id in my commits now
e
sure np
v
Hi @Eduardo, made changes as per comments but not sure why CI macOS taking time. It happened yesterday as well. Here is the link for actions tab https://github.com/ploomber/ploomber/runs/5363227128?check_suite_focus=true
e
yeah, I noticed the same thing with one of my builds. some tests require R so we install it as part of the CI, looks like the server where we pull R isn't responding reliably. no worries, we can ignore it
👍 1
i'll review the PR later today, thanks for your contribution!
👍 1
v
Hi @Eduardo / @Ido (Ploomber) any thoughts on Sankey charts related to issue #540: replace dependency on pygraphviz? Also, according to the article author wants us to use js based solutions one of them is d3-dag. This is js integration as far as my understanding and research. Let me know if you have any other understanding or expectations.
i
did you try looking on d3-dag?
What’s wrong with it?
v
It requires either npm package or html template as per this doc at the moment graphviz is handling and rendering output for us handling all the intermediate steps
e
can we bundle the package? maybe if the user does 
ploomber build
we could download the library and call it to generate the diagram?
and once it's local, then we don't need to download again
i guess this is all we need
Copy code
<https://unpkg.com/d3-dag@0.9.0>
v
yes, this can be used for html template removing npm dependency. We have to write DAG building logic to be passed to template. I'm yet to test this for small poc and following are expectations order wise (previous is mandatory for current step): 1. Bundling library to check on install, code is rendering some dummy graph 2. Dynamic graph generation through Python dag builder logic 3. Check for available customization options (wrapping/ sizing nodes and any on hover functionalities on node and at least better or almost similar representation to existing plots) do share additional inputs/ articles if you've any.
e
sounds good. thanks a lot for working on this! asking users to install pygraphviz has been a big pain point since the early days and this will have a big impact on everyone!
v
Hi @Eduardo & @Ido (Ploomber), I tried steps 1 and 2 for small poc and attached image is the output and along with html output file and function that generates it. For python to interact with templated html we require jinja2 dependency. I'm currently exploring on d3js stack for the dynamic graph generation in required format and progress so far made is based on tweaked examples.
Copy code
def gen_plot():
    print("executing gen_plot")
    template= jinja2.Template(open('./assets/index.html').read())
    updated = template.render({
        'custom_heading': "Python generated",
        'json_data': open("./assets/data-dag.json").read()
    })
    with open("out.html", 'w') as f:
        f.write(updated)
out.html
🙌 1
e
this looks great! Thanks a lot for working on this. we already use jinja2 so that's fine
👍 1
i
@Vinay Narayana This is great! I saw that in the PR. Is there a way to print the products of the tasks as well? Does the colors on the paths mean anything? Also the color are still indicating the execution status right?
v
Hi @Ido (Ploomber), the color on the paths were just random ones and doesn't mean anything and shared is not a dag that got generated from ploomber but with same random dynamic data passed. Feel free to share any thoughts if you have on this. PS: got busy lately, will update on this item once I make progress.
👍 1
i
Oh so it’s just a sample from the D3 directory? I thought you had it working with a ploomber pipeline
No worries. When you do, make sure to print the products of the tasks as well? And validate that the colors are indicating the execution status?
👍 1
v
Hi, please find the sample in the git link for the d3dag https://github.com/Vinay26k/dag-visualization/tree/gh-pages Code: https://github.com/Vinay26k/dag-visualization/tree/d3dag Direct link (has both code base link + demo page): https://vinay26k.github.io/dag-visualization/ For sample I have taken the ploomber plot ml-basic dag and added code for the same to parse it through dag.py script. Command executed to generate dag data from ploomber => 
print(nx.readwrite.json_graph.node_link_data(G))
👍 1
e
awesome! can you start working on adding this code to the repo? Let me add a few pointers to the github issue, and we can discuss there
👍 1
3 Views
Open in Slack
PreviousNext
Powered by