This message was deleted.
# hamilton-help
s
This message was deleted.
t
as stated below, you can
pip install graphviz
in your environment. Or, you can use
pip install sf-hamilton[visualization]
to install Hamilton and all the required packages for visualizations
h
pip install sf-hamilton[visualization]
worked -thx
😁 1
🔥 1
s
@Hvuj let us know how you get on with the dask integration. We have a few ideas on improvements, so if you find rough edges or I wish I could do “X”, do let us know 🙂
🙏 1
👀 1
h
@Stefan Krawczyk thx - got a question: based on these docs https://hamilton.dagworks.io/en/latest/reference/result-builders/Dask/#hamilton.plugins.h_dask.DaskDataFrameResult it says It assumes a single partition. could you elaborate? though here it says : https://hamilton.dagworks.io/en/latest/reference/graph-adapters/DaskGraphAdapter/ the only caveat is delayed and dask dtypes. i wonder how it works behind the scenes i am using compute_at_end=False and probably will do some delayed
s
> it says It assumes a single partition. > could you elaborate? that’s in reference to how to deal with “constant” values. e.g. a scalar along with a dataframe. If you look at the code you’ll see what I mean. If you find issues, fairly straightforward to inject your own version of the result builder — would then love contributions back.
👀 1
> i am using compute_at_end=False > and probably will do some delayed Note, you probably don’t want to mix and match using dask delayed with dask dataframes. But if you find a need for it, do let me know — we can maybe add a thing or two.
h
thx 🙂 yea seems to work as expected using delayed false and compute at end false (obviously) i will want to do delayed and compute at end false since ill build more tests upon other tests so having compute manually by calling .compute() on the dask df seems to work also as expected
👍 1
i was wondering if its possible to return partitions. im building a dask graph with X partitions so i wonder how it works (ill look at the code if needed) when i pass a dask.series to the driver. i think i need to create the visualize graph to understand better
s
Hamilton isn’t opinionated here — you’ll need to manage how the data is laid out.
👍 1
i was wondering if its possible to return partitions.
do you mean the number of partitions? Or a single partition? or?
h
number of partitions -
s
what would you want to do with that information? Just trying to understand.
h
if i have X partitions and i feed it to the driver and it returns all 10 as single one (unifies them?) if it works the same as dask then great if it returns all 10 as 1 - this can impact memory usage
s
Yeah so Hamilton won’t be doing anything special here; depends on your code to load, etc 🙂
h
Super great news. No idea how I stumbled upon hamilton but this is exactly what I was looking for! Great tool
Thx 😀
s
> No idea how I stumbled upon hamilton 😆 > Great tool Thanks! Tell your friends! We’d love more stars 🙂 and blog posts 😉