This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

watching @Elijah Ben Izzy at PyCon Lithuania about Hamilton ⭐

Hey everyone - I wrote a new post about Hamilton’s lineage capabilities — https://towardsdatascience.com/lineage-hamilton-in-10-minutes-c2b8a944e2e6

This message was deleted.

This message was deleted.

This message was deleted.

Hello everyone, my name is Alex, we help top-notch developers and startups find each other. 🙂 Through our platform, founders and CTOs can quickly build their engineering dream teams. And top IT specialists can find long-term remote work with competitive rates. Today we've launched on Product Hunt and would appreciate your feedback! https://www.producthunt.com/posts/expert-remote P.S. if anyone lost jobs in the current market, feel free to leave your CV here: https://developers.expertremote.io 😉

set the channel description: This is the one channel that will always include everyone. It’s a great spot for general Hamilton conversations. We’ll redirect you to a channel if there’s an appropriate one. For promotional content please use #C06071JDNLQ.

In case it’s helpful we’ve been building some

hamilton tutorials

in an aptly named repository called https://github.com/DAGWorks-Inc/hamilton-tutorials. The idea is to help give some ideas/ways to structure/iterate on Hamilton code to accomplish something. They are

notebook

focused because that’s an easy way to wrap documentation and code. For example, you can run them in google collab easily. https://github.com/DAGWorks-Inc/hamilton-tutorials/tree/main/2023-10-09 is the latest version. It contains three directories: • one for PDF summarization - i.e. LLM workflow to summarize a PDF. ◦ contains a single notebook that walks through some ways to build a LLM workflow. • one for the Titanic Data set - i.e. a data/feature engineering + ML pipeline workflow ◦ contains two notebooks ▪︎ one walks through data & feature engineering. ▪︎ the other fitting a statistical model and builds off the first. (FYI the model isn’t sophisticated). • one for some Hamilton reference code snippets ◦ this should mirror what’s in the docs, but it could be useful for you to save time looking something up. Pull requests/comments/issues suggestions welcome.

Hi all. I wanted to surface a few best practices that we have in the documentation that would be helpful to browse if you’re getting started: • https://hamilton.dagworks.io/en/latest/concepts/best-practices/ It covers things like: • migrating to Hamilton • function naming • output immutability • code organization If there’s things missing, or topics you’d like it to cover, please create an issue, or comment here 🙂

This message was deleted.

BTW does anyone want to use Hamilton to orchestrate command line tasks intermingled with regular python code? We’ve got a discussion related to SLURM here — with a generic run a command line prototype at the bottom. Would love any thoughts/input.

Hey everyone! Recently: 1. I got tired of setting up experiment tracking boilerplate code for MLFlow or Weights&Biases 2. I was excited to try the recent FastUI framework to create web apps UI on top of FastAPI (i.e., competing with Streamlit, Shiny, Dash, etc.) So I decided to build a lightweight local experiment manager plugin for Hamilton! You can access it via

hamilton.plugins.h_experiments

see the example here I go into more details about the building process in our latest blog: https://blog.dagworks.io/p/building-a-lightweight-experiment The best part about open source tools like FastAPI, FastUI, and Hamilton is that you own your platform. It's easy to get started and you can incrementally add the features YOU need!

FYI - meetup thread can be found here - https://hamilton-opensource.slack.com/archives/C03M328JBTM/p1708449808237289

Office hours here - https://hamilton-opensource.slack.com/archives/C03AJNGDGQL/p1709659885737809

FYI - https://hamilton-opensource.slack.com/archives/C03M328JBTM/p1710888451833979

Have to say, although I missed the hangout @Roel Bertens’s your template and work for the demo has been a huge help to a newbie like me trying to get a feature generation pipeline migrated over to Hamilton. Thank you very much!

I was playing around with OpenAI’s GPT functionality. So I created a quick one to help translate code to Hamilton — it still hallucinates a bit, but could be something to try — https://chat.openai.com/g/g-Yl6yWH39S-hamilton-code-engineer would love feedback. If it’s wrong you tell it and it usually corrects itself, though I still need to teach it about all the decorators…

I blogged about getting started with Hamilton for a simple workflow. If I misrepresented anything, let me know and I'll try to fix it. Thanks. https://pyright.blogspot.com/2024/07/dag-hamilton-workflow-for-toy-text.html

I fixed the DAGWorks name and link. Apologies. The DAGWorks logo is nice, but I have a visual preference for the yellow-orange-purple colored star Hamilton-Burr ones. Is it OK for me to continue to use those logos (the Hamilton one)? Thanks, Stefan.

Thanks for the kind words, Elijah. Feel free to do whatever you want with the blog. I'm working on the stuff Stefan pointed out.

The follow up blog post on tinkering with the graphviz output - it may not be of relevance to you. But you do get mentioned a fair bit and I want to make sure I don't misrepresent anything. Thanks. https://pyright.blogspot.com/2024/07/graphviz-editing-dag-hamilton-graph-dot.html

I took the big PNG Hamilton logo Stefan gave me and ran it through and online PNG to SVG converter. It appears to be legit and renders pretty nicely. Thanks.

I was looking, to no avail, for a way to set

final_vars

in the Builder, something like

driver.Builder().with_final_vars(...)...

. I understand I can achieve the same effect via

dr.execute(final_vars=final_vars)

, however providing final_vars at the build stage has a potential of greatly reducing the build time, or doesn't it? What am I missing?

Another blog post. Not super germane, but it does link to the project, present output, and talk about the project a little. Thanks. https://pyright.blogspot.com/2024/08/embedding-svg-in-graphviz-generated-svg.html

Hi guys, I am a huge fan of hamilton (many thanks to @Stefan Krawczyk and @Elijah Ben Izzy), but I´ve missed a easy way to schedule my hamilton dataflows and futhermore I´d like to be able to parameterize my as much of my production deployment using yaml files. To address these "shortcomings" I have created my own python library/framework called FlowerPower. https://github.com/legout/flowerpower FlowerPower is a simple workflow framework based on the fantastic python libraries Hamilton and APScheduler (Advanced Python Scheduler). Hamilton is used as the core engine to create Directed Acyclic Graphs (DAGs) from your pipeline functions and execute them in a controlled manner. It is highly recommended to read the Hamilton documentation and check out their examples to understand the core concepts of FlowerPower. APScheduler is used to schedule the pipeline execution. You can schedule the pipeline to run at a specific time, at a specific interval or at a specific cron expression. Furthermore, APScheduler can be used to run the pipeline in a distributed environment. In this case you need to setup a data store (e.g. postgres, mongodb, mysql, sqlite) to store the job information and an event broker (e.g. redis, mqtt) to communicate between the scheduler and the workers. At least a data store is required to persist the scheduled pipeline jobs after a worker restart, even if you run on a single machine. Regards, Volker

Logo stuff - colleague (our dba) on logo: "The star reminds me of a Spanish Shawl nudibranch" Not sure if it was intentional. The logo is one of the more attractive one's I've seen.