Hi guys, I have updated and refactored Flowerpowe...
# general
v
Hi guys, I have updated and refactored Flowerpower. Give it a try. I am happy for any feedback. FlowerPower is a simple workflow framework based on two fantastic Python libraries: • Hamilton: Creates DAGs from your pipeline functions • APScheduler: Handles pipeline scheduling https://github.com/legout/flowerpower
🔥 2
🙌 2
Hi guys, I have added the following features to FlowerPower: • MQTT based pipeline trigger: A pipeline can subscribe to a mqtt topic and runs once a new topic message is published • HTTP based pipeline trigger: You can trigger a pipeline execution using http requests • Easy configuration of Hamiltons OpenTelemetry Adapter that exports to Jaeger
🙌 3
🔥 2
FlowerPower continued: Next feature implemented: • Flowerpower pipelines and configs can be loaded from any fsspec compatible filesystem (s3, gcfs, adlfs, github, gitlab,...). This means, you can easily run your Flowerpower pipelines on any machine with Flowerpower installed or by using a Flowerpower Docker container on a AWS Lambda, AWS ECS, Google Cloud Run, Fly.io,...
🙌 3
t
By the way, let me know if you'd like the OpenTelemetry adapter to emit more information!
v
Haven´t looked in detail into the Telemetry exported data yet. I think the most important are (and I do not mind, whether this info comse from opentelemetry or the hamiton ui tracker): • success info for each run • logs for each run, especially for debugging, when pipeline runs fails from time to time