Hi guys, I have the following scenario, for which I do not yet, what´s a good practice to handle it. I am subscribed to a mqtt topic. On a new message, a hamilton pipeline is executed, that gehts the message payload and topic as input. Every payload has a "config" an "data" part. The config holds metadata and control flags. The data in the payload final written into a delta table. Unfortunately, the data is not identical over all messages, so the data processing in the pipeline might be not exactly identical. Is it possible somehow, to use only one pipeline for several, slightly different data processing flows? Can I control the flow based on the inputs or any node output somehow?

What are the differences between them?

Can I control the flow based on the inputs or any node output somehow?

I assume you want something like tree execution? E.g. go down this branch if X > 3? With the graceful error adapter you could hack something similar in. You’d short circuit the graph in the right places by throwing a sentinel error…

Nice Idea. Thanks. Will look into this adapter and give it a try.