Hey guys! Which pipeline/workflow system have you guys been using for managing your Pinot data, in case you're happy with the pipeline system? I'd like to evaluate a few different options to check which ones would suite me best. So far I only tried Airflow and, I'll be honest, didn't like it that much. So I'm looking at possible alternatives.

Specifically for ingesting data - you can evaluate Pinot minion based ingestion

Not for the ingestion itself - this I'm already doing in the minion -, but rather data pipeline management systems, like Airflow, Luigi, Argo etc

Ok, then I’ll let the community chime in. Thoughts @Ken Krugler ?

One of our clients uses Airflow, and they seem happy with it. But that’s up to their ops person, who’s a Python programmer.

Ah, cool. I'm a Python programmer too. I'm currently taking a look at Argo, seems like a good solution too. Thanks!