🚀 Add tags to pipeline triggers, include tags in the pipeline trigger API response, query pipeline triggers via tags (eg.

?tag[]=some_tag&tag[]=another_tag

)

🚀 Use custom block templates in Streaming Pipelines

🚀 Customize the row limit in SQL block

🚀 Add API endpoint for trigger in the left side panel on the trigger detail page

🚀 Show the exact line number where an error occurred when running a block on the edit pipeline page

🚀 Batch run git sync instead of for each pipeline run

🚀 Support exporting data to Snowflake destination using batch upload

🚀 Set the schema of the DB used in Mage

🚀 Control how many sets of dynamic children blocks can run concurrently: https://docs.mage.ai/design/data-pipeline-management#pipeline-level-concurrency

🚀 Concurrent pipeline runs for a single pipeline, configured at a pipeline level: https://docs.mage.ai/design/data-pipeline-management#pipeline-level-concurrency

🚀 Concurrent block runs inside a single pipeline, configured at a block level: https://docs.mage.ai/design/data-pipeline-management#pipeline-level-concurrency

🚀 Customize K8s executor config: https://docs.mage.ai/production/configuring-production-settings/compute-resource#kubernetes-executor

🚀 Automatically delete logs after N days: https://docs.mage.ai/production/observability/logging#set-log-retention-period

🚀 Custom endpoint for S3 logger: https://docs.mage.ai/production/observability/logging#s3

Any plans to add debugger options ? (like visual studio code for debugging python files)

Hello, I wanted to ask, if there can be added a feature in Data Integration tool, to choose which

decoder

is used when performing the stream, or is it already developed (for example for PostgreSQL), it would be nice to use UTF-8 by default.

I noticed doing a lot of testing of the integration pipeline sources, and looking into the mage-integrations source directory, each submodule has a README which is linked to in the UI. It would be cool to have the contents of that README directly available in the front end! A first step for this could be to have the API resource for integration sources load and return the readme.md file contents.

Could we have a sap connector for data integration pipelines??

Also can we have collibra integration??

I would like to have an easier way changing the order of incoming variables. Currently it seem to depend on which parent was added first, but this can have strange behaviour when connecting that block to a new pipeline and the order is switched. Now it seems I can edit the

metadata.yaml

file myself to edit the order. But would love to have a simple way to do the ordering in the UI 🙂

Any plans to being able to use polars instead of pandas? I know currently polars is supported in addition to pandas, but would be nice, as an option, to phase out pandas completely from the pipeline.

Vim keybinds like jetbrains/vscode for editing windows

When will CDC available for other database, as of now its only available for postgresql ?

Mage is having LLM feature to generate Code block with Open API but it is showing result of inbuilt code blocks. Can we have feature similar to chatgpt ,like after prompting requirements mage will generate Code blocks for Pipeline mostly for pyspark Code?

🚀 Pipeline landing times: schedule a pipeline to run and finish by a specific time - https://docs.mage.ai/guides/schedule-pipelines#landing-times

🚀 Global data products (https://docs.mage.ai/orchestration/global-data-products/overview): Generate and orchestrate the final output of a pipeline (aka data product). Data products can be globally referenced from any pipeline and its data output can be used in any block.

🚀 Custom block and pipeline templates (shared already but here are docs for the feature): https://docs.mage.ai/custom-templates/overview

Could we get a way to filter the project-wide pipeline runs screen by things like tags, pipelines, etc.

Hi Team, We are using elastic search as one of the destinations for data from Kafka, is it possible to have that in Mage ? I couldn’t find that in singer.io

Hello, On the Pipeline Triggers page, can we have an option to automatically rerun a pipeline once it completes, regardless of whether it ends in failure or success?