<@U04S8PLJWMT> does Agent engine platform availabl...
# replication-ask-ai
s
@kapa.ai does Agent engine platform available for the self hosted open source airbyte?
k
Based on the knowledge sources, the Agent Engine platform is a cloud-hosted service (available at
app.airbyte.ai
) and is not the same as self-hosted open source Airbyte. Here's the distinction: • Agent Engine (hosted/platform mode): A cloud platform managed by Airbyte that handles connector management, credential storage, entity caching, and multi-tenant deployments. This requires an Airbyte subscription and credentials (
AIRBYTE_CLIENT_ID
+
AIRBYTE_CLIENT_SECRET
) from
app.airbyte.ai
. [About Agent Engine] • Open source agent connectors: If you don't subscribe to Airbyte's cloud platform, you can still use Airbyte's agent connectors as standalone Python packages directly in your app — without running the Airbyte platform at all. However, in this case, you are responsible for managing user credentials yourself, and you cannot use features like the context store. [Open source authentication] So, to directly answer your question: the Agent Engine platform (with managed credentials, UI, entity caching, etc.) is not available for self-hosted open source Airbyte — it is a cloud service. What is available for open source/self-hosted use are the agent connector Python packages, which you can import and use independently. [Agent connectors] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai what is agent connectors, what is the difference between it with the previous connector? use gitlab connector as example
k
What Are Agent Connectors? Airbyte's agent connectors are lightweight Python packages that equip AI agents to call third-party APIs through strongly typed, well-documented tools. They are designed for real-time, on-demand data access within AI agent applications — not for batch data movement. [Agent connectors] Agent Connectors vs. Traditional (Replication) Connectors [Agent connectors] GitLab Connector as an Example The GitLab agent connector (
airbyte-agent-gitlab
) connects to the GitLab REST API (v4) and gives AI agents real-time access to projects, issues, merge requests, commits, pipelines, groups, branches, releases, tags, members, milestones, and users. [GitLab connector] What it can do (supported entities & actions): • Projects, Issues, Merge Requests, Users, Commits, Groups, Branches, Pipelines, Releases, Tags, Milestones, Members — all support
List
,
Get
, and
Search
actions. What it cannot do (unsupported): • Create a new project, delete an issue, merge a merge request, or trigger a pipeline — write operations are not currently supported. [GitLab connector] Installation
Copy code
uv pip install airbyte-agent-gitlab
Usage (Open Source mode)
Copy code
from airbyte_agent_gitlab import GitlabConnector
from airbyte_agent_gitlab.models import GitlabPersonalAccessTokenAuthConfig

connector = GitlabConnector(
    auth_config=GitlabPersonalAccessTokenAuthConfig(
        access_token="<your_gitlab_personal_access_token>"
    )
)

@agent.tool_plain  # assumes you're using Pydantic AI
@GitlabConnector.tool_utils
async def gitlab_execute(entity: str, action: str, params: dict | None = None):
    return await connector.execute(entity, action, params or {})
[GitLab connector] Usage (Hosted mode — credentials managed by Airbyte Cloud)
Copy code
from airbyte_agent_gitlab import GitlabConnector, AirbyteAuthConfig

connector = GitlabConnector(
    auth_config=AirbyteAuthConfig(
        customer_name="<your_customer_name>",
        organization_id="<your_organization_id>",  # Optional for multi-org clients
        airbyte_client_id="<your-client-id>",
        airbyte_client_secret="<your-client-secret>"
    )
)

@agent.tool_plain
@GitlabConnector.tool_utils
async def gitlab_execute(entity: str, action: str, params: dict | None = None):
    return await connector.execute(entity, action, params or {})
[GitLab connector] In summary: agent connectors complement (not replace) traditional replication connectors. Use agent connectors when your AI agent needs live, on-demand data access. Use replication connectors when you need scheduled, bulk data movement into a warehouse. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
s
@kapa.ai ok, got it. this is not what I'm looking for. so if we have a ai platform, we would like to enable it with ability to create connections on our hosted airbyte, is there any mcp server support this or we need to host our own mcp?
k
Based on the knowledge sources, there are two relevant MCP server options, but with important caveats regarding your specific use case (managing connections on a self-hosted Airbyte instance): 1. Airbyte Agent Engine MCP (Hosted — mcp.airbyte.ai) This is a fully hosted, remote MCP server that requires no installation. However, it connects to Airbyte's cloud platform (
app.airbyte.ai
), not your self-hosted Airbyte instance. [Agent Engine MCP] 2. PyAirbyte MCP Server (Local — Self-Hosted) This is a local MCP server you run yourself, and it is more relevant to your use case. It provides tools to: • List available Airbyte connectors • Validate configurations • Run sync operations • Manage connections in your Airbyte Cloud workspace (via
AIRBYTE_CLOUD_CLIENT_ID
,
AIRBYTE_CLOUD_CLIENT_SECRET
,
AIRBYTE_CLOUD_WORKSPACE_ID
) You run it using
uvx
with the following config:
Copy code
{
  "mcpServers": {
    "airbyte": {
      "command": "uvx",
      "args": [
        "--python=3.11",
        "--from=airbyte@latest",
        "airbyte-mcp"
      ],
      "env": {
        "AIRBYTE_MCP_ENV_FILE": "/path/to/my/.mcp/airbyte_mcp.env",
        "AIRBYTE_CLOUD_MCP_SAFE_MODE": "1",
        "AIRBYTE_CLOUD_MCP_READONLY_MODE": "0"
      }
    }
  }
}
[PyAirbyte MCP] Important Limitation The knowledge sources only mention the PyAirbyte MCP connecting to Airbyte Cloud (via cloud credentials), not to a self-hosted open source Airbyte instance. Whether it supports connecting to a self-hosted Airbyte via its REST API is not explicitly stated in the available documentation. If your goal is specifically to manage connections on a self-hosted Airbyte, you may need to explore the PyAirbyte GitHub Issues or Discussions to confirm whether self-hosted Airbyte is supported. [PyAirbyte MCP] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.