Hi Team Is there any plan to incorporate spark in airbyte airbytehq #contributing-to-airbyte

Channels

p0-source-airtable-stream-failures

airbyte-enterprise-pricing

advice-data-warehouses

well-seeded-accounts

help-api-cli-orchestration

advice-data-orchestration

advice-data-quality

movedata-general

infra-dev-alerts

infra-dev-alerts-webhook

prod1-alerts-test

airbyte-for-power-users

memes-and-banter

movedata-future-evolution-of-the-data-stack

p0-amazon-ads-03-30

marcos-test-discourse

airbyte-plus-airflow

write-for-the-community

advice-reverse-etl

movedata-speakers

partner-productpair-hightouch

databricks-airbyte316

community-strategy

license-questions

movedata-building-connectors-in-minutes

movedata-navigating-the-modern-data-stack-with-open-source-headless-bi

movedata-data-orchestration-is-not-just-running-jobs-on-a-schedule

movedata-using-airbyte-on-day-1of-our-startup

movedata-using-airbyte-to-build-your-dremio-open-data-lakehouse

movedata-ingesting-data-with-airbyte-into-a-high-speed-data-lakehouse-using-trin

movedata-in-2023-data-trust-will-be-more-important-than-ever-before

p0-instagram-incompatible-metrics

movedata-mobilize-the-worlds-data

connector-builds

movedata-ducky-data-crunching-on-the-laptop---the-pendulum-swings

movedata-traditional-data-catalogs-will-be-replace-by-active-metadata-platforms

movedata-prefect-ing-self-hosted-airbyte

movedata-if-you-build-it-will-they-come-how-to-activate-your-modern-data-stack

movedata-data-analysts-are-setup-to-fail-and-its-our-fault

movedata-guardrails-not-stop-signs

movedata-dataops-on-the-open-modern-data-stack

movedata-better-data-testing-with-the-data-error-generating-process

movedata-open-source-communities-shape-modern-data-stacks

movedata-moving-data-reliably-ingestion-observability-are-better-together

movedata-airbyte-connection-management-scale

movedata-prep-your-pipelines---reverse-etl-and-the-coming-great-flood

movedata-cicd-for-data-building-devtest-data-environments-with-lakefs

movedata-why-bi-is-not-enough

office-hour-12-october

movedata-the-data-ecosystem-is-ready-for-etl-to-be-dead

understanding-airbyte

p0-cloud-syncs-failing-dec-15

movedata-data-engineering-is-software-engineering-and-software-engineering-is-da

p1-auto-detect-schema-issues-2-1-23

oss-master-build-failure

example-of-channel

advice-data-visualization

movedata-who-needs-a-traditional-data-warehouse-when-you-can-get-real-time-analy

advice-data-privacy

movedata-the-end-of-the-pipeline

movedata-keynote-building-the-foundations-of-data-movement

movedata-the-best-data-warehouse-is-a-lakehouse

movedata-five-causes-of-data-quality-issues

movedata-streaming-made-modern

movedata-modern-data-management---how-to-achieve-data-discovery

movedata-dev-first-open-source-data-observability-solution

movedata-building-a-real-time-user-facing-dashboard-with-airbyte-redpanda-and

movedata-a-different-way-to-work

movedata-let-your-data-team-choose-their-own-tools

movedata-bring-your-own-infra

movedata-airbyte-orchestration-in-gcp

airbyte-plus-dagster

feedback-and-requests

help-connector-development

advice-data-transformation

hacktoberfest-2022

dev-frontend6323

help-infrastructure-deployment

events-and-conferences

airbyte-cloud-pricing

help-connections-issues

shameless-plugs

advice-data-ingestion

advice-data-architecture

oss-master-build-failure

cloud-master-build-failure

contributing-to-airbyte

troubleshooting

connector-development

#contributing-to-airbyte

Title

v

Vaibhav Kumar

07/13/2021, 10:22 AM

Hi Team, Is there any plan to incorporate spark in airbyte?

u

user

07/14/2021, 6:15 AM

Hi! I'm not sure I understand your question. What do you think we should do with Spark?

c

charles

07/16/2021, 9:11 AM

As of now i could figure out we Airbyte is using plain java to sync data to destinations, so do you have any plan to leverage the power of distributed system like spark in Airbyte?

2 Views