@amanda.folson Do we have an issue with follow-ups to the new ToC column? I spotted a few small issues already: • If the ToC exceeds the height of the browser viewport, there’s no way to scroll down to get at the lower links • If a long heading breaks over multiple lines, the list becomes hard to parse because it’s hard to tell apart 1 heading over multiple lines from multiple short headings on their own line

List of all issues labeled with "documentation" https://github.com/meltano/meltano/labels/documentation

I added a few this morning post-migration. Some need to be cleaned up (more labels, more details) but it looks like everything is where it should be 🎉

@amanda.folson Do we already have an issue for changing the links in https://docs.meltano.com/contribute/ and elsewhere from GitLab to GitHub?

Popped https://github.com/meltano/meltano/issues/5969 on the

env

key

@taylor @amanda.folson This page currently suggests that we don’t support analysis at all, but that will change with Superset support in 2.0: https://docs.meltano.com/guide/analysis. Do we have an issue to update that?

@amanda.folson small docs PR for you to review https://github.com/meltano/meltano/pull/5990

Do we have an issue for changing https://docs.meltano.com/contribute/merge to cover GitHub PRs rather than merge requests?

@amanda.folson Found a broken link in https://docs.meltano.com/guide/production#meltano-ui:

meltano user add

under the second bullet points at https://docs.meltano.com/guide/command-line-interface.html#user, which should be https://docs.meltano.com/reference/command-line-interface#user. Do we have an issue for things like this? Or automatic link checks?

Does https://docs.meltano.com/reference/command-line-interface#job put jobs in meltano.yml? What does that look like? It’s not documented under https://docs.meltano.com/concepts/project#meltanoyml-project-file cc @taylor

Is there any documentation for the meltano python core library ? I was struggling to find one

Spotted non-working import statements for paginators in guide: https://sdk.meltano.com/en/latest/guides/pagination-classes.html

from singer_sdk.paginators import BaseHATEOASPaginator
from singer_sdk.paginators import BaseOffsetPaginator

should be

from singer_sdk.pagination import BaseHATEOASPaginator
from singer_sdk.pagination import BaseOffsetPaginator

according to https://sdk.meltano.com/en/latest/reference.html#pagination and https://github.com/meltano/sdk/blob/main/singer_sdk/pagination.py

Good day. There's a dead link to the GitLab container registry on this page (https://docs.meltano.com/guide/containerization). I wanted to just submit a contribution, but I am not sure about the direction there. It would make sense to link to ghcr, but packages has not been enabled on Meltano yet (https://github.com/orgs/meltano/packages), so there is nothing really to link to. Additionally, it seems that the ghcr publishes were put in place mostly for the scanning, but I am not sure if the use has since expanded to make images generally available via ghcr.

Hi, is it possible to check version 2.2x documentation. We are still not on the upgrade path for Meltano

Hi! Not on the website at the moment but you can explore the

v2.20.0

tag in GitHub: https://github.com/meltano/meltano/tree/v2.20.0/docs You could also try git-checking out that tag and building the docs.

Hello, I'm concerned the documentation for https://hub.meltano.com/extractors/tap-mongodb--meltanolabs/#settings may be incorrect - alternatively, there may be a bug - is this the best place to discuss, or would #C068YBQQF1V be better?

Hi all, I've been doing some reading on incremental replication on Meltano's documentation (https://docs.meltano.com/guide/integration/#replication-methods) and I am kind of confused by the following code that appears above in https://docs.meltano.com/guide/integration/#setting-metadata. Isn't it a requirement to always include a

replication-key

if you are doing a

replication-method

of INCREMENTAL?

extractors:
          - name: tap-postgres
            metadata:
              some_entity_id:
                replication-method: INCREMENTAL
                replication-key: id
              other_entity:
                replication-method: FULL_TABLE
              "*":
                replication-method: INCREMENTAL
              "*_full":
                replication-method: FULL_TABLE

Hello Everyone 👋 I am trying to use meltano for data ingestion into snowflake data lake from various rdbms source (orcl, msssql, Postgres ) , I am glad that I found all these db extractors available already I was able to install metlano in a docker container and do basic full table sync into snowflake , planning to incorporate meltano into our dbt+airflow project if it works for all various batch data ingestion patterns we currently have Here are some patterns, i might need some help on 1. Can we use custom sql query as source instead of a table, may be I can create a view in source db and use that as source but wanted to confirm if there is a way to use sql query as source , we need custom sql since we call few functions which is hard to re design in snowflake 2. can we do a

pre sql or post sql

?, we do have some huge tables where we don’t have a primary key , so to minimize the el job run time can extract last 3 months of data (based on creation date since we know older data won’t change) from source and delete the data with same extraction logic in target table prior to starting the target load , is there a way to achieve this ? (Transformer ??) 3. also appreciate if someone can help me on overriding the table name in target , let’s say I have a table cust in source and I want it to be loaded into target as cust_source 4. For incremental extraction (using a timestamp column last_update_date ) can we override the value we pass in to timestamp column? Ex:- can we override the delta logic to go few hours back from last run time , something like

where last_update_date>= add_hours(-6,LastRunTime)

Appreciate if you can point me to docs or share any example Yml files

Hi there, has anyone managed to build something like

Powered by Fivetran

with Meltano? I need a way to allow our less technical users to authenticate the datasources so we can take over with everything else

Hi Team, I am looking for documentation from where I can find out how Meltano handle schema changes, need answers to below mentioned questions? • what will happen if a new table gets added? • what will happen if a new column gets added? • what will happen if a table is renamed? • what will happen if a column is renamed? • what will happen if a column is removed? • what will happen if a table is removed? • what will happen if a row is deleted? • what will happen if a row is updated? • How to select and unselect columns and tables

can anyone answer my question ? https://meltano.slack.com/archives/C068YB1BMD5/p1715156309562229

New installation instructions for Meltano using uv 🎉 https://docs.meltano.com/getting-started/installation/#install-meltano

Is there a documentation on how taps work to determine the latest record read from the source so that we only pull incremental data? Or is that something that's stored on job/stream? Is it that 'state' table I see behind meltano backend state db? https://sdk.meltano.com/en/latest/context_object.html it says 'state file' but I assume this is the backend state table, if I set postgres? I only see much fewer records than the number of streams being sycned (one per schedule), so is it stored somewhere else or maybe it's not tracking it properly?

Another question. For the backend state, I assume we only need either meltano[postgres] or meltano[s3] but not both? If I have remote postgres set-up, then that's all that's needed, s3 is not needed anymore? I saw that when we have both set-up, it still does create a lock file in s3.

Are the docs for "multiple variants" out of date? https://docs.meltano.com/guide/plugin-management/#multiple-variants

Hi, given the Azure credential change here https://github.com/meltano/meltano/pull/8215 I think these docs are now out of date - it only describes the connection string option, not DefaultAzureCredential https://docs.meltano.com/concepts/state_backends#azure-blob-storage Happy to make a PR if so?

Hi 👋 , Is there a CLI command for reflecting back the names of all the plugins that are part of a project? I.e. something like:

meltano config list

I've tried looking under: https://docs.meltano.com/reference/command-line-interface/#config but that only provides options to list config for a single plugin. I did find:

meltano job list

but this lists all jobs not all plugins. I also found:

meltano config meltano list

but this lists meltano configuration excluding plugins. thanks in advance 🙏

Hi everyone, I've been trying to set up the

tap-ga4

extractor, but I'm having trouble with configuring the client secrets: Meltano Tap-GA4 Extractor. My questions are: 1. Is it possible to use service account credentials instead of OAuth credentials? 2. If so, which configuration setting enables this? I attempted to pass a service account JSON (exported from GCP) as the value for the

client_secrets

setting in the

tap-ga4

configuration, but I received "Failed to set value" errors for each parameter. I would appreciate any help, or better - an example of the config file for a working extractor using GCP service account credentials.

Hey team, I'm looking for documentation or info about mapping arrays of embedded objects. For instance, the tap-shopify has an Order stream that contains an array line_items, which are the items bought in an order. Is there a way to map these line_items to their own table/stream?