in the table or schema_pattern variables in the ingest recipes is it possible to use regex expressions? There's a bunch of 'system' tables in my schema that I want to exclude from ingesting. Their naming convention is i<sequencenr> so would be easiest to exclude with a regex pattern...

Hi Niels! This works for me

"schema_pattern": {
        # allow only names without numbers
        "allow": ["[A-Za-z_]*$"],
        "deny": [],
    },

Thanks for the example! must have tried an invalid pattern or something. Good to see it working!

Also, pay attention to table name please. table_patterns are applied to the fully qualified table name, which depends on the database used. For example: https://demo.datahubproject.io/dataset/urn:li:dataset:(urn:li:dataPlatform:snowflake,demo_pipeline.public.all_entities,PROD)/schema Here the table name will be

demo_pipeline.public.all_entities

and not

all_entities

. So, you have to write your regex for the fully qualified name