https://slingdata.io logo
Hi! Im replicating Postgres table to
# sling-help
i
Hi! Im replicating Postgres table to Clickhouse, in my setup sling automatically configure target table with NULL-able columns, is there a way to control that (which column is NUllbale or not) ? I tried this, but stil see colums Nullable(T) in click
Copy code
streams:
  bi.bi_tasks:
    mode: full-refresh
    primary_key: id
    update_key: create_ts
    columns:
      id: uuid | value is not null
      create_ts: timestamptz | value is not null
      is_deleted: bool | value is not null
f
value is not null
is a runtime constraint, which is evaluated as data passses through sling process. For DDL construction, right now you have to use
table_ddl
. I do think there is some UX improvement we can make here with
columns
to control nullable columns.
i
Thank you! Currently I use table_dll to define table structure I need, however with 100+ colums it look a bit messy in YAML.
f
@igrp agreed! here's the plan to improve the UX, something like this
Copy code
yaml
streams:
  bi.bi_tasks:
    mode: full-refresh
    primary_key: id
    update_key: create_ts
    columns:
      id: uuid not null # not null would translate into DDL
      create_ts: timestamptz not null  # same here
      is_deleted: bool not null | value is not null  # ddl + runtime constraints enabled here