Hello, I'd like to understand what the work requir...
# contributing-to-starrocks
k
Hello, I'd like to understand what the work required is for this ticket: https://github.com/StarRocks/starrocks/issues/69350 This is something I may be willing to pick up, but I'm not sure I fully understand the scope of the changes required. Would this be a change only required for the frontend? bucket transforms are super important for working large scale iceberg datalakes and without them StarRocks really struggles to ingest async MV's as it ends up needing to scan the full table all at once rather than working through smaller slices (buckets) of the data. Being able to
PARTITION BY bucket(1024, tenant_id)
would be huge.
so far I've identified that it will require changes to the sql changer to support the function in mvs. However, I imagine this also will require changes to some of the materialized view logic. Though I'm not sure where. It seems to me that if the partition expression is supported (and we already have
__iceberg_transform_bucket
). That it would be able to generate the partition values for the source table.
k
you can post your thoughts on the issue, not all the engineers are watching this channel.
k
Posted some of my thoughts on the issue. If you know someone in particular who understands this area well could you direct them to it? If I fully understand the changes required and they are relatively straight forward I may be able to implement them and open a PR
k
yeah, I can redirect the issue to the corresponding expert.
party 1