Hi team, I got another question about upsert featu...
# troubleshooting
a
Hi team, I got another question about upsert feature table. I found a table takes too much heap and I wanted to add a bytes type column for the new primary key in the following way. pk is previous primary key and it’s uuid string, and new_pk is new primary key, and use toUtf8(pk) to fill the value in all existing segments. Will it help save heap in this way?Is there any way to transform previous primary key, and store it in just 16 bytes? And what’re the right steps to make it take effect for the existing segments?
Copy code
"schemaName": "test_bytes_pk",
    {
      "name": "pk",
      "dataType": "STRING",
      "maxLength": 1024
    },{
      "name": "new_pk",
      "dataType": "BYTES",
      "maxLength": 1024
    }

transform:
     {
        "columnName": "new_pk",
        "transformFunction": "toUtf8(pk)"
      }
h
The actual question might be how to configure the column data type (BYTES or STRING), the transform function (return BYTES or STRING), and whether need to enable the hashFunction in upsertConfig, so we can achieve using 16 bytes of memory for the primary key. Or maybe there is nothing to do with the column data type, and we only need to enable the hashFunction?
Hi @Neha Pawar, could you help with this question of primary key memory usage?
Using hash function does not require changing the data type of the column.