Running starrocks 3.5 shared data in AKS with blob...
# questions-and-troubleshooting
j
Running starrocks 3.5 shared data in AKS with blob storage, how do you move your data to a new blob storage account? I've updated everything in my helm values and redeployed. The UI shows the new blob store, but the data is still being written to the old blob store. When I go into mysql and run
DESC STORAGE VOLUME builtin_storage_volume
I see the details from my old blob storage
So it appears that there isn't a way to alter the location of the storage volume? What is the point of doing an
azcopy
of the blob storage if I have create a new storage volume and copy all of the old tables to new tables using the new volume?
Ok, well this seems to have worked
Copy code
ALTER STORAGE VOLUME builtin_storage_volume
SET (
    "azure.adls2.endpoint" = "https://<new-premium-account>.<http://dfs.core.windows.net|dfs.core.windows.net>",
    "azure.adls2.shared_key" = "<new-premium-key>"
);
But the documentation specifically says not to alter "path-related properties" or it will put the tables in read only mode? https://docs.starrocks.io/docs/3.5/sql-reference/sql-statements/cluster-management/storage_volume/ALTER_STORAGE_VOLUME/
r
In this case, does creating and dropping storage volume not work?
j
You can't drop the volume if it's in use, but yea that would be nice. I'm not convinced altering the endpoint actually worked. Everything I'm reading says that it will appear to work at first but it's just pulling from cache, and loading new data could cause problems. Has anyone actually moved their blob storage to a new account and changed the volume to point to the new storage endpoint, without having to make new tables and copy all the data anyways?
r
Please disregard my other message. Currently StarRocks does not support moving blob storage to another account.
j
Ok thank you. What about creating a new blob storage account and then a new storage volume in starrocks pointing to that storage account, setting that as the default, then creating a new database and tables, then manually copying via INSERTs into the new DB/tables from the old DB/tables? We're currently using a standard Azure storage account and the transaction costs and limits are killing us, so we really need to move to premium.
It seems like there's support for multiple storage volumes? https://docs.starrocks.io/docs/3.5/category/storage-volume/. Although it's not clear what happens if you move all of your data and set the new volume as default, then delete your old volume and storage account.
Hmm.... It looks like the
_statistics_
database uses the builtin_storage_volume. Is it possible to create new statistics database, copy all of the data, drop the old
_statistics_
database and rename the new one to
_statistics_
?
@Robert Raharjo What are our options if we need to move from a Standard Storage Account to a Premium account? Is there any way to do this with our current Starrocks Cluster or do we need to build a new Starrocks Cluster using the premium storage? I've combed through the docs as much as I can and can't get a straight answer there, if you have any of this documented that would be really helpful.
r
Yes, the easiest way is to manually create a new storage volume on your new blob account, create the db and table (use CREATE TABLE LIKE for the fastest way) and manually copying the data using INSERT statements. For the database name issue you can use ALTER DATABASE RENAME on the old one.
j
Thank you. So I can do that with the
_statistics_
database and then drop the
builtin_storage_volume
and associated storage account, just leaving my new volume
premium_storage_volume
as the default and only storage volume in starrocks?
r
You cannot drop the
builtin_storage_volume
. Once you fully migrate all the data to your
premium_storage_volume
(make sure it's enabled), set it as the default storage volume. You can then drop the databases from your
builtin_storage_volume
, disable the storage, and simply ignore it.
👍 1
j
I did this in our test cluster this morning and so far so good! Hopefully nothing unforeseen crops up in our prod cluster. Thanks for the help!