Hello all, I am having a problem when using FILE to read CSV from s3 which contain LARGEINTs. I am using starrocks 4.0.6 on a shared nothing cluster. My csv contains, for example :

-39673313213476023434815093321191113562

My FILE statement looks like this :

SELECT CAST($1 as LARGEINT) FROM FILES
(
    "path" = "s3://.../test.csv",
    "format" = "csv",
    "csv.column_separator" = ",",
    "csv.row_delimiter" = "\r\n",
    "csv.enclose" = "\"",
    "csv.trim_space" = "true",
    "aws.s3.endpoint" = "...",
    "aws.s3.access_key" = "...",
    "aws.s3.secret_key" = "...",
    "aws.s3.enable_path_style_access" = "true",
    "aws.s3.ssl_enabled" = "false"
)

The output of the statement does not return the value in the file. I've tried with and without cast :

file:          -39673313213476023434815093321191113562
select cast:   -39673313213476021963817764313291030528
select nocast: -39673313213476020000000000000000000000

It seems that even when casting explicity the csv data to largeint, something is lost (precision or otherwise). Is there way to get this to load correctly, or have I encountered a bug ?

Have you tried auto_detect_types=false?

yep, it is true with auto_detect_types=false everything is read as string and the cast works fine, but it also means I need to cast manually lots of other fields in my CSV that are otherwise detected correctly. If there's no other solution, I'll go that way. I also wonder what intermediate type is detected when I don't use

auto_detect_types=false

. I can't find one that will give a final largeint with the odd suffix.

Default is to sample 500 rows, so perhaps it is just guessing int? You can make it sample all the rows and see what happens.

I've tried with the full file, via auto_detect_sample_rows, but it does the same thing. I think I'll try running the unit tests of the be and see if I can understand better.

the auto sampling will convert numbers into double which causes the precision lost.

I understood that without the cast it was losing precision that corresponds to a double, as in the "select nocast" value, but when casting the value from what I'd expect to be the double into largeint it "invents" the lost precision and I don't understand where it gets that from:

select nocast: -39673313213476020000000000000000000000
select cast:   -39673313213476021963817764313291030528

Thanks for the help, I haven't got TUs running yet, but if I get a chance I'll see if can be improved.