This message was deleted.
# generals
Slackbot
02/26/2024, 9:58 AMThis message was deleted.
k
Karan Kumar
02/26/2024, 10:29 AMa
Abinash
02/26/2024, 10:36 AMHey Karan, I haven't set them yet... The default storage cleaner duration in 86400 seconds. Is this irrespective of whether cleaner is enabled or not?
k
Karan Kumar
02/26/2024, 10:37 AMdruid.msq.intermediate.storage.cleaner.enabledKaran Kumar
02/26/2024, 10:37 AMThis needs to be set on the overlord
Karan Kumar
02/26/2024, 10:37 AMwith the msq durable storage configs also set
a
Abinash
02/26/2024, 10:39 AMThanks !!
Abinash
02/26/2024, 12:39 PM@Karan Kumar I have tried launching deep queries(mentioned below) with 'resultFormat': 'csv' but the result received is in json format. ex: {"col1":"val1","col2":"val2"}
Is there a way that response is generated in csv form {"val1","val2"}.
Query:
{
'query': ' select * from DATASOURCE ORDER BY __time',
'resultFormat': 'csv',
'context':
{'executionMode': 'ASYNC', 'selectDestination': 'durableStorage', 'maxNumTasks': '7', 'maxRowsQueuedForOrdering': 2147483647, 'maxSegmentPartitionsOrderedInMemory': 2147483647}
}
k
Karan Kumar
02/26/2024, 12:41 PMcc @Adarsh Sanjeev would know.
a
Adarsh Sanjeev
02/26/2024, 12:50 PM@Abinash It looks like you are specifying the format outside the query context (which would be ignored).
You can specify the format in the GET request to fetch results.
https://druid.apache.org/docs/latest/querying/query-deep-storage/#get-query-results
a
Abinash
02/26/2024, 1:13 PMHi @Adarsh Sanjeev, I am sorry but what you have pointed is how to get a response or GET request. I am using following curl command to verify the result
curl -X GET 'IP:PORT/druid/v2/sql/statements/query-ID/results?page=0&resultFormat=csv'
My above comment is about POST request in which 'resultFormat' lies outside 'context' based on following documentation.
https://druid.apache.org/docs/latest/api-reference/sql-api/#request-body
a
Adarsh Sanjeev
02/26/2024, 1:26 PM@Abinash Sorry, I missed this. Do you mean that you are setting the result format both while submitting the query and while fetching the results as csv, but the results returned are in json? Could you also check with any other format? The default should be json, so I suspect that the value is not being set due to some reason.
a
Abinash
02/26/2024, 1:34 PMYes, I am setting the csv format in both POST and GET request. Are there any logs that I can check to verify resultFormat received by druid?
a
Adarsh Sanjeev
02/26/2024, 5:15 PMIf you check the logs for the task which ran in the tasks tab, you should be able to see the context which should contain the result format.
Adarsh Sanjeev
02/26/2024, 5:16 PMI tried the below query locally and it seems to work as expected and return the results in csv
Copy code
curl "<http://localhost:8888/druid/v2/sql/statements>" \
--header 'Content-Type: application/json' \
--data '{
"query": "SELECT * FROM wikipedia",
"resultFormat": "csv",
"context": {
"executionMode":"ASYNC"
}
}'a
Abinash
02/27/2024, 7:37 AM@Adarsh Sanjeev Thanks a lot for detailed answers 🙌 I realised where I was going wrong 😵💫 I am using Druid 27.0.0. But support for different result formats started in 28.0.0.0.
3 Views