Hello community - I have a use case to have custom...
# generalp
Pankaj Kaushal
10/14/2024, 10:52 AMHello community - I have a use case to have custom function. I tried to achieve the same using ExprMacroTable.ExprMacro. But i am unable to do. The UDF is not getting registered with druid. Can some one help on this ?
l
Laksh Singla
10/14/2024, 11:18 AMCan you please share how you are trying to register the custom function (mostly stuff in the *Module files of the extension), and how you are trying to call it?
p
Pankaj Kaushal
10/14/2024, 1:32 PMfor the custom function, i wrote a class that implements ExprMacroTable.ExprMacro.
This is how i am registering the module
public class ExampleExtensionModule implements DruidModule
{
public static final List<Class<? extends ExprMacroTable.ExprMacro>> EXPR_MACROS =
ImmutableList.<Class<? extends ExprMacroTable.ExprMacro>>builder()
.add(CustomFunctionFactory.class)
.build();
@Override
public void configure(Binder binder)
{
binder.bind(ExprMacroTable.class).to(GuiceExprMacroTable.class).in(LazySingleton.class);
for (Class<? extends ExprMacroTable.ExprMacro> exprMacroClass : EXPR_MACROS) {
addExprMacro(binder, exprMacroClass);
}
}
public static void addExprMacro(final Binder binder, final Class<? extends ExprMacroTable.ExprMacro> clazz)
{
Multibinder.newSetBinder(binder, ExprMacroTable.ExprMacro.class)
.addBinding()
.to(clazz);
}
}
select custom_fun(col1)
from "ds2"
l
Laksh Singla
10/14/2024, 1:36 PMYou’ll have to create the SQL binding if you want to use it with SQL. You can still test out that the custom function is registered properly using the native (json) query. Since the SQL binding is directly invoking the expression macro, you should look at the implementations of DirectOperatorConversion
p
Pankaj Kaushal
10/14/2024, 1:46 PMthanks a lot, i will try that.
Pankaj Kaushal
10/15/2024, 1:22 PM@Laksh Singla I followed DirectOperatorConversion.
This is how i am configuring module.
Copy code
public class ExampleExtensionModule implements DruidModule
{
@Override
public void configure(Binder binder)
{
SqlBindings.addOperatorConversion(binder, CustomOperatorConversion.class);
ExpressionModule.addExprMacro(binder, CustomExprMacro.class);
}
}l
Laksh Singla
10/16/2024, 4:22 AMis there a stack trace attached? Run the query with “debug”:true, and maybe that will help see the error
p
Pankaj Kaushal
10/16/2024, 5:01 AM@Laksh Singla I can see this in broker.log. I dont see any error in other logs.
l
Laksh Singla
10/16/2024, 5:06 AMYou don’t have to turn on the debug logging. You’d wanna set “debug”:true in the query context
p
Pankaj Kaushal
10/16/2024, 5:07 AMThis is the query i am using, SELECT sleep_t("name") from "ds2".
I have taken reference from https://github.com/apache/druid/tree/master/extensions-core/testing-tools/src/main/java/org/apache/druid.
I have changed implementation of eval method for my class that implements ExprMacro to return string type.
Pankaj Kaushal
10/16/2024, 5:10 AM@Laksh Singla After enabling “debug”:true in the query context. I am getting
druidException / Error while applying rule DruidQueryRule(SORT_PROJECT), args [rel#525:LogicalProject.NONE.[](input=RelSubset#524,exprs=[SLEEP_T($2)]), rel#532:DruidQueryRel.NONE.[](query={"queryType":"scan","dataSource":{"type":"table","name":"ds2"},"intervals":{"type":"intervals","intervals":["-146136543-09-08T082332.096Z/146140482-04-24T153627.903Z"]},"resultFormat":"compactedList","limit":1001,"columns":["__time","id","name","sales","status"],"legacy":false,"context":{"debug":true,"queryId":"8d933e5f-7576-46ef-890b-60cf86cf48bc","sqlOuterLimit":1001,"sqlQueryId":"8d933e5f-7576-46ef-890b-60cf86cf48bc","sqlStringifyArrays":false,"useNativeQueryExplain":true},"granularity":{"type":"all"}},signature={__time:LONG, id:LONG, name:STRING, status:LONG, sales:LONG})]
l
Laksh Singla
10/16/2024, 5:11 AMwhat’s the query?
p
Pankaj Kaushal
10/16/2024, 5:12 AMSELECT sleep_t("name") from "ds2"
l
Laksh Singla
10/16/2024, 5:21 AMI am not sure what the issue is. You should attach a debugger and see what’s causing this issue
p
Pankaj Kaushal
10/16/2024, 5:34 AMsure, i will try.
Could you pls share any reference document to create custom function, i will follow that and see whats missing in current flow that i m using.
l
Laksh Singla
10/16/2024, 10:31 AMI don’t think there is a guide. You can look at the simple functions like those implemented by DirectOperatorConversion to figure out the issue.
Laksh Singla
10/16/2024, 10:32 AMThe easiest way to troubleshoot would be to set the breakpoint and identify the cause