Hi Team, i am trying to write

UDF - Scalar function

steps i've followed is as below: created java project with the package name

org.apache.pinot.scalar.ScalarFunc

. Pom file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="<http://maven.apache.org/POM/4.0.0>"
         xmlns:xsi="<http://www.w3.org/2001/XMLSchema-instance>"
         xsi:schemaLocation="<http://maven.apache.org/POM/4.0.0> <http://maven.apache.org/xsd/maven-4.0.0.xsd>">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.apache.pinot.scalar.ScalarFunc</groupId>
    <artifactId>ScalarFunc</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.pinot</groupId>
            <artifactId>pinot-common</artifactId>
            <version>0.11.0</version>
        </dependency>
    </dependencies>


</project>

Main.java

package org.apache.pinot.scalar.ScalarFunc;

import org.apache.pinot.spi.annotations.ScalarFunction;

public class Main {
    public static void main(String[] args) {
//        System.out.println("Hello world!");
    }

    @ScalarFunction
    static String getdata(String ref){
        return "hurray testing is working";
    }
}

created jar file using

mvn clean install

and moved that jar file to

pinot/plugins.

and than reinstalled the pinot. tried to run

select getdata(AirTime) from airlineStats limit 10

this query. here AirTime is column name. but its returning error as below:

[
  {
    "message": "QueryExecutionError:\norg.apache.pinot.spi.exception.BadQueryRequestException: Unsupported function: getdata not found\n\tat org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:304)\n\tat org.apache.pinot.core.operator.transform.TransformOperator.<init>(TransformOperator.java:65)\n\tat org.apache.pinot.core.plan.TransformPlanNode.run(TransformPlanNode.java:71)\n\tat org.apache.pinot.core.plan.SelectionPlanNode.run(SelectionPlanNode.java:71)",
    "errorCode": 200
  }
]

am i doing anything wrong? reference: https://docs.pinot.apache.org/users/user-guide-query/scalar-functions#scalar-functions

/**
 * Registers the scalar functions via reflection.
 * NOTE: In order to plugin methods using reflection, the methods should be inside a class that includes ".function."
 *       in its class path. This convention can significantly reduce the time of class scanning.
 */
static {
  long startTimeMs = System.currentTimeMillis();
  Set<Method> methods = PinotReflectionUtils.getMethodsThroughReflection(".*\\.function\\..*", ScalarFunction.class);

Also, make the method public

Thank you @Kishore G i've changed main file and created new scalarfunction file as below: ScalarFunction.java

package org.apache.pinot.scalar.ScalarFunc;

import java.lang.annotation.Annotation;

public class ScalarFunction implements Annotation {

    @Override
    public Class<? extends Annotation> annotationType() {
        return null;
    }


    @org.apache.pinot.spi.annotations.ScalarFunction
    static public String getdata(String ref){
        return "ye ye testing is working";
    }
}

Main.java

package org.apache.pinot.scalar.ScalarFunc;
import org.apache.pinot.spi.utils.PinotReflectionUtils;

import java.lang.annotation.Annotation;
import java.lang.reflect.Method;
import java.util.Set;

public class Main {
    public static void main(String[] args) {
//        System.out.println("Hello world!");
    }

    static {
        long startTimeMs = System.currentTimeMillis();
        Set<Method> method = PinotReflectionUtils.getMethodsThroughReflection(".*\\.function\\..*",org.apache.pinot.scalar.ScalarFunc.ScalarFunction.class);
    }
}

is this okay, or i need to change anything?

Hey Vishal, the Annotations were moved to

pinot-spi

module in recent release. I will updated this article to reflect it. For now, can you try changing the dependency to

<dependencies>
        <dependency>
            <groupId>org.apache.pinot</groupId>
            <artifactId>pinot-spi</artifactId>
            <version>0.11.0</version>
        </dependency>
    </dependencies>

Thanks for your response @Kartik Khare, i am using

@org.apache.pinot.spi.annotations.ScalarFunction(names = "getdata")

from spi only. and using

0.11.0

version

Ok, I saw the code. Please. change your packge from

package org.apache.pinot.scalar.ScalarFunc;

to ``package org.apache.pinot.custom.function;`

package org.apache.pinot.custom.function;

public class ScalarFunction {
    private ScalarFunction() {
    }

    @org.apache.pinot.spi.annotations.ScalarFunction(names = "getdata")
    public static String getdata(String ref){
        return "ye ye testing is working";
    }
}

changed as above but still same🥲

Yep, that's fine. Can you try removing the names thing and also copying the jar to lib/ folder instead

okay let me try

Will it be possible to send jar file here. Also the command you are using to launch pinot

Jar file:

It's working on my end. Used your jar without any changes. Just copied it to lib folder ran

bin/quick-start-batch.sh

~/apache-pinot-0.11.0-bin % ls -lh lib
total 303976
-rw-r--r--@ 1 kharekartik  staff   3.4K Dec 22 13:01 ScalarFunc-1.0-SNAPSHOT.jar
-rw-r--r--  1 kharekartik  staff   148M Sep  5 02:27 pinot-all-0.11.0-jar-with-dependencies.jar

Okay let me try to download from website