Awesome @Luy! Please read here to configure a simple alert: https://thirdeye.readthedocs.io/en/latest/basic_config.html https://thirdeye.readthedocs.io/en/latest/templates.html

@Cyril de Catheu what is this error meaning?

can you give more context?

Could you check and configure one simple alert here? http://40.122.195.23:1426/ dataset name is "alert_system_v1" and metric is "avg_milliwatts_by_min"

Can you try this one and tell me what you get in the thirdeye logs (ie in the logs of the docker container)

detectionName: 'test'
description: 'this is a test'

# Tip: Type a few characters and look ahead (ctrl + space) to auto-fill.
metric: avg_milliwatts_by_min
dataset: alert_system_v1

# Configure multiple rules with "OR" relationship.
rules:
- detection:
  - name: detect_threshold
    type: THRESHOLD             # Configure the detection type here. See doc for more details.
    params:                     # The parameters for this rule. Different rules have different params.
      max: 7000

gle.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: org.apache.pinot.client.PinotClientException: Error when running pql:SELECT SUM(avg_milliwatts_by_min) FROM alert_system_v1 WHERE datetime_as_epoch >= 1650902400000 AND datetime_as_epoch < 1651680000000 GROUP BY dateTimeConvert(datetime_as_epoch,'1MILLISECONDSEPOCH','1MILLISECONDSEPOCH','1:MINUTES') TOP 100000\n\tat com.google.common.cache.LocalCache.loadAll(LocalCache.java:4151)\n\tat com.google.common.cache.LocalCache.getAll(LocalCache.java:4104)\n\tat com.google.common.cache.LocalCache$LocalLoadingCache.getAll(LocalCache.java:5000)\n\tat org.apache.pinot.thirdeye.detection.cache.builder.TimeSeriesCacheBuilder.fetchSlices(TimeSeriesCacheBuilder.java:107)\n\tat org.apache.pinot.thirdeye.detection.DefaultDataProvider.fetchTimeseries(DefaultDataProvider.java:104)\n\t... 13 common frames omitted\nWrapped by: java.lang.RuntimeException: fetch time series failed\n\tat org.apache.pinot.thirdeye.detection.DefaultDataProvider.fetchTimeseries(DefaultDataProvider.java:112)\n\tat org.apache.pinot.thirdeye.detection.DefaultInputDataFetcher.fetchData(DefaultInputDataFetcher.java:59)\n\tat org.apache.pinot.thirdeye.detection.components.ThresholdRuleDetector.runDetection(ThresholdRuleDetector.java:68)\n\tat org.apache.pinot.thirdeye.detection.wrapper.AnomalyDetectorWrapper.run(AnomalyDetectorWrapper.java:184)\n\t... 10 common frames omitted\nWrapped by: org.apache.pinot.thirdeye.detection.DetectionPipelineException: Detection failed for all windows for detection config id 9223372036854775807 detector detect_threshold:THRESHOLD for monitoring window 1650902400000 to 1651680000000.\n\tat org.apache.pinot.thirdeye.detection.wrapper.AnomalyDetectorWrapper.checkMovingWindowDetectionStatus(AnomalyDetectorWrapper.java:246)\n\tat org.apache.pinot.thirdeye.detection.wrapper.AnomalyDetectorWrapper.run(AnomalyDetectorWrapper.java:202)\n\tat org.apache.pinot.thirdeye.detection.DetectionPipeline.runNested(DetectionPipeline.java:266)\n\tat org.apache.pinot.thirdeye.detection.algorithm.MergeWrapper.run(MergeWrapper.java:126)\n\tat org.apache.pinot.thirdeye.detection.DetectionPipeline.runNested(DetectionPipeline.java:266)\n\tat org.apache.pinot.thirdeye.detection.algorithm.DimensionWrapper.run(DimensionWrapper.java:333)\n\t... 6 common frames omitted\nWrapped by: org.apache.pinot.thirdeye.detection.DetectionPipelineException: Detection failed for all nested dimensions for detection config id 9223372036854775807 for monitoring window 1650902400000 to 1651680000000.\n\tat org.apache.pinot.thirdeye.detection.algorithm.DimensionWrapper.checkNestedMetricsStatus(DimensionWrapper.java:431)\n\tat org.apache.pinot.thirdeye.detection.algorithm.DimensionWrapper.run(DimensionWrapper.java:352)\n\tat org.apache.pinot.thirdeye.detection.DetectionPipeline.runNested(DetectionPipeline.java:266)\n\tat org.apache.pinot.thirdeye.detection.algorithm.MergeWrapper.run(MergeWrapper.java:126)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\t... 1 common frames omitted\nWrapped by: java.util.concurrent.ExecutionException: org.apache.pinot.thirdeye.detection.DetectionPipelineException: Detection failed for all nested dimensions for detection config id 9223372036854775807 for monitoring window 1650902400000 to 1651680000000.\n\tat java.util.concurrent.FutureTask.report(FutureTask.java:122)\n\tat java.util.concurrent.FutureTask.get(FutureTask.java:206)\n\tat org.apache.pinot.thirdeye.detection.yaml.YamlResource.runPreview(YamlResource.java:881)\n\tat org.apache.pinot.thirdeye.detection.yaml.YamlResource.yamlPreviewApi(YamlResource.java:842)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)\n\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)\n\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)\n\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160)\n\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)\n\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)\n\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)\n\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)\n\tat org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)\n\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)\n\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)\n\tat org.glassfish.jersey.internal.Errors.process(Errors.java:315)\n\tat org.glassfish.jersey.internal.Errors.process(Errors.java:297)\n\tat org.glassfish.jersey.internal.Errors.process(Errors.java:267)\n\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)\n\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)\n\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)\n\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)\n\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)\n\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)\n\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)\n\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)\n\tat io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)\n\tat io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)\n\tat io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:45)\n\tat io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:39)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)\n\tat io.dropwizard.bundles.redirect.RedirectBundle$1.doFilter(RedirectBundle.java:52)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:239)\n\tat io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)\n\tat io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:67)\n\tat org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)\n\tat org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:505)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)\n\tat java.lang.Thread.run(Thread.java:750)\n","level":"ERROR","logger":"org.apache.pinot.thirdeye.detection.yaml.YamlResource","thread":"dw-616 - POST /yaml/preview?start=1650902400000&end=1651680000000&tuningStart=0&tuningEnd=0","message":"Error running preview with payload detectionName: 'test'\ndescription: 'this is a test'\n\n# Tip: Type a few characters and look ahead (ctrl + space) to auto-fill.\nmetric: avg_milliwatts_by_min\ndataset: alert_system_v1\n\n# Configure multiple rules with \"OR\" relationship.\nrules:\n- detection:\n - name: detect_threshold\n type: THRESHOLD # Configure the detection type here. See doc for more details.\n params: # The parameters for this rule. Different rules have different params.\n max: 7000","timestamp":1653402066749} front_1 | {"method":"POST","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36","uri":"/yaml/preview","requestTime":5056,"protocol":"HTTP/1.1","contentLength":1480,"remoteAddress":"208.110.84.210","timestamp":1653402066750,"status":500}

ok the error happens because this version of ThirdEye is legacy and uses PQL query language. Pinot will only use SQL in the future. TOP in the query should be replaced by LIMIT. I tried a hotfix. You can rebuild the image, relaunch docker-compose and see if it’s better. Please know that we don’t support this version of TE anymore so I won’t do more changes to the codebase.

then, what is the best way to use ThirdEye data alert for our custom csv dataset?

If you want to use this OSS ThirdEye version, you will have to do a small amounts of development to maintain the system yourself. Not much I think but this requires to look at the code a bit and do some debugging. StarTree is now offering a managed platform with both Pinot and a fully rewritten ThirdEye, in SaaS or hosted in your cloud. What is your use case for ThirdEye? Disclaimer: I’m working at StarTree.

thirdeye docker rebuilding?

yes

I rebuilt thirdeye docker, but still same error.

hum strange maybe you hit a layer cache when the image was rebuilt ? could you try rebuilding without cache?

I tried to remove docker image, and then rebuilt it.

ok did you restart the docker-compose? What is the exact error message?

give me a sec

exception":"java.net.UnknownHostException: host.docker.internal

why are you using host.docker.internal ? I understood you were using 172.17.0.1

When I deleted and rebuilt docker image, it was reset all we changed. I'll replace them soon.

what is the full error message?

thirdeye-error.txt