https://pulsar.apache.org/ logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
a
As a first step I would check the logs of the broker pod which should give you an indication what is going wrong. There should be an INFO level log saying 
Searching for connectors in /some/path
o
I already checked, it does not throw any
a
Do you see a log saying 
Found connector ...
?
o
when I run this command
Copy code
curl -X POST <http://serviceurl:8080/admin/v3/sinks/reloadBuiltInSinks>
it just returns nothing the next line which indicates success else an error would have appeared, the log has nothing as well
a
It has to be in the broker log, it will be printed when the reload happens
o
ok let me check
here are the broker logs, they don't have any connectors' related error, some are also empty
a
Do you mind attaching the logs? (redacting sensitive info if any of course) Hard to say what’s really happening here
o
the logs are from the broker pods, just making sure I am on the right area
a
Yes, the proxy logs just contain the requested route usually
o
Copy code
I have no name!@pulsar-dev-broker-0:/pulsar$ cat logs/pulsar_gc_1.log 
[2023-11-21T13:12:52.813+0000][0.003s] Heap region size: 1M
[2023-11-21T13:12:52.815+0000][0.004s] Using G1
[2023-11-21T13:12:52.815+0000][0.004s] Heap address: 0x00000000f0000000, size: 256 MB, Compressed Oops mode: 32-bit
[2023-11-21T13:12:52.815+0000][0.005s] Mark closed archive regions in map: [0x00000000fff00000, 0x00000000fff6aff8]
[2023-11-21T13:12:52.815+0000][0.005s] Mark open archive regions in map: [0x00000000ffe00000, 0x00000000ffe47ff8]
[2023-11-21T13:12:53.443+0000][0.632s] GC(0) Pause Young (Normal) (G1 Evacuation Pause)
[2023-11-21T13:12:53.443+0000][0.633s] GC(0) Using 4 workers of 4 for evacuation
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) MMU target violated: 10.7ms (10.0ms/11.0ms)
[2023-11-21T13:12:53.454+0000][0.644s] GC(0)   Pre Evacuate Collection Set: 0.0ms
[2023-11-21T13:12:53.454+0000][0.644s] GC(0)   Evacuate Collection Set: 10.2ms
[2023-11-21T13:12:53.454+0000][0.644s] GC(0)   Post Evacuate Collection Set: 0.4ms
[2023-11-21T13:12:53.454+0000][0.644s] GC(0)   Other: 0.6ms
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) Eden regions: 64->0(56)
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) Survivor regions: 0->8(8)
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) Old regions: 2->5
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) Humongous regions: 4->2
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) Metaspace: 8193K->8193K(1058816K)
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) Pause Young (Normal) (G1 Evacuation Pause) 68M->12M(130M) 11.300ms
[2023-11-21T13:12:53.454+0000][0.644s] GC(0) User=0.02s Sys=0.01s Real=0.02s
[2023-11-21T13:12:54.053+0000][1.243s] GC(1) Pause Young (Normal) (G1 Evacuation Pause)
[2023-11-21T13:12:54.054+0000][1.243s] GC(1) Using 4 workers of 4 for evacuation
[2023-11-21T13:12:54.074+0000][1.263s] GC(1) MMU target violated: 11.0ms (10.0ms/11.0ms)
[2023-11-21T13:12:54.074+0000][1.263s] GC(1)   Pre Evacuate Collection Set: 0.0ms
[2023-11-21T13:12:54.074+0000][1.263s] GC(1)   Evacuate Collection Set: 19.5ms
[2023-11-21T13:12:54.074+0000][1.263s] GC(1)   Post Evacuate Collection Set: 0.8ms
[2023-11-21T13:12:54.074+0000][1.264s] GC(1)   Other: 0.2ms
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) Eden regions: 56->0(60)
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) Survivor regions: 8->4(8)
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) Old regions: 5->13
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) Humongous regions: 2->2
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) Metaspace: 18759K->18759K(1067008K)
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) Pause Young (Normal) (G1 Evacuation Pause) 68M->16M(130M) 20.570ms
[2023-11-21T13:12:54.074+0000][1.264s] GC(1) User=0.04s Sys=0.00s Real=0.02s
[2023-11-21T13:12:54.193+0000][1.383s] GC(2) Pause Young (Concurrent Start) (Metadata GC Threshold)
[2023-11-21T13:12:54.193+0000][1.383s] GC(2) Using 4 workers of 4 for evacuation
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) MMU target violated: 11.0ms (10.0ms/11.0ms)
[2023-11-21T13:12:54.214+0000][1.403s] GC(2)   Pre Evacuate Collection Set: 0.0ms
[2023-11-21T13:12:54.214+0000][1.403s] GC(2)   Evacuate Collection Set: 19.8ms
[2023-11-21T13:12:54.214+0000][1.403s] GC(2)   Post Evacuate Collection Set: 0.3ms
[2023-11-21T13:12:54.214+0000][1.403s] GC(2)   Other: 0.2ms
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) Eden regions: 13->0(60)
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) Survivor regions: 4->4(8)
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) Old regions: 13->13
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) Humongous regions: 2->2
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) Metaspace: 20437K->20437K(1069056K)
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) Pause Young (Concurrent Start) (Metadata GC Threshold) 29M->16M(130M) 20.473ms
[2023-11-21T13:12:54.214+0000][1.403s] GC(2) User=0.04s Sys=0.00s Real=0.02s
[2023-11-21T13:12:54.214+0000][1.403s] GC(3) Concurrent Cycle
[2023-11-21T13:12:54.214+0000][1.403s] GC(3) Concurrent Clear Claimed Marks
[2023-11-21T13:12:54.214+0000][1.404s] GC(3) Concurrent Clear Claimed Marks 0.020ms
[2023-11-21T13:12:54.214+0000][1.404s] GC(3) Concurrent Scan Root Regions
[2023-11-21T13:12:54.222+0000][1.411s] GC(3) Concurrent Scan Root Regions 7.539ms
[2023-11-21T13:12:54.222+0000][1.411s] GC(3) Concurrent Mark (1.411s)
[2023-11-21T13:12:54.222+0000][1.411s] GC(3) Concurrent Mark From Roots
[2023-11-21T13:12:54.222+0000][1.411s] GC(3) Using 4 workers of 4 for marking
[2023-11-21T13:12:54.229+0000][1.419s] GC(3) Concurrent Mark From Roots 7.738ms
[2023-11-21T13:12:54.229+0000][1.419s] GC(3) Concurrent Preclean
[2023-11-21T13:12:54.229+0000][1.419s] GC(3) Concurrent Preclean 0.083ms
[2023-11-21T13:12:54.229+0000][1.419s] GC(3) Concurrent Mark (1.411s, 1.419s) 7.859ms
[2023-11-21T13:12:54.230+0000][1.419s] GC(3) Pause Remark
[2023-11-21T13:12:54.234+0000][1.424s] GC(3) Cleaned string and symbol table, strings: 8050 processed, 13 removed, symbols: 58296 processed, 29 removed
this two has nothing in them
🤯 1
a
Oof - these should not be empty
o
that's what I got
from what I saw the helmchart deployment has some built in connectors by default, when I specified the volume only the one I added to the volume director exist as a file but returns an empty array when I check via api So @Alexander Preuß is there a way to add extra connector via the helm chart values ?
y
in k8s, the pod log is sending to console, you will need kubectl logs to see the log.
you need to configmap to register the nar files.
rest api just query the metadata. it is not query the directory.
o
@Yu Wei Sung please how do I configure this, this is really dealing with me
@Yu Wei Sung I also check the kubectl logs but nothing was found
the kubectl logs has just bunch of prometheus data
a
@Yu Wei Sung correct me if I’m wrong but if he is using standard apache pulsar distribution, there should be no need to register connectors via ConfigMap, it should just use the default 
ConnectorsManager
o
@Alexander Preuß I think yes, I don't have any issue with the standalone pulsar instance, I am pretty much good with that but running this on k8 is pretty much easy as well but the connectors management is kind of a headache
Hello guys, thanks for all your help, I have been able to solve the issue by updating the tags to latest
Copy code
defaultPulsarImageTag: latest
this helps me to use the latest pulsar images and it comes with the connector I was looking for
👍 1
5 Views
Open in Slack
PreviousNext
Powered by