<@U078Y59KC3B> I am running standalone Pulsar for ...
# ask-ai
s
@RunLLM I am running standalone Pulsar for tests but it is getting killed with exit code 137. What can I do to ensure that Pulsar either uses less memory or doesn't exceed the threshold for being killed?
r
Exit code 137 typically indicates that a process was killed due to exceeding memory limits. To address this issue with Pulsar, you can try the following: 1. Adjust JVM Memory Settings: You can configure the Java Virtual Machine (JVM) options to limit the memory usage of Pulsar. Set the
-Xms
(initial heap size) and
-Xmx
(maximum heap size) options to lower values in the Pulsar configuration. For example, you can set these options in the
PULSAR_MEM
environment variable or directly in the Pulsar startup script. 2. Configure Memory Limits: If you are running Pulsar in a containerized environment, ensure that the container has sufficient memory allocated. You can adjust the memory limits in your container orchestration tool (e.g., Docker, Kubernetes) to provide more memory to the Pulsar container. 3. Optimize Pulsar Configuration: Review and optimize other Pulsar configurations that might impact memory usage, such as reducing the number of threads or adjusting the message retention settings. 4. Monitor Resource Usage: Use monitoring tools to keep an eye on the memory usage of the Pulsar process. This can help you identify if there are specific patterns or workloads that cause memory spikes. Would you like more detailed steps on any of these suggestions? Sources: - StreamNative Advanced Configuration - Pulsar Client Libraries 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM can you show me what are the exact env variables I need to set?
r
It seems likely that you can set the following environment variables to manage memory usage for Pulsar: 1. PULSAR_MEM: This variable is used to configure the JVM memory settings for Pulsar. You can set it to specify the initial and maximum heap size. For example:
Copy code
export PULSAR_MEM="-Xms512m -Xmx1g -XX:MaxDirectMemorySize=1g"
This configuration sets the initial heap size to 512 MB, the maximum heap size to 1 GB, and the maximum direct memory size to 1 GB. 2. PULSAR_EXTRA_OPTS: This variable can be used to pass additional JVM options. For example, you can disable IPv4 stack preference if needed:
Copy code
export PULSAR_EXTRA_OPTS="-Djava.net.preferIPv4Stack=false"
3. PULSAR_GC: If you need to configure garbage collection settings, you can use this variable. For example, to use the G1 garbage collector:
Copy code
export PULSAR_GC="-XX:+UseG1GC -XX:MaxGCPauseMillis=200"
These environment variables can be set in your shell or added to the Pulsar startup script to ensure they are applied when Pulsar starts. Would you like more detailed guidance on configuring these variables? Sources: - StreamNative Advanced Configuration 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM With docker compose, I am running Pulsar standalone as follows:
Copy code
pulsar:
    image: "apachepulsar/pulsar:4.0.4"
    container_name: "pulsar-res-ct"
    hostname: "pulsar-res-ct"
    command: "bin/pulsar standalone"
    environment:
      PULSAR_MEM: "-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m"
    ports:
      - "6650:6650"
      - "8086:8080"
How do I add healthcheck to this?
r
To add a health check for your Pulsar standalone container in Docker Compose, use the
healthcheck
section. For Pulsar, a simple check is to use the built-in healthcheck script. Example:
Copy code
pulsar:
    image: "apachepulsar/pulsar:4.0.4"
    container_name: "pulsar-res-ct"
    hostname: "pulsar-res-ct"
    command: "bin/pulsar standalone"
    environment:
      PULSAR_MEM: "-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m"
    ports:
      - "6650:6650"
      - "8086:8080"
    healthcheck:
      test: ["CMD", "bin/pulsar-admin", "brokers", "healthcheck"]
      interval: 30s
      timeout: 10s
      retries: 3
This uses the
pulsar-admin brokers healthcheck
command, which is the recommended way to check broker health in Pulsar containers. Would you like more details or troubleshooting tips? Sources: - Pulsar Docker Compose Example 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM how can I create a topic and namespace using the docker image?
r
To create a namespace and topic using the Pulsar Docker image, first exec into the running container, then use the
bin/pulsar-admin
CLI. Example:
Copy code
docker exec -it pulsar-res-ct bash
bin/pulsar-admin namespaces create my-tenant/my-namespace
bin/pulsar-admin topics create-partitioned-topic my-tenant/my-namespace/my-topic -p 4
This creates a namespace and a partitioned topic with 4 partitions. Would you like more details or examples for non-partitioned topics? Sources: - Pulsar Admin CLI Tutorial - How to create a topic 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM how can I create a tenant and namespace using the docker image?
r
To create a tenant and namespace using the Pulsar Docker image, exec into your running Pulsar container and use the
bin/pulsar-admin
CLI:
Copy code
docker exec -it pulsar-res-ct bash
bin/pulsar-admin tenants create my-tenant
bin/pulsar-admin namespaces create my-tenant/my-namespace
This creates a tenant named
my-tenant
and a namespace named
my-namespace
under that tenant. Would you like more details on setting admin roles or allowed clusters? Sources: - Pulsar Admin CLI Tutorial - Create a namespace 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.