https://pulsar.apache.org/ logo
Join Slack
Powered by
Hi everyone! I am launching a single node Pulsar ...
# general
a
Hi everyone! I am launching a single node Pulsar cluster via docker-compose and following documentation for setting the yaml file up. I would like to create a pre-configured sink and pre-configured tenants/ns/topics. I am doing this just by creatinga. script that has the pulsar-cli commands to do those operations. I would like those resources to be available to the user on broker start up. My question is, where would it be best to pass in this script? as an entrypoint in the broker container? or the pulsar-init container?
d
Neither, really — use a third, separate one-shot container that runs after the broker is healthy.upulsar-init: runs 
initialize-cluster-metadata
against ZooKeeper before the broker exists, and then exits. Your 
pulsar-admin
commands for tenants/namespaces/topics/sinks talk to the broker’s REST admin API on :8080, which isn’t up yet.
you want something like this:
Copy code
broker:
    image: apachepulsar/pulsar:4.0.6
    # ... your existing config ...
    environment:
      PULSAR_PREFIX_functionsWorkerEnabled: "true"   # needed if the sink runs in-broker
    volumes:
      - ./connectors:/pulsar/connectors              # your sink NAR
    healthcheck:
      test: ["CMD", "bin/pulsar-admin", "brokers", "healthcheck"]
      interval: 5s
      timeout: 10s
      retries: 30
      start_period: 30s

  pulsar-bootstrap:
    image: apachepulsar/pulsar:4.0.6
    container_name: pulsar-bootstrap
    networks: [pulsar]
    depends_on:
      broker:
        condition: service_healthy
    volumes:
      - ./bootstrap.sh:/bootstrap.sh:ro
      - ./sink-config.yaml:/sink-config.yaml:ro
    entrypoint: ["/bin/bash", "/bootstrap.sh"]
    restart: on-failure
and the bootstrap script can use the pulsar-admin CLI tool since it’s base image is pulsar, e.g.
Copy code
#!/usr/bin/env bash
set -euo pipefail

PA="bin/pulsar-admin --admin-url <http://broker:8080>"
$PA tenants create acme 2>/dev/null || true
$PA namespaces create acme/prod 2>/dev/null || true
$PA namespaces set-retention acme/prod --size 1G --time 24h

$PA topics create-partitioned-topic <persistent://acme/prod/events> -p 3 2>/dev/null || true

$PA sinks create \
  --tenant acme --namespace prod --name my-sink \
  --sink-config-file /sink-config.yaml \
  --inputs <persistent://acme/prod/events> 2>/dev/null \
  || $PA sinks update --tenant acme --namespace prod --name my-sink \
       --sink-config-file /sink-config.yaml

echo "bootstrap complete"
a
gotcha, this approach makes much more sense. Thank you! 🙂
🙏 1
2 Views
Open in Slack
PreviousNext
Powered by