pulsar

Custom resources for Ansible Pulsar Operator (will be used for pulsar management)

are you referring to StreamNative Operator?  please check the sn platform doc. <https://docs.streamnative.io/docs/networking>

Does the streamnative operator use pulsar operator in the backend?

<https://github.com/streamnative/charts/tree/master/charts/pulsar-operator>

<https://docs.streamnative.io/docs/deploy-pulsar>

there is an example in the first link. yes, you can add annotation to point to internal LB.

that's related to the streamnative operator though right?

when I attempt to add proxy annotations fields in the pulsarproxy crd it does not validate

the operator is a part of the SN platform.  It is not in official apache pulsar.
You need to feed custom config so operator knows what to do.

so I'm attempting something like the following:
```apiVersion: <http://pulsar.streamnative.io/v1alpha1|pulsar.streamnative.io/v1alpha1>
kind: PulsarProxy
metadata:
  name: proxy-pulsar-2
  namespace: pulsar
spec:
  brokerAddress: brokers-broker.pulsar.svc.cluster.local
  dnsNames: [broker.pulsar.svc.cluster.local]
  image: streamnative/pulsar:2.8.0.9
  issuerRef:
    name: 'pulsar'
  pod:
    resources:
      requests:
        cpu: 200m
        memory: 256Mi
  replicas: 1
  config:
    tls:
      enabled: false
    annotations:
      <http://service.beta.kubernetes.io/azure-load-balancer-internal|service.beta.kubernetes.io/azure-load-balancer-internal>: "true"```

error: error validating "pulsarproxy.yaml": error validating data: ValidationError(PulsarProxy.spec.config): unknown field "annotations" in io.streamnative.pulsar.v1alpha1.PulsarProxy.spec.config; if you choose to ignore these errors, turn validation off with --validate=false

I don't see a field for this in the crd

sn-platform helm chart works with operators.  you can use helm template to generate the CR that operator can understand.   or you can use k explain proxycluster.spec.custom to find out what you can do

k explain pulsarproxy.spec.customization
KIND:     PulsarProxy
VERSION:  <http://pulsar.streamnative.io/v1alpha1|pulsar.streamnative.io/v1alpha1>

DESCRIPTION:
     Customization allows the desired manifests of operator managed resources to
     be customized.

not super clear there, but I guess I'll use the sn-operator to see what it does

Is the sn-operator open source? trying to figure out the distinction between it and the other operators.

For reference it looks like it wasn't in custom or customization, but in "apiObjects"

```  apiObjects:
    externalService:
      metadata:
        annotations:
          <http://service.beta.kubernetes.io/azure-load-balancer-internal|service.beta.kubernetes.io/azure-load-balancer-internal>: "true"```

sn-operator will be GA soon.   currently, sn-platform helm chart is more flexible.

Yeah, so does using the sn-platform or sn-operator in a production environment (potentially) require a license with streamnative? Trying to figure out what I can look at orchestrating

also, the operators are published in operator hub with olm support.  SN will release sn-operator GA soon.

Alright, I'd imagine using the streamnative platform will eventually require some sort of licensing in the future.

if you have any issure or feature request, please submit issue to streamnative “charts” github repos.

we plan to replacing sn-platform helmchart with sn-operator.  sn-platform will stay as is (free).

We are currently just trying to POC pulsar as is to see if it works for our needs.

<https://github.com/yuweisung/pulsar-ops>

I used this repo to go through some deployment cases

Is there an operator like what Strimzi is for Kafka?

Apache License: <https://github.com/datastax/kaap>

SN: <https://github.com/streamnative/pulsar-resources-operator>

Their documentation says:
&gt; The Pulsar Resources Operator is an independent controller, it doesn’t need to be installed with the pulsar operator.
Am I to understand that there is indeed an "operator" that is provided by StreamNative?

I think you are right, maybe something changed, I've found this:  <https://artifacthub.io/packages/helm/streamnative/pulsar-operator>

Looks like it was changed on March:
<https://github.com/streamnative/charts/pull/1157/files>

As I understand it: <https://github.com/datastax/kaap>
Is now the only open-source maintained operator for deploying and managing Pulsar cluster components?

For managing Pulsar level resources, i.e topics, etc there is <https://github.com/streamnative/pulsar-resources-operator>

Anyone have a good example of combining both of these to have a fully k8s defined Pulsar cluster?

Why would you like to control Pulsar using K8S resources? What's your use-case and why do you think keeping the state of tenants or functions in K8s is better than having it on Pulsar only?

You can create tenants, namespaces, topics etc programmatically or using API. Controlling it using K8S resources is just one option SN proposes.

If you look at the controller, the operator is just using Admin API, so this is what you could do directly in most of the cases.

My point is what you need is Kaap, resources-operator is just an layer over Pulsar Admin to reflect the state of the K8S resources in Pulsar. Just a new API interface.

My point is that if I want to control the state of the pulsar resources using code I would either end up needing to use something like a Pulsar terraform provider or if using the API directly write my own state reconciliation between desired and actual state.
Using the Pulsar resources operator would bring management of Pulsar resources to the same level of abstraction as we use for everything else (we are also using Crossplane to manage non-k8s resources using k8s manifests).

Just add a few comments here:

1. pulsar-resource-operator and pulsar terraform provider are two common IaC tools that our customers use for provisioning and managing Pulsar resources. 
2. pulsar-resource-operator is independent from a cluster operator. You just need to define a PulsarConnection: <https://github.com/streamnative/pulsar-resources-operator/blob/main/docs/pulsar_connection.md> . Once you create a cluster, you can define a PulsarConnection based on the output of cluster creation. Because you just need the service urls and potentially authentication information.
3. in regards to cluster provisioning, I don't think DataStax maintains this operator anymore and also I don't think it is even production ready. StreamNative cluster operator is part of our private cloud distribution. We offer trial if you would like to try it out.

KAAP worked reasonably well for me, just one actual bug I ran into: <https://github.com/datastax/kaap/issues/175>
Apart from that I was able to couple both it and the pulsar-resources-operator to have a fairly good pure k8s setup.

<@U02P20T50SK> or <@U01B9PMHVGX> if either of you know if this is actively maintained I will submit a patch to fix it.

Yes, it's maintained even though there hasn't been much activity lately.

Patches are welcome, <@U01B9PMHVGX> or I can merge it for you.

Your issue sounds similar to this one <https://github.com/datastax/kaap/issues/156|https://github.com/datastax/kaap/issues/156>

We use it and it works well for us. I understood it’s production-ready, and in the maintenance mode

If anyone is interested I pushed up a basic example of how these two connect together here: <https://github.com/josephglanville/pulsar-k8s>

Nice <@U01V8JUPC85> I'll share it within a team.

This should do the trick: <https://github.com/datastax/kaap/pull/176>

Hmm that is weird.
So the proxy is setup to use a DNS name like `pul-function-ca` instead of `pul-function` not sure which one is wrong yet, going to boot up a test cluster and check

Ok so the difference between the two is `ca` is a cluster IP, the other is not.

I think it's fine if functions k8s pods talk to the cluster IP service.

<@UL4AYG7QT> something bungled the latest release of the `pulsar-resources-operator` helm chart.
Somehow the chart version and appVersion got messed up so when you install 0.5.4 you are actually getting 0.5.3 which means Kubernetes secrets for functions/sources/etc are bugged still.
See this excerpt from the Helm index:
```$ curl -sSL <https://charts.streamnative.io/index.yaml> | yq '.entries."pulsar-resources-operator"[0]' pr.yaml
{
  "apiVersion": "v2",
  "appVersion": "v0.5.3",
  "created": "2024-08-12T03:51:31.540993382Z",
  "description": "Pulsar Resources Operator Helm chart for Pulsar Resources Management on Kubernetes",
  "digest": "c03e92253ff6895cf64cf0266929474f97697121cebbfecb625391587fb7c8b7",
  "icon": "<https://raw.githubusercontent.com/streamnative/charts/master/static/logo.svg>",
  "kubeVersion": "&gt;= 1.18.0-0",
  "name": "pulsar-resources-operator",
  "sources": [
    "<https://github.com/streamnative/pulsar-resources-operator>"
  ],
  "type": "application",
  "urls": [
    "<https://github.com/streamnative/pulsar-resources-operator/releases/download/pulsar-resources-operator-v0.5.4/pulsar-resources-operator-v0.5.4.tgz>"
  ],
  "version": "v0.5.4"
}```

The tarball also includes the 0.5.3 CRDs;
```              secrets:
                additionalProperties:
                  description: SecretKeyRef indicates a secret name and key
                  properties:
                    key:
                      type: string
                    name:
                      type: string```

Manually updating the release to use 0.5.4 CRDs and docker image does work so the code is fine, just the release process went wrong somewhere.

Created a GH issue for tracking: <https://github.com/streamnative/pulsar-resources-operator/issues/234>

Ran into a few issues getting going with TLS using KAAP and latest upstream Pulsar images: <https://github.com/datastax/kaap/issues/177>

<@U02P20T50SK> would you accept a patch to fix <https://github.com/datastax/kaap/issues/177> by making cert transformation/keystore loading happen in an initContainer?

To be minimally disruptive/not add dependencies to folks already using lunastreaming or other images that have the extra bits we can default the container image for the initContainer to be the same as the app containers.

This way it shouldn't really change anything for existing users except the logs for that part of the startup will be emitted from init container instead.

There actually might be an even better solution to this problem which is requesting the cert-manager Issuer to create JKS stores directly which was introduced back in version 1.2 of cert-manager: <https://cert-manager.io/docs/releases/release-notes/release-notes-1.2/>

Can someone please advise on the closest compatible Kubernetes version less than 1.30  for Apache Pulsar 2.9.4?

k8s 1.29. I suggest upgrade Apache pulsar to 3.o.x <https://pulsar.apache.org/contribute/release-policy/>

what is the current state of the pulsar-operator? As far as I could tell, right now only the operators from streamnative and from datastax exist.

That's true. Apache comes with helm chart only.

We are using KAAP, it works pretty well and it's also Apache licensed.