Hi folks I m trying to set up logging from a k8s cluster I t Fluent Community #fluent-operator

Hi folks, I'm trying to set up logging from a k8s ...

jamiejackson

08/13/2024, 7:08 PM

Hi folks, I'm trying to set up logging from a k8s cluster. I tried the walk-through but it doesn't yield a working minikube setup with Elasticsearch as the output.

config.yaml

jamiejackson

08/13/2024, 7:13 PM

The first thing I notice is that

./deploy-es.sh

yields a TLS endpoint that also needs credentials. The prescribed

ClusterOutput

config doesn't take that into account. I think that

spec.es.tls = true

takes care of the TLS part but I don't know how to supply credentials. I don't think the

ClusterOutput

CR is documented (or if it is, I haven't found it), so I'm left guessing.

Pat

08/14/2024, 10:12 AM

Does

kubectl explain

give you the details for the CRD? The walkthrough was done a while ago so may need an update - it's all OSS though so can be contributed to via a PR: https://github.com/kubesphere-sigs/fluent-operator-walkthrough Looks like the CRDs are here: https://github.com/fluent/fluent-operator/blob/master/apis/fluentbit/v1alpha2/clusteroutput_types.go There definitely used to be docs somewhere for them, it should be autogenerated from kubebuilder... ah here linked from the main readme: https://github.com/fluent/fluent-operator/blob/master/docs/fluentbit.md#ClusterOutput However looks like the spec link is broken.

jamiejackson

08/14/2024, 2:24 PM

Copy code

% kubectl explain ClusterOutput
GROUP:      <http://fluentbit.fluent.io|fluentbit.fluent.io>
KIND:       ClusterOutput
VERSION:    v1alpha2

DESCRIPTION:
    ClusterOutput is the Schema for the cluster-level outputs API
    
FIELDS:
  apiVersion    <string>
    APIVersion defines the versioned schema of this representation of an object.
    Servers should convert recognized schemas to the latest internal value, and
    may reject unrecognized values. More info:
    <https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources>

  kind  <string>
    Kind is a string value representing the REST resource this object
    represents. Servers may infer this from the endpoint the client submits
    requests to. Cannot be updated. In CamelCase. More info:
    <https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds>

  metadata      <ObjectMeta>
    Standard object's metadata. More info:
    <https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata>

  spec  <Object>
    OutputSpec defines the desired state of ClusterOutput

jamiejackson

08/14/2024, 2:25 PM

Is that the resource/output that you expected?

jamiejackson

08/14/2024, 2:27 PM

FYI, I'm using operator v3.0 instead of the v1.0.2 that the walk-through would have installed under the covers, in case that matters.

jamiejackson

08/14/2024, 6:07 PM

I may have figured out how to pass credentials.

clusteroutput.yaml

jamiejackson

08/14/2024, 6:08 PM

But I haven't figured out how to use TLS/https.

tls: true

tls: On

doesn't pass validation.

jamiejackson

08/14/2024, 9:15 PM

At this point, I would settle for any working combination of the components. I'm two days into trying to get this working. If I could get something working, I could build on that success but I've got nothing to show for my work so far.

Open in Slack

Previous Next