-
Notifications
You must be signed in to change notification settings - Fork 354
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug:
After upgrading to logging-operator 6.1.0 (from 6.0.3) the FluentBit pods do not start anymore.
We are using AKS v1.33.3 and the deployment is done via Helm-Chart. With the old logging-operator version 6.0.3 FluentBit just works fine.
Expected behaviour:
FluentBit should work
Steps to reproduce the bug:
Deployment of the latest logging-operator version (6.1.0) on K8s 1.33.3
Additional context:
Logging k8s resource:
apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
labels:
app.kubernetes.io/instance: logging-operator
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: logging-operator
app.kubernetes.io/version: 6.1.0
helm.sh/chart: logging-operator-6.1.0
name: logging-operator
namespace: logging-operator
spec:
clusterDomain: cluster.local.
controlNamespace: logging-operator
enableRecreateWorkloadOnImmutableFieldChange: true
fluentd:
bufferStorageVolume:
pvc:
spec: {}
bufferVolumeImage:
pullPolicy: Always
bufferVolumeMetrics:
prometheusRules: true
serviceMonitor: true
bufferVolumeResources:
limits:
memory: 50M
requests:
cpu: 1m
memory: 10M
configCheckResources:
limits:
memory: 128Mi
requests:
cpu: 50m
memory: 128Mi
configReloaderImage:
pullPolicy: Always
configReloaderResources:
limits:
memory: 128Mi
requests:
memory: 32Mi
fluentOutLogrotate:
age: "10"
enabled: true
path: /fluentd/log/out
size: "10485760"
image:
pullPolicy: Always
livenessProbe:
exec:
command:
- /bin/sh
- -c
- |
LIVENESS_THRESHOLD_SECONDS=${LIVENESS_THRESHOLD_SECONDS:-300}; if [ ! -e /buffers ]; then
exit 1;
fi; touch -d date -d "@$(($(date +%s) - $LIVENESS_THRESHOLD_SECONDS))" /tmp/marker-liveness; if [ -z "$(find /buffers -type d -newer /tmp/marker-liveness -print -quit)" ]; then
exit 1;
fi;
initialDelaySeconds: 600
periodSeconds: 60
metrics:
prometheusRules: true
serviceMonitor: true
readinessDefaultCheck:
bufferFileNumber: false
bufferFreeSpace: true
bufferFreeSpaceThreshold: 90
failureThreshold: 1
initialDelaySeconds: 5
periodSeconds: 30
successThreshold: 3
timeoutSeconds: 3
resources:
limits:
memory: 1024Mi
requests:
cpu: 50m
memory: 50Mi
scaling: {}
security:
podSecurityContext:
fsGroup: 101
runAsGroup: 101
runAsNonRoot: true
runAsUser: 100
seccompProfile:
type: RuntimeDefault
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
status:
configCheckResults:
de68d16c: true
watchNamespaces:
- '*'
FluentbitAgent k8s resource
apiVersion: logging.banzaicloud.io/v1beta1
kind: FluentbitAgent
metadata:
labels:
app.kubernetes.io/instance: logging-operator
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: logging-operator
app.kubernetes.io/version: 6.1.0
helm.sh/chart: logging-operator-6.1.0
name: logging-operator
namespace: logging-operator
spec:
image:
pullPolicy: Always
livenessProbe:
httpGet:
path: /
port: 2020
metrics:
prometheusRules: true
serviceMonitor: true
readinessProbe:
httpGet:
path: /api/v1/metrics/prometheus
port: 2020
resources:
limits:
memory: 400Mi
requests:
cpu: 50m
memory: 50Mi
security:
podSecurityContext:
runAsUser: 0
seccompProfile:
type: RuntimeDefault
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- FOWNER
drop:
- ALL
privileged: false
readOnlyRootFilesystem: true
tolerations:
- effect: NoSchedule
operator: Exists
Pods
kubectl get pod -n logging-operator
NAME READY STATUS RESTARTS AGE
logging-operator-95744c9d7-4frv2 1/1 Running 0 15m
logging-operator-fluentbit-bphm7 0/1 CrashLoopBackOff 9 (11s ago) 15m
logging-operator-fluentbit-hb2vg 0/1 CrashLoopBackOff 9 (31s ago) 15m
logging-operator-fluentbit-mchdg 0/1 CrashLoopBackOff 9 (11s ago) 15m
logging-operator-fluentbit-mqwtk 0/1 CrashLoopBackOff 9 (11s ago) 15m
logging-operator-fluentd-0 2/2 Running 1 (101s ago) 15m
Logs
After some time the FluentBit pod gets a SIGTERM. No further information is visible in the logs of the FluentBit or the logging-operator pods.
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [engine] caught signal (SIGTERM)
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ warn] [engine] service will shutdown in max 5 seconds
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input] pausing tail.0
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input] pausing storage_backlog.1
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [engine] service has stopped (0 pending tasks)
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input] pausing tail.0
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input] pausing storage_backlog.1
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [output:forward:forward.0] thread worker #0 stopping...
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [output:forward:forward.0] thread worker #0 stopped
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [output:forward:forward.0] thread worker #1 stopping...
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [output:forward:forward.0] thread worker #1 stopped
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322521 watch_fd=1
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322527 watch_fd=2
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322507 watch_fd=3
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=775460 watch_fd=4
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=778229 watch_fd=5
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774495 watch_fd=6
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=796821 watch_fd=7
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=803814 watch_fd=8
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774602 watch_fd=9
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=779982 watch_fd=10
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774676 watch_fd=11
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774574 watch_fd=12
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774625 watch_fd=13
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=776182 watch_fd=14
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774526 watch_fd=15
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=776127 watch_fd=16
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=776161 watch_fd=17
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3626750 watch_fd=18
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627886 watch_fd=19
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3628575 watch_fd=20
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3625497 watch_fd=21
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627861 watch_fd=22
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3629079 watch_fd=23
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627880 watch_fd=24
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3625564 watch_fd=25
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3619522 watch_fd=26
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627252 watch_fd=27
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627813 watch_fd=28
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322449 watch_fd=29
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322451 watch_fd=30
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322450 watch_fd=31
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3661476 watch_fd=32
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3628793 watch_fd=33
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627330 watch_fd=34
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=776130 watch_fd=35
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3619585 watch_fd=36
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627720 watch_fd=37
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322463 watch_fd=38
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=780078 watch_fd=39
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774478 watch_fd=40
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=774729 watch_fd=41
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3640264 watch_fd=42
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627183 watch_fd=43
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627255 watch_fd=44
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627297 watch_fd=45
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=803811 watch_fd=46
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=780038 watch_fd=47
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3614489 watch_fd=48
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=780044 watch_fd=49
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2323274 watch_fd=50
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=800088 watch_fd=51
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627712 watch_fd=52
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3626934 watch_fd=53
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322548 watch_fd=54
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322508 watch_fd=55
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3617458 watch_fd=56
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3616607 watch_fd=57
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3618840 watch_fd=58
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3619198 watch_fd=59
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=779984 watch_fd=60
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=780017 watch_fd=61
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3629054 watch_fd=62
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3627716 watch_fd=63
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322493 watch_fd=64
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=796747 watch_fd=65
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=803037 watch_fd=66
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3624437 watch_fd=67
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322555 watch_fd=68
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322542 watch_fd=69
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3618905 watch_fd=70
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3619605 watch_fd=71
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3621124 watch_fd=72
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322531 watch_fd=73
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3628253 watch_fd=74
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3628975 watch_fd=75
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=3639679 watch_fd=76
logging-operator-fluentbit-5z5l5 fluent-bit [2025/10/15 10:07:18] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=2322547 watch_fd=77
kubectl describe pod -n logging-operator logging-operator-fluentbit-bphm7
Name: logging-operator-fluentbit-bphm7
Namespace: logging-operator
Priority: 0
Priority Class Name: medium
Service Account: logging-operator-fluentbit
Node: aks-nodepool-19102626-vmss00004y/10.1.0.4
Start Time: Wed, 15 Oct 2025 12:56:51 +0200
Labels: app.kubernetes.io/instance=logging-operator
app.kubernetes.io/managed-by=logging-operator
app.kubernetes.io/name=fluentbit
controller-revision-hash=5b5c6876bd
pod-template-generation=5
Annotations: checksum/fluent-bit.conf: 14de2df863185dc709be8b1f106a8cad8e63c3677eb1618e3d0c4354cf049bbe
Status: Running
IP: 10.244.3.66
IPs:
IP: 10.244.3.66
Controlled By: DaemonSet/logging-operator-fluentbit
Containers:
fluent-bit:
Container ID: containerd://039a56b641382e2a195f23acf6f8e257403b8054e5c9cfa949572f188fb5df24
Image: ghcr.io/fluent/fluent-bit:4.0.3
Image ID: ghcr.io/fluent/fluent-bit@sha256:d6cc0a0a45a4fc61acde53939ed91672ed6fada7bb87710976309c0831232a79
Port: <none>
Host Port: <none>
Command:
/fluent-bit/bin/fluent-bit
-c
/fluent-bit/etc-operator/fluent-bit.conf
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 15 Oct 2025 13:17:33 +0200
Finished: Wed, 15 Oct 2025 13:18:02 +0200
Ready: False
Restart Count: 11
Limits:
memory: 400Mi
Requests:
cpu: 50m
memory: 50Mi
Liveness: http-get http://:2020/ delay=0s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:2020/api/v1/metrics/prometheus delay=0s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/buffers from buffers (rw)
/fluent-bit/etc-operator from config (rw)
/tail-db from positiondb (rw)
/var/lib/docker/containers from varlibcontainers (ro)
/var/log/ from varlogs (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-drb9d (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
varlogs:
Type: HostPath (bare host directory volume)
Path: /var/log
HostPathType:
varlibcontainers:
Type: HostPath (bare host directory volume)
Path: /var/lib/docker/containers
HostPathType:
config:
Type: Secret (a volume populated by a Secret)
SecretName: logging-operator-fluentbit
Optional: false
positiondb:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
buffers:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-drb9d:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
Optional: false
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 21m default-scheduler Successfully assigned logging-operator/logging-operator-fluentbit-bphm7 to aks-nodepool-19102626-vmss00004y
Normal Pulled 21m kubelet Successfully pulled image "ghcr.io/fluent/fluent-bit:4.0.3" in 503ms (503ms including waiting). Image size: 45295166 bytes.
Normal Pulled 20m kubelet Successfully pulled image "ghcr.io/fluent/fluent-bit:4.0.3" in 652ms (652ms including waiting). Image size: 45295166 bytes.
Normal Pulled 19m kubelet Successfully pulled image "ghcr.io/fluent/fluent-bit:4.0.3" in 965ms (965ms including waiting). Image size: 45295166 bytes.
Warning Unhealthy 19m (x8 over 21m) kubelet Liveness probe failed: Get "http://10.244.3.66:2020/": dial tcp 10.244.3.66:2020: connect: connection refused
Normal Pulled 19m kubelet Successfully pulled image "ghcr.io/fluent/fluent-bit:4.0.3" in 471ms (471ms including waiting). Image size: 45295166 bytes.
Normal Started 18m (x5 over 21m) kubelet Started container fluent-bit
Normal Created 18m (x5 over 21m) kubelet Created container: fluent-bit
Normal Pulled 18m kubelet Successfully pulled image "ghcr.io/fluent/fluent-bit:4.0.3" in 470ms (470ms including waiting). Image size: 45295166 bytes.
Warning BackOff 11m (x31 over 18m) kubelet Back-off restarting failed container fluent-bit in pod logging-operator-fluentbit-bphm7_logging-operator(8ef01935-0c7f-4ce9-945b-784045d11c2b)
Normal Killing 6m19s (x10 over 20m) kubelet Container fluent-bit failed liveness probe, will be restarted
Normal Pulling 77s (x11 over 21m) kubelet Pulling image "ghcr.io/fluent/fluent-bit:4.0.3"
Warning Unhealthy 76s (x56 over 21m) kubelet Readiness probe failed: Get "http://10.244.3.66:2020/api/v1/metrics/prometheus": dial tcp 10.244.3.66:2020: connect: connection refused
Environment details:
- Kubernetes version: v1.33.3
- Cloud-provider/provisioner: AKS
- logging-operator version: 6.1.0
- Install method: Helm
- Logs from the misbehaving component (and any other relevant logs):
/kind bug
hahawu42 and husira
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working