Hey team we updated to `v0 91 1` and have started seeing the OpenObserve #general

Hey team, we updated to `v0.91.1` and have started...

Matt Monahan

07/09/2026, 2:15 PM

Hey team, we updated to

v0.91.1

and have started seeing these big "gaps" in our metrics. I was worried our ingest was messed up but it appears to be working fine if I just go query the metrics directly.

Matt Monahan

07/13/2026, 1:57 PM

Unfortunately I still get the gap even for a single host and a short interval, like this 2 hour one. I dont have the enterprise version so I cant clear the cache, but also I dont think this happened before we upgraded to version

v0.91.1

and encountered this bug.

Matt Monahan

07/13/2026, 1:57 PM

Do you think it might help for me to just delete these streams and let them re-generate?

Gaby

07/13/2026, 1:58 PM

I have seen this happen with "log" streams since like 30 versions ago

Hengfei

07/13/2026, 2:24 PM

@Gaby in the old version i got this issue, but i didn't see it for a long time.

Hengfei

07/13/2026, 2:25 PM

i have no idea what happened, i just run this promql query in our cluster select for past 1day, still working fine.

Hengfei

07/13/2026, 2:28 PM

can you try open chrome console and set:

Copy code

window.use_cache=false

and refresh data again. if you don't know how to do they, can you try add a ENV to your deploy:

Copy code

ZO_RESULT_CACHE_ENABLED=false

and restart your pod. refresh again. also, check if there any error in the console of chrome.

Gaby

07/13/2026, 2:28 PM

I will try disabling cache, that's probably the reason. Will report back later. Thanks!

🙏 1

Hengfei

07/13/2026, 2:30 PM

@Gaby if you got this issue again, you can DM me, i will help to find out the root cause.

💪 1

Matt Monahan

07/13/2026, 5:20 PM

Hey so this was our fault! The drives that O2 was using on its mapped volumes were fine, but the OS drive on the node it was on apparently filled up to 100% facepalm2 I thought i had checked that but i guess not, because once i made the node happy and brought O2 back up the dashboards can now load days worth of data even with a ton of data points. Sorry to bother y'all 🙏

Hengfei

07/14/2026, 12:54 AM

So the issue is disk usage is 100%?

Matt Monahan

07/14/2026, 1:55 AM

I think so 😅 I swear I had checked the node health but I guess not

Hengfei

07/14/2026, 2:07 AM

Okay, are you running with s3 or just a single node with local disk?

Matt Monahan

07/14/2026, 2:50 AM

Single node local disk

Hengfei

07/14/2026, 2:50 AM

Okay, got it, thank you let me know the root cause

🤘 1

Matt Monahan

07/14/2026, 2:51 AM

Yeah sorry it took me a bit to figure it out 😅

2 Views

Open in Slack

Previous Next