Anyone building/using AI agents to automate DevOps and SRE tasks? If yes, how are you handling memory? Would love to have a quick chat about this

it's okay to have multiple backends. I have local files, notion docs, and notes apps that said, I try to have these three patterns at least lightly implemented: • a new session hook that adds 'wake-up' text to the session, which summarizes recent memory events. I typically keep this hidden, so that it only goes to the initial prompt/agent, and not to my screen. • keep a few local files that are polled by wake-up, and have a clean-up rule (like, only have x entries, or x days). You should curate what things to track, whether that's projects, tasks, lookups, or write-ups • each memory system should be tooled so that when the memory is used, your hooks will update the local files. In the end, I found this to be more powerful than trusting other script systems. I can use any backend I like, and I typically employ a simple yaml/toml cli tool to keep the bash/zsh scripts small. If you take care, you'll quickly be able to turn even ADO, github issues, and more into memory systems. To be clear, I work with diverse clients and so I don't always know what will be the next memory system. In the end, asking an agent to 'remember' something by writing to a storage backend is the easy part. Curating how the these memories cycle into and out of view will be the secret sauce every time.

thats awesome, thanks a lot for that How do you segregate memory by customer in that case, and how do you manage memory decay (old stuff that no longer works), so when you load that into the agent context, it has valid/good memory?

segregation: • the harness you use will often let you scope rules for hooks. In my case, I use either claude code or pi-coding-agent. Both let me have per project hooks that introduce recent lookups either via mcp or agent tools. • I generally have a centralized script for handling the yaml/json transforms, and then hook scripts that call it with a set of rules as ENV vars. The hook script can live in the repo of each client while the central scripts keep things DRY decay: • all the lookups have a date and are added to the top of the list when put into a ledger • I can specify either a max ledger size or max date • when the lookup script fires, it always adds first, then calculates if the ledger is too long (max size) or any entries are old (max date) I also have an optional 'archive' step that can summarize a number of entries. That's because my wake-up script uses a simple template md:

this is your wake-up for `acme`. When using the mcp/tool for this, keep these references in mind.

long term (manually edited) memory:
$yq $LONGTERM_DIR/acme.yaml .coding_standards_docs

recent lookup `acme`:
$yq $LEDGER_DIR/ledger.acme.yaml .recents

basically, I'll have the template run any lines that start with $ as command calls. Not safe for shared machines, mind you, and some sanitizing ought to be done (i.e. only allow yq or a few other cli tools) Also, I have the wakeup calculate the tokens of what it just generated, and there's an ENV for TOKEN_WARN_THRESHOLD that just pops a small message saying the token usage of a memory system is getting high.

got it, thats great do you measure "confidence" in any way, so if a memory no longer works, or works partially, you rate that so other agents know about it before deciding to use that piece of memory or not? Did you evaluate other solutions before deciding to build it yourself?

I don't have a great system for rating, but I don't think I've ever been "in wanting" for one. That said, I would recommend hooking up a skill for the wakeup and doing an 'eval' on it. I think that's the kind of low-effort work for maintaining the system. More specifically, get familiar with https://agentskills.io/skill-creation/evaluating-skills I usually have one or two skills that focus on wake-up prompts, and I try to make the wake-up templates as close to 'pure data' as is reasonable