Applied AI for infrastructure: log analysis with a local model

Reading hundreds of warning lines after an anomaly is slow and repetitive. A language model can help with one specific part of the job: summarizing, grouping by probable cause and suggesting an indicative priority. It does not fix problems and it does not replace the person who administers the system. This guide describes a minimal pipeline in which the logs of a Linux host are extracted, scrubbed of sensitive data and passed to a model running locally.

The use case and its boundaries#

The task is summarization: given an excerpt of journalctl, produce a list of event groups with an estimated severity and a suggested check. The result is text to be read, not an action to be executed. This boundary decides everything else: as long as the output is a suggestion, a model mistake costs a few minutes of verification; if it became a command executed automatically, it would cost far more.

Why local#

Logs contain IP addresses, usernames, hostnames, paths and, by mistake, sometimes command parameters or credentials printed by a service. Sending them to an external service means handing third parties information about your own infrastructure, with the confidentiality and compliance implications that follow. A locally run model keeps the data on your network and makes its version, configuration and cost known. The price is quality: a small general-purpose open-weight model is less capable than the larger hosted ones, and it is worth checking that its license is compatible with the intended use.

Architecture#

journalctl -> filter and cleanup -> masking -> local model -> summary -> human review

The model is served by a runtime such as Ollama, which by default exposes an HTTP API on the local interface (port 11434). Runtime and model are installed by following the project's documentation; with Ollama the model is downloaded with ollama pull <NOME_MODELLO>. Other runtimes, such as the llama.cpp server, offer similar interfaces. The API has no authentication: it must stay reachable only from the machine that uses it, or from a well-delimited management network, as described in network segmentation with VLANs. It must not be exposed to the Internet.

Extraction and masking#

Extraction is limited to the lines that matter: priority warning or higher, last hour.

journalctl -p warning --since "1 hour ago" -o short-iso --no-pager

Before reaching the model, the text goes through a script that replaces sensitive data with placeholders. The same value always gets the same placeholder (<IP_1>, <UTENTE_2>), so the model can still notice that an address appears many times without knowing what it is.

import re
import sys
 
SEGRETI = re.compile(r"(?i)\b(pass(?:word|wd)?|token|secret|api[_-]?key)=\S+")
PATTERNS = {
    "MAC": re.compile(r"\b(?:[0-9A-Fa-f]{2}[:-]){5}[0-9A-Fa-f]{2}\b"),
    "IP": re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b"),
    "EMAIL": re.compile(r"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"),
    "UTENTE": re.compile(
        r"(?<=user )[\w.-]+|(?<=user=)[\w.-]+|(?<=for )[\w.-]+(?= from)", re.IGNORECASE
    ),
    "TOKEN": re.compile(r"\b[A-Za-z0-9_-]{32,}\b"),
}
 
 
def maschera(testo: str) -> str:
    mappa = {}
 
    def sostituzione(tipo):
        def _sub(m):
            chiave = (tipo, m.group(0))
            if chiave not in mappa:
                numero = sum(1 for t, _ in mappa if t == tipo) + 1
                mappa[chiave] = f"<{tipo}_{numero}>"
            return mappa[chiave]
 
        return _sub
 
    testo = SEGRETI.sub(r"\1=<SEGRETO>", testo)
    for tipo, regex in PATTERNS.items():
        testo = regex.sub(sostituzione(tipo), testo)
    return testo
 
 
if __name__ == "__main__":
    sys.stdout.write(maschera(sys.stdin.read()))

Calling the local model#

The Ollama API accepts a JSON request; with stream: false the response arrives all at once in the response field. The example uses only the standard library and reads the model name from an environment variable, without pinning its version.

import json
import os
import sys
import urllib.request
 
URL = "http://127.0.0.1:11434/api/generate"
MODELLO = os.environ["MODELLO_LOG"]
 
 
def main() -> None:
    righe = sys.stdin.read().splitlines()
    log = "\n".join(f"{n}: {riga}" for n, riga in enumerate(righe, start=1))
    with open("prompt.txt", encoding="utf-8") as f:
        prompt = f.read().replace("{LOG}", log)
    corpo = json.dumps(
        {"model": MODELLO, "prompt": prompt, "stream": False, "options": {"temperature": 0}}
    ).encode("utf-8")
    richiesta = urllib.request.Request(URL, data=corpo, headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(richiesta, timeout=600) as risposta:
        print(json.load(risposta)["response"])
 
 
if __name__ == "__main__":
    main()

export MODELLO_LOG="<NOME_MODELLO>"
journalctl -p warning --since "1 hour ago" -o short-iso --no-pager \
  | tail -n 300 \
  | python3 maschera.py \
  | python3 riassumi.py > riepilogo.md

temperature: 0 reduces variability between runs, without guaranteeing identical answers. The 300-line limit should be tuned to the model's context.

The prompt#

An effective prompt is explicit about role, constraints and output format. The numbered lines are there so you can ask the model to cite its evidence.

You are an assistant for Linux system administration. You receive numbered
log lines that have already been masked: placeholders such as <IP_1> or
<UTENTE_1> are intentional.
 
Tasks:
1. Group the lines by probable cause.
2. For each group give a severity (low, medium, high) and a one-line summary.
3. Flag suspicious patterns, such as repeated attempts or services restarting in a loop.
 
Rules:
- Use only the information present in the log. If it is not enough, write "cannot be determined from the log".
- Do not invent file names, commands, addresses or versions.
- Do not suggest destructive commands. Describe checks as suggestions.
- For each group, cite the numbers of the supporting lines.
 
Format: Markdown list, one group per item, with the fields
Severity, Summary, Lines, Suggested check.
 
Log:
{LOG}

Honest limits#

• Hallucinations. The model can attribute a plausible cause that is not in the logs. The prompt reduces the risk, it does not eliminate it.
• Limited context. An excerpt that is too long gets ignored or truncated: the analysis is only valid for what the model sees.
• No replacement for alerts. The summary sits alongside deterministic rules, such as those described in monitoring with Prometheus and Grafana, and does not replace them.

Evaluating the results#

The comparison is done on your own logs, not on generic numbers. Collect a small set of excerpts, already masked, and for each one have a person write down the expected groups. Then compare the model's answers: correct groups, invented groups, important events omitted, summaries not supported by the cited lines. The same set serves to compare different models and prompts and to repeat the test after every change. The final criterion is practical: the summary must save time without introducing errors that force you to reread the whole log.

Ballpark hardware#

A small model, in a quantized version, can run on CPU alone, with longer response times. A GPU with enough memory speeds things up noticeably, but inside a VM it requires passthrough and therefore more configuration work. The memory needed grows with the size of the model and with the context requested. An hourly analysis, run from a timer, does not need low latency: it is often best to start with the smallest model that gives useful results and grow only if needed.