Eurobase
Log InSign Up

Eurobase research · October 2026

Agumon v1.0

A small language model as a web application firewall

Giovanni Liber

I was looking at the logs of my service, and I asked myself:

Would it be possible to have an AI model answer allow or block on these malicious requests I see passing through? After all, I already have the manifest, I already know what is authorised on this application. How complicated could it be?

How it went

  1. No fine-tuning

    From there I decided to test Clef, without fine-tuning, to see whether it could understand requests like these.

  2. MacBook

    When I saw that it answered fairly well, I decided to build a micro dataset and run a fine-tune on my MacBook.

  3. MiniCPM5-2B

    From there I immediately understood that the numbers were interesting, and that it was worth pushing on. I found MiniCPM5-2B, a recent Chinese model, on which I ran all the training sessions, each one larger than the last.

  4. Mac Studio

    I collected traffic synthetically from my browser, ran training sessions on a Mac Studio, then compared it against the various benchmarks. From the start it seemed clear that the model was able to recognise malicious requests.

  5. H100 on RunPod

    I ran the final training on one H100 SXM on RunPod.

Why it's worth exploring

I think this is a kind of model worth exploring, because in tomorrow's world, having models smarter than the tools we have today, able to handle the complexity of security, is essential.

Every WAF can be evaded. I have 10 years of experience, I know it well.

Every security tool can be bypassed, and the same surely holds for this fine-tune I built. Still, I see it as a starting point: proof that it can be done, that it works, and that it is worth thinking about building something with it.

What comes next

  • A new dataset

    I will build a new, much larger dataset.

  • A new security benchmark

    I will build a new security benchmark that will investigate all of the most complex requests available.

We moved from string matching and signatures, to complex policies, to ML algorithms that act on traffic. It is entirely natural to think that the next step will be exactly this: small system-one decision models, tuned on companies' real traffic, working together with much larger AI models.

Technical details

of attacks blocked
98.1%
of legitimate requests blocked
0.004%
per decision on one H100
6.2 ms
Held-out test: 510 real sites never seen in training. OWASP CRS at paranoia level 1, open-appsec with its bundled basic model, both at their defaults.
MetricAgumon v1.0OWASP CRS 4.20open-appsec 1.1.36
Attacks blocked98.1%87.4%91.9%
Legitimate requests blocked0.004% (4 of 96,447)11.9%15.6%
  • Model. LoRA adapter on MiniCPM5-2B-Base: rank 8, last 16 of 42 layers, 4.78M parameters. The adapter and the base model are Apache-2.0. It reads one raw HTTP request and decides block or allow in a single forward pass.
  • Data. 408,806 requests, one epoch. Legitimate traffic recorded by our crawler on public sites; attack payloads from SecLists, PayloadsAllTheThings, OWASP CRS regression tests and nuclei-templates, grafted into recorded requests, each with its unchanged clean twin.
  • Test. The decision threshold was frozen on a separate calibration split before the test set was opened.
  • Latency. One H100 SXM, one request at a time: 10.9 ms median with vLLM at its defaults, 6.2 ms with a minimal engine written for this one task.
  • Limits. It reads the first 2,000 characters of the URL, 1,000 of each header and 2,500 of the body. It sees nothing about rates, sessions or responses. Its weakest attack families are encoded and obfuscated payloads, CRLF injection and LDAP injection.
  • What failed. Adding 25,807 legitimate requests without attack twins cut attacks blocked from 93.2% to 75.5% on the calibration split. Every new legitimate request shape now comes with its attack twin.