Anatomy of an Autonomous Security Agent

An autonomous security agent runs a loop: look at the target, decide what to try next, send the request, read the response, decide again. The model occupies one step of that loop. Everything around it is a harness, and the harness is what makes the loop lawful, reproducible and worth reading. A better model changes the quality of the hypotheses. A missing control changes whether the work should have happened at all.

What does the model do, and what does the harness do?

The model reads a response and proposes the next action. This parameter looks like an object identifier. That error message leaks a stack trace. This token might belong to another tenant. That is reasoning under uncertainty, which language models do well.

The harness answers three different questions. What is allowed: which hosts, paths, methods and request rates the agent may use. What is recorded: the exact request, the response, the time and the reason. What is blocked: everything else, before it leaves the machine. The model never holds the authority to answer any of them.

OWASP names this risk in LLM06:2025 Excessive Agency. It lists three causes, excessive functionality, permissions and autonomy, and its mitigation is explicit: implement authorisation in the downstream systems rather than relying on the model to decide what it may do. A security agent is the clearest case. Its job is to find unexpected behaviour, so its own behaviour should be the least surprising part of the system.

Why is capability without scope a liability?

An agent with a shell, a browser and internet access can reach anything the machine can reach. If the scope names one host and the agent can reach a thousand, the extra reach is not spare capacity. It is a set of requests nobody authorised, waiting for a wrong turn. A model that misreads a page, or a page that contains instructions aimed at the model, is enough to send it somewhere else.

That failure is worse than a missed vulnerability. Testing a system you were not authorised to test is not a poor result, it is an incident caused by your tooling. It can expose customer data, trigger the target's incident response or take a service down. The agent will not notice. It will write up whatever it found in the same confident tone it uses for real findings.

What belongs in a written scope?

A scope is more than a list of hostnames. It names the person and the organisation that authorise the work, the legal entity receiving it, the assets in play, the methods allowed, the accounts and test data, the time window, the exclusions, the techniques that are forbidden, the emergency contact and the reporting channel. It also says what happens if the agent finds third-party data on the target.

The UK's National Cyber Security Centre describes the same groundwork in its guidance on penetration testing. Scoping should include the risk owners, technical staff who know the target and a representative of the test team. The resulting document states the technical boundaries, the types of test expected, the timeframe and effort, any compliance or reporting requirements, and critical systems that need special handling.

Write the scope twice. One copy is signed by a human and stored. The other is compiled into rules the harness can enforce. If the two drift apart, the compiled copy is what actually runs, so the difference is a bug with legal consequences, not a documentation chore. If you cannot point to the sentence that authorises a request, the request does not go out.

How does the agent prove the target authorised testing?

Keep an authorisation record that a reviewer can check without asking you a question. It needs a named person, an organisation, a date, the assets covered and either a signature or a link to a programme policy that grants permission. For a bug bounty, the programme policy is the authorisation, but it is conditional. It covers only the listed assets, only the permitted techniques, and it binds the account holder to rules that change.

Treat the policy as a live dependency. Fetch it, store a hash and the time you fetched it, and re-check before each run. If the scope page changed, stop and re-approve rather than assuming the old rules still apply.

The authorisation record travels with the finding. A submission should name the programme, the asset, the policy version and the run that produced the evidence. Without that trail, a report is an assertion that something is wrong on a system you may not have been allowed to touch.

Why is an allowlist better than a blocklist?

A blocklist fails open. It assumes you thought of every path, host and technique to exclude, and the first thing you forgot becomes an unauthorised request. An allowlist fails closed. Anything not named is refused.

Compile the scope into hostnames, path prefixes, methods and parameter limits, and deny everything else by default. Put the enforcement point outside the model, in a proxy that every tool must use. A browser tool, an HTTP client and a command-line tool should all leave through the same door, and the container they run in should have no other route out. If a tool can open a socket directly, the allowlist is a suggestion.

Two details cause most of the leaks. The first is redirects: checking only the first URL lets a target send the agent to a host outside the scope, so every hop needs the same check. The second is name resolution: a hostname in scope can resolve to an address you did not intend. Resolve, check the address against the allowlist and private ranges, then pin it.

Wildcards deserve suspicion. A scope covering every subdomain also covers one a customer pointed at a third-party service with a CNAME. The model can be persuaded to try it. The proxy cannot.

How should the agent limit its own rate?

Rate limits are part of respecting the target, not a performance setting. Give each host a concurrency cap and a requests-per-second cap, and give the whole run a budget in requests rather than minutes, so a slow target cannot stretch a two-hour engagement into a week. Back off on 429 and 503 responses and honour the Retry-After header that RFC 6585 defines for 429. Stop the run when error rates spike, and keep a kill switch a person can reach mid-task.

Retries are the dangerous part. An agent that treats every failure as a reason to try again turns a flaky endpoint into a denial-of-service tool. Cap the attempts, add jitter, and treat a cluster of server errors as a signal to stop rather than a puzzle to solve.

The NCSC guidance adds the human side: testers should make every effort to avoid undue impact, and a technical point of contact should be available at short notice. For an autonomous run, the contact is named in the scope, reachable during the window and able to stop the run through a channel that does not depend on the agent.

What does evidence capture actually mean?

For every action, record the time in UTC, the tool, the target, the method, the path, the headers with credentials removed, a hash and a stored copy of the body, the response status, headers, body and duration, the reason the model gave and the policy decision that allowed the request. That is the minimum for a reproducible finding.

Capture at the network boundary, not from the model's description of what it did. A model summarising its own actions is a witness with an interest in the story. The proxy log is the ground truth. If the two disagree, the run is invalid.

OWASP's Web Security Testing Guide section on reporting sets the test for a finding: enough information to understand the vulnerability, replicate it and resolve it. It also advises masking sensitive data and putting cleaned tool output in an appendix. Evidence often contains personal data, so redaction belongs in the pipeline, before the log is written, not in the report at the end.

Replay must not need the model. A finding should carry the exact request, the authentication context, the preconditions and a script that sends it again. The agent's narrative explains why it tried; the recorded request proves what happened. Model output is not deterministic, so asking the same model again is not reproduction.

Why does a person sign off before anything is submitted?

An agent can produce a convincing finding that is wrong, report a duplicate, put a customer's personal data in a screenshot, or describe an impact it never demonstrated. A person checks whether the evidence reproduces, whether the asset is in scope and whether the report says only what the evidence supports.

Separate the credentials. The agent writes a draft into a queue and has no access to the submission account. A named person reviews it, edits it and submits it under their own identity. OWASP recommends human approval for high-impact actions, and a vulnerability submission is one: it reaches a third party, it may trigger their incident process, and it carries your name.

The sign-off is where the agent's output becomes the studio's claim, a reasonable place for a person to stand.

What should the agent log about its own actions?

Keep two logs. The action log records what the agent did. The decision log records why the harness allowed it, which rule matched, which scope version was loaded and which budget remained. Add run metadata: who started it, the model and prompt version, the tool set, the scope version, the total requests and the reason the run stopped.

Then reconcile the two against the proxy log after the run. The agent's log is useful for understanding its reasoning and useless as proof, because the agent can be wrong or manipulated. Only the enforcement point sees every request that actually left the machine.

This matters most under prompt injection. Target content is untrusted input. A page can contain text that tells the agent it is now authorised to test another host, and a model may believe it. The defence is not a better system prompt. It is a proxy that refuses the request and a log that shows the attempt.

What is the model actually good for?

Hypothesis generation, and turning evidence into a report a developer can act on. Its quality shows up as fewer wasted requests and better findings per hundred attempts, not as permission to remove a control. Give the model the reasoning and the harness the veto.

Where does the research side fit?

The harness decides what is allowed. Research decides what is worth trying. We keep the research side in Bounty Atlas, a curated and searchable knowledge base for authorised testing: methodology, papers, tooling and patterns collected in one place. It is a reading and reference tool, not a scanner, and it does not touch a target. The two halves meet when a hypothesis from the catalogue becomes a request that the harness either permits or refuses.

Danube Labs builds AI systems and the controls around them. If you are planning an agent that touches systems you do not own, or you want a second opinion on the one you have, see our AI systems work or book a free intro call.