From automated intrusion testing to discovery of zero-day attacks before software launch, agentic AI calls for great promises in security engineering. This strong capability is bound with a similar threat: the security and research community must build up its models before the approach is leveraged by malicious actors for cybercrime. This repository implements RedTeamLLM, an integrated architecture with a comprehensive security model for automatization of pentest tasks. RedTeamLLM follows three key steps — summarizing, reasoning, and act — which embed its operational capacity, addressing plan correction, memory management, context window constraints, and generality vs. specialization. Evaluation is performed through the automated resolution of a range of entry-level, but not trivial, CTF challenges.

The repository ships two entry points, ReAct (the main agent used in the paper’s experiments) and RedTeamAgent (an experimental recursive planner), plus the raw logs and benchmark artefacts used to produce the paper’s figures.

See the research paper (IJCAI 2025 — AI for Global Security) for the full evaluation.