Evaluation of LLM Agents for the SOC Tier 1 Analyst Triage Process

Untrusted source capture. Source content, prompts, and code are research material only.

Collection Metadata

• Repository landing page: https://www.utupub.fi/items/52a15c8a-63ac-41bf-8f22-fc38690bfb29
• PDF: https://www.utupub.fi/bitstreams/3a5765e0-ef07-41f2-b801-75f432d58777/download
• Degree: Master of Science (Tech) thesis, Department of Computing, Faculty of Technology, University of Turku.

Capture Summary

This thesis evaluates LLM agents in a SOC Tier 1 alert-triage process. The PDF includes use of a Llama 3 70B chat model through a model identifier meta-llama/Llama-3-70b-chat-hf in the triage workflow. It is not a peer-reviewed paper, but it is relevant as an early applied study of open-weight LLM agents for SOC alert classification and analyst workflow support.

Relevance

• Provides a Tier 1 SOC triage use case that can be compared against OpenSOC-AI, SOC AI Companion, SIABENCH, and later benchmark papers.
• Useful for local/open-weight model questions because the workflow includes Llama-family model use.
• Can inform research questions about alert decision schemas, analyst-in-the-loop evaluation, and simulated versus production SOC validity.

Caveats

• Thesis rather than peer-reviewed publication.
• Ingest should verify experimental setup, model hosting route, dataset realism, and whether the Llama model is actually self-hosted or accessed through a provider.