Evaluating Prompt Injection Defenses in Large Language Models: A Multi-Model Empirical Study of Security–Usability Trade-offs

Abstract

The increasing integration of Large Language Models (LLMs) into organisational and consumer-facing systems has introduced novel security vulnerabilities, among which prompt injection attacks represent a particularly critical threat. These attacks exploit the natural language interface of LLMs to manipulate model behaviour without requiring access to underlying architectures or parameters. Despite a growing body of research, there remains a lack of comprehensive empirical evaluations examining the effectiveness of mitigation strategies across multiple models. This study presents a systematic experimental evaluation of prompt injection attacks and layered defensive controls across four open-source LLMs: Gemma 3, Llama 3, Mistral, and Phi-3 Mini. The analysis employs a set of quantitative metrics, including detection rate, false positive rate, attack success rate reduction, accuracy degradation, and performance overhead. The results indicate that while the implementation of layered controls significantly reduces the success rate of prompt injection attacks, these improvements are accompanied by measurable reductions in model usability and output fidelity. Furthermore, the findings reveal that certain defensive mechanisms may inadvertently alter prompt semantics in ways that diminish the effectiveness of intrinsic safety features, thereby enabling previously unsuccessful attacks. These outcomes suggest that prompt injection constitutes a structural vulnerability inherent to current LLM architectures. The paper concludes by advocating for adaptive and context-aware defence strategies and outlines key directions for future research.