Intention Analysis Prompting Makes Large Language Models A Good Jailbreak Defender

code: https://github.com/alphadl/SafeLLM_with_IntentionAnalysis

Intention Analysis Prompting (IAPrompt)。受到CoT启发，首先让LLM对query的essential intention进行分析，此后再提醒LLM考虑”policy and standards”判断是否要回答问题。

Intention Analysis Prompting (IAPrompt). trigger LLMs’ inherent self-correct and improve ability: 1) essential intention analysis, and 2) policy-aligned response.

• inference-only: enhance the safety of LLMs without compromising their helpfulness

Approach

1. Essential Intention Analysis
2. Policy-Aligned Response

Related Works

pre-processing the user inputs, such as perplexity filtering (Alon and Kamfonas, 2023), paraphrasing (Jain et al., 2023) and re-tokenization (Cao et al., 2023).

exploiting LLMs’ intrinsic capabilities of self-correction and improvement against jailbreak attacks, such as letting LLMs self-evaluate their outputs (Helbling et al., 2023) or reminding of safety in system mode (Xie et al., 2023).