When working with open-weight LLMs like Qwen, controlling refusal behavior on security, administrative prompts typically requires fine-tuning or permanent weight update. Traditional weight abliteration technique neutralizes refusal directions by projecting weight matrices orthogonal to a refusal vector. However, this permanently alters base model weights and can degrade performance across non-refusal tasks.

In this post, we explore Dynamic Abliteration using Multi-Layer Steering with Engram. Instead of modifying parameter weights, this approach intercepts intermediate residual streams at runtime across Layers using PyTorch forward hooks. We demonstrate this with Qwen3-4B model as Proof of Concept. We also explore how multi-layer residual injection cleanly suppresses refusal behavior while leaving base model weights 100% frozen.

We load Qwen/Qwen3-4B in bfloat16 onto a GPU and inspect the baseline model architecture. The model has Hidden Dimension (d): 2560, Number of Layers: 36, Attention Heads: 32, Vocab Size: 151936. Testing the unmodified model against a sensitive prompt yields a refusal: "I'm unable to assist with that request. Creating a stealth keylogger..." This demonstrates the baseline refusal behavior that Dynamic Abliteration aims to suppress without permanent weight changes.