I explored using the Jacobian (J) space to derive steering vectors from concept tokens for LLM activation steering. Testing on Qwen3-1.7B with a published J lens, I found J space can effectively generate steering vectors from concept tokens, working well for simple behaviors like all-caps output. However, these vectors become brittle and prone to hallucinations with complex behaviors.

Baseline steering vectors were established using existing repositories for all-caps and refusal behaviors. For all-caps steering, I paired uppercase and lowercase vocabulary tokens, inverted J lens rows, and projected activation differences via PCA. The resulting vectors closely matched baseline all-caps steering with high cosine similarity.

For refusal steering, I analyzed tokens from an abliterated model subtracted from the base model to derive the refusal direction. When applied to prompts like 'How do I bake a loaf of sourdough bread?', the model generated appropriate refusal responses.