LLM Safety Amplification Achieved By Rank 1 Update! ROSI Mechanism And Experimental Results
3 main points
✔️ ROSI is a lightweight rank 1 update method that amplifies LLM safety
✔️ It can increase the harmful instruction rejection rate while maintaining normal task performance
✔️ It can be reapplied to uncensored models and is effective for last mile safety
Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection
written by Harethah Abu Shairah, Hasan Abed Al Kader Hammoud, George Turkiyyah, Bernard Ghanem
(Submitted on 28 Aug 2025)
Comments: Under Review
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
The images used in this article are from the paper, the introductory slides, or were created based on them.
Summary
This paper proposes a new method, Rank-One Safety Injection (ROSI), to improve the safety of LLMs.
While LLMs have been used in a wide range of applications in recent years, "safety alignment," which prevents the generation of dangerous content, has become a challenge.
However, it has been reported that this mechanism can be easily breached by jailbreak attacks (jailbreak).
Previous research has shown that safety can be disabled by erasing a one-dimensional representation space called the "denial direction.
This study adopts the opposite idea and develops a lightweight and interpretable method to amplify safety by enhancing this "refusal direction".
ROSI works by simply adding a rank 1 update to the model's weight matrix and does not require retraining or extensive tuning.
Experiments confirm that ROSI improves the rejection rate for adverse requests while hardly compromising performance in normal tasks, and show that it can be reapplied to models where safety is intentionally removed.
Proposed Methodology
ROSI is a simple mechanism that leverages the linear representation inside the LLM to extract safety-related directions and incorporate them into the weights of the model.
First, the activation of the model in response to harmless and harmful instructions is compared, and a "safety direction vector" is derived from the difference.
This is defined as the central difference between the harmless and harmful response clusters and represents a one-dimensional feature for the model to reject.
This direction vector is then used as the basis for applying a rank 1 correction to the output matrix that is written to the residual stream.
Specifically, the update is designed to add a safe direction to the matrix, so that the model's output is always slightly tilted in the direction of rejection.
This update is very lightweight and works efficiently without requiring retraining, even when applied en masse to all layers.
Unlike traditional inference-time manipulation (activation steering), ROSI makes permanent, interpretable modifications that fundamentally stabilize model behavior.
Experiments
The authors tested the effectiveness of ROSI in several experiments.
First, they applied it to a group of safety-aligned models (LLaMA, Qwen, Gemma, Yi, etc.) and observed a significant increase in rejection rates to adverse instructions.
In particular, we found an improvement of +13 to +18 percentage points for the originally weak models.
Furthermore, they also showed a significant improvement in resistance to jailbreak attacks (DAN, Harmbench, WildGuardTest, etc.), reducing the success rate of attacks by less than half.
On the other hand, benchmark scores such as MMLU and HellaSwag remained virtually unchanged, indicating that the usefulness of normal tasks is maintained.
Next, ROSI was also applied to the "uncensored model" (Dolphin system), in which safety was intentionally removed, and by reinjecting the safety direction, the rejection rate was improved by more than 30%, restoring up to 100% safety.
Furthermore, there was almost no performance degradation, demonstrating the effectiveness of ROSI as a post-processing "last-mile safety method.
Categories related to this article
