Ramakrishna Paramahamsa Kompella

Researchers pinpoint cross-lingual refusal circuit in multilingual MoE model

Researchers have identified a specific circuit within a multilingual Mixture-of-Experts (MoE) model, named sarvam, that is responsible for refusing harmful requests. This circuit's ability to refuse is language-invarian…