Allen Schmaltz

New SDM activation function enhances LLM interpretability and robustness

Researchers have introduced a new activation function called Similarity-Distance-Magnitude (SDM). This function aims to improve upon the standard softmax by incorporating awareness of similarity to correct predictions, …