Can MTP models be used as standalone smaller models? (e.g. DS4 Flash/Pro)
A user on Reddit's r/LocalLLaMA forum is inquiring about the potential of using intermediate prediction heads from Multi-Token Prediction (MTP) trained models as standalone, smaller models. The discussion specifically references DeepSeek's DS4 Flash and DS4 Pro models as examples, questioning if these internal components could be extracted and utilized independently. AI