PEFT of SLM for Telecommunications Customer Support: A Comparative Study of LoRA Configurations with Energy Consumption Analysis
Researchers explored parameter-efficient fine-tuning (PEFT) using LoRA configurations on the Qwen2.5-3B model for telecommunications customer support. They developed a synthetic data generation method and evaluated 16 LoRA configurations, including energy consumption and LLM-as-a-judge assessments. The study found that traditional validation loss metrics did not correlate with qualitative performance, highlighting the need for more comprehensive evaluation methods. AI
IMPACT Highlights the limitations of standard validation loss for evaluating fine-tuned models, suggesting a need for better qualitative assessment methods in domain-specific AI.