Abstract: Knowledge distillation enhances student performance by transferring knowledge from teachers. However, existing approaches often suffer from a lack of knowledge diversity. To address this ...