Multitask Learning in Setting Teacher Student

Multi-task learning (MTL) is a learning paradigm in which neural networsk are trained using multiple related target labels, with the hope that the model's performance on generalization will improve: by leveraging information contained in related tasks, MTL aims to produce better models compared to the single task learning (STL) baseline. The core idea behind MTL is that simultaneous optimization on multiple tasks forces the model to learn a more general latent representation which encodes more information about data as to produce more robust models and improve overall performance. The aim of this master thesis is to study multi-task learning in a simple and controllable teacher-student setting and to provide insight about hyperparameters selection, which may guide future research and applications in more complex scenarios. We start by showing that MTL improves generalization in our artificial setting for related tasks only, and then explore different settings of noise, number of samples and number of auxiliary tasks.