« Back to Glossary Index

The Mixture of Experts is an ensemble learning technique developed within the field of deep learning. It introduces the idea of training experts (which are other neural networks) on specific subtasks, decomposing a more complex predictive modelling problem.

Typically, in an ensemble scenario, all models are trained on the same dataset, and their results are combined through majority voting or averaging. In contrast, with the Mixture of Experts approach, each “expert model” within the ensemble is only trained on a subset of data where it can achieve optimal performance, thus limiting the model’s focus. To decide which expert must process a specific subset of data, an addition model is used, called “gate network” or “router”.