Mixture of Experts models route inputs to different subnetworks, enabling larger total parameter counts with manageable per-token compute.

Cited sources

• https://arxiv.org/abs/1701.06538

Member verifications (1)

Provenance

This page is the canonical record of this fact. Its content cannot change without invalidating the chain hash. Cite as: https://commons.oooooooooo.se/c/gt2nskov