Cuts the activation memory of CTC loss from growing linearly with vocabulary size while keeping the loss and first-order gradients equivalent, directly benefiting large-vocabulary speech recognition training; the claim is small and the description accurate. The limit is that conclusions come from a single setting with no empirical comparison against mainstream CTC implementations.
Memory-efficient CTC loss with exact loss and first-order gradient equivalence under vocabulary reduction, and activation memory that no longer scales linearly with vocabulary size.
★15