cut-cross-entropy
Subscription: Developer
Active
Subscription: Developer
A memory-efficient implementation of the linear cross-entropy loss function from Apple Research, reducing GPU memory usage during language model training by avoiding materialization of the full logit matrix while maintaining numerical accuracy.