FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

LM workload tf32 by priyakasimbeg · Pull Request #902 · mlcommons/algorithmic-efficiency · GitHub

LM workload tf32 - #902

Merged
priyakasimbeg merged 119 commits into
devfrom
lm_workload_tf32
Jan 29, 2026
Merged

LM workload tf32#902
priyakasimbeg merged 119 commits into
devfrom
lm_workload_tf32

Conversation

priyakasimbeg commented Jan 29, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Add LM model AlgoPerf.

This model is based on the Nanodo-style models, with the following main modifications:

  1. Use RoPE positional encoding as opposed to the additive positional embedding
  2. Use RMSNorm instead of Layernorm
  3. Use QK Norm
  4. Use GLU for feed-forward modules

Model dimensions:

Parameter Description  
D embed dimension 1024
H number of attention heads 8
N number of transformer block layers 12
V vocab size 50_257
F FF inner dimension 4096
L sequence length 1024
Total params   177,307,136

priyakasimbeg requested a review from a team as a code owner January 29, 2026 04:58

github-actions Bot commented Jan 29, 2026
edited
Loading

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

priyakasimbeg changed the title Lm workload tf32 LM workload tf32 Jan 29, 2026
priyakasimbeg merged commit c8e0a32 into dev Jan 29, 2026
29 checks passed
github-actions Bot locked and limited conversation to collaborators Jan 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL