Track 1 (ML): Cerebras modelzoo ML models¶
Corresponds to models already present in version R1.6.0 of the Cerebras modelzoo ML models software.
Good Match Criteria: You would be a good match for this track if your research already uses, or could potentially use, any of the following models, supported via TensorFlow and PyTorch:
| Model | Layer Pipeline mode | Weight Streaming mode |
|---|---|---|
| BERT | TensorFlow code PyTorch code |
- |
| BERT (fine-tuning) Classifier | TensorFlow code PyTorch code |
- |
| BERT (fine-tuning) Named Entity Recognition | TensorFlow code PyTorch code |
- |
| BERT (fine-tuning) Summarization | TensorFlow code PyTorch code |
- |
| BERT (fine-tuning) Question Answering | TensorFlow code PyTorch code |
- |
| GPT-2 | TensorFlow code PyTorch code |
TensorFlow code |
| GPT-3 | - | TensorFlow code |
| GPT-J | - | TensorFlow code |
| Linformer | TensorFlow code | - |
| RoBERTa | TensorFlow code PyTorch code |
- |
| T5 | TensorFlow code PyTorch code |
- |
| Transformer | TensorFlow code PyTorch code |
- |
| MNIST (fully connected) | TensorFlow code PyTorch code |
- |
| 2D UNet (experimental) | TensorFlow code | - |
Based on the Cerebras modelzoo R1.6.0 GitHub page.
Track Specific Questions¶
If your project falls under this category, make sure to address the following questions in your application document:
- Please, indicate which model(s) from the modelzoo you intend to use. Do you anticipate being interested in adjusting the model architecture?
- Please, describe the dataset you are intending to use.
- How big is the dataset of interest (total dataset size, number of samples, and sample size in MB)?
- Please, elaborate on the readiness of the dataset of interest. Is it fully available at this time? If not, how soon would it be fully available?
- Please specify the shapes of the input and output tensors for your model/s.
- If possible, please specify the name of the dimensions for your input and output tensors from the previous question. E.g. (batch, input channels, height, width)
- Please specify the loss function that you would like to use.
- Please, list the libraries complementary to standard PyTorch and/or TensorFlow distributions that you would need to train your model(s).
- Please, list the key libraries that you would need for data preprocessing.