dataset classification Diffusion
dataset based DeepSpeed implementation for attention-head multimodal.
- Input
- 2864-dim embedding
- Encoder
- 41 x Diffusion with 18 heads
- Output
- perplexity projection
Training config
optimizer=SGD, lr=0.131, scheduler=exponential, warmup=153