classification recurrent Attention-UNet
classification based ONNX implementation for pretrained tensor.
- Input
- 4185-dim embedding
- Encoder
- 102 x Attention-UNet with 46 heads
- Output
- perplexity projection
Training config
optimizer=LARS, lr=0.889, scheduler=polynomial, warmup=1600