kaiokendev/SuperCOT-dataset
General NLPEnglishmit
Kaiokendev/SuperCOT-dataset is a General NLP dataset in English from kaiokendev in Parquet format. It is distributed under the mit license and falls in the 10K<n<100K size category, and has been downloaded 22 times.
About kaiokendev/SuperCOT-dataset
epochs: 3
learning rate: 3e-4
lora rank: 8
lora alpha: 16
lora dropout: 0.05 for cutoff 1024 13B, otherwise no dropout due to gradient checkpointing
masking: none
mbatch size: 4 (1 for 30B)
batch size: 8 (2 for 30B)
val set size: 0.2
sdp implement...
Details
- Task
- General NLP
- Language
- English
- Format
- Parquet
- Rows / instances
- N/A
- Size
- 10K<n<100K
- Creator
- kaiokendev
- Year
- 2023
- License
- mit
- Downloads
- 22
- Likes
- 46