Machine Learning Research
More Reliable Pretraining: Pretraining Method Helps AI Learn Useful Representations
Pretraining methods generate basic representations for later fine-tuning, but they’re prone to certain issues that can throw them off-kilter. New work proposes a solution.