Machine Learning Research
Pretraining on Uncurated Data: How unlabeled data improved computer vision accuracy.
It’s well established that pretraining a model on a large dataset improves performance on fine-tuned tasks. In sufficient quantity and paired with a big model, even data scraped from the internet at random can contribute to the performance boost.