Pretraining models on the ImageNet dataset has been a mainstream research approach for years, but in today’s digital world where data is growing by orders of magnitude the 10 year-old ImageNet dataset is now considered relatively small in size. That motivated Facebook to explore how pretraining a machine learning model on a large-scale, weakly-supervised dataset could impact model performance.
Source: Facebook Model Pretrained on Billions of Instagram Hashtags Achieves SOTA Results on Top-1 ImageNet
Model available in PyTorch, link at the bottom.