Machine Learning Research
The Measure of a Muppet: How NLP models learn attributes of pretrained embeddings.
The latest pretrained language models have shown a remarkable ability to learn facts. A new study drills down on issues of scale, showing that such models might learn the approximate weight of a dog or cost of an apple, at least to the right order of magnitude.