Machine Learning Research
Does Your Model Generalize or Memorize: Researchers find models with more parameters copy more bits from training sets
Benchmarks can measure how well large language models apply what they’ve learned from their training data to new data, but it’s harder to measure the degree to which they simply memorized their training data. New work proposes a way to gauge memorization.