Data Mixing, Minus the Guesswork: Research from the team behind CausalMix offers a technique to select fine-tuning examples
When you’re building a fine-tuning dataset, finding a mix of subject domains that yields a high-performing model can require expensive trial and error, because fine-tuned performance also depends on the quality of the individual examples.