Machine Learning Research
More, Better Open Source Options: Alibaba releases Qwen 2.5 models, raising the bar for open weight LLMs
The parade of ever more capable LLMs continues with Qwen 2.5.
Machine Learning Research
Machine Learning Research
Reducing Memorization in LLMs: A technique that masks tokens in large language models, protecting data privacy
Studies have established that large language models can memorize the text passages they’ve been trained on repeatedly and regurgitate them when prompted in adversarial and, though rarely, in benign ways.
Machine Learning Research
OpenAI Forges Chains of Thought: OpenAI’s o1 models excel in reasoning, outperform GPT-4o in math and coding
Preliminary versions of OpenAI’s new model family were trained explicitly to think step-by-step, yielding outstanding marks in math, science, and coding — but users can’t see their reasoning steps.
Machine Learning Research
2D-to-3D Goes Mainstream: AI systems from Stability AI and Shutterstock transform 2D images into 3D meshes in seconds
Traditionally, building 3D meshes for gaming, animation, product design, architecture, and the like has been labor-intensive. Now the ability to generate 3D meshes from a single image is widely available.
Machine Learning Research
Balancing Web Data Distributions: Automated method organizes large datasets for better model performance
Datasets that were scraped from the web tend to be unbalanced, meaning examples of some classes (say, cats) are plentiful while examples of others (say, caterpillars) are scarce.
Machine Learning Research
Making LLMs Explainable: Google’s Gemma Scope probes how large language models think
Researchers have probed the inner workings of individual layers of large language models. A new tool applies this approach to all layers.
Machine Learning Research
Models Ranked for Hallucinations: Measuring language model hallucinations during information retrieval
How often do large language models make up information when they generate text based on a retrieved document? A study evaluated the tendency of popular models to hallucinate while performing retrieval-augmented generation (RAG).
Machine Learning Research
Long Context Gets Up to Speed: AI21 Labs’ Jamba 1.5 outpaces transformers in long-text processing
A new model generates tokens faster than current transformers, especially when processing long inputs.
Machine Learning Research
Multimodal to the Max: 4M-21 multimodal model excels in handling diverse input and output types
Researchers introduced a model that handles an unprecedented number of input and output types, including many related to performing computer vision tasks.
Generative AI
Agentic Coding Strides Forward: Genie coding assistant outperforms competitors on SWE-bench by over 30 percent
An agentic coding assistant boosted the state of the art in an important benchmark by more than 30 percent.
Machine Learning Research
Scaling Laws for Data Quality: Scaling laws reveal the impact of data quality in vision-language model training
When training vision-language models, developers often remove lower-quality examples from the training set. But keeping only the highest-quality examples may not be ideal, researchers found.
Generative AI
Open Models for Math and Audio: Alibaba advances open-weight LLMs with Qwen2 Math and Audio variants
Alibaba followed up its open-weights Qwen2 large language models with specialized variations.