Tech & Society
Language Models, Extended: Large language models grew more reliable and less biased in 2022.
Researchers pushed the boundaries of language models to address persistent problems of trustworthiness, bias, and updatability.
Language
Business
Programmer’s Best Friend: Code generation services took off in 2022.
Behind schedule on a software project? There’s an app for that. Language models fine-tuned on computer code proved capable of generating software routines similar to the work of experienced developers — though the results can be hit-or-miss.
Machine Learning Research
Memorize Less; Retrieve More: How small language models can perform specialized tasks.
Large language models are trained only to predict the next word based on previous ones. Yet, given a modest fine-tuning set, they acquire enough information to learn how to perform tasks such as answering questions.
Culture
How Alexa Says Goodnight: Amazon Echo uses generative AI to create bedtime stories.
Too exhausted (or unimaginative) to tell your child a bedtime story? Amazon’s smart displays can spin bespoke tales on demand. A feature called Create with Alexa generates children’s stories complete with illustrations, music, and sound effects on the Amazon Echo Show device.
Tech & Society
More Plausible Text, Familiar Failings: ChatGPT hasn’t overcome the weaknesses of other large language models
Members of the AI community tested the limits of the ChatGPT chatbot, unleashing an avalanche of tweets that made for sometimes-great, sometimes-troubling entertainment.
Tech & Society
Translating a Mostly Oral Language: How Meta Trained an NLP Model to Translate Hokkein
Most speech-to-speech translation systems use text as an intermediate mode. So how do you build an automated translator for a language that has no standard written form? A new approach trained neural networks to translate a primarily oral language.
Hardware
Built to Scale: Andromeda Supercomputer from Cerebras Speeds up AI
A new computing cluster delivers more bang per chip. Cerebras unveiled Andromeda, a supercomputer based on its processors. Unlike conventional clusters, the system’s processing speed rises linearly with additional processors.
Machine Learning Research
Right-Sizing Models for the Dataset: Finding the Best Data-To-Parameter Ratio for NLP Models
The route to improving transformer-based language models like GPT-3 and Gopher, which are trained on immense quantities of text scraped from the web, has been to increase their size. But research shows that, given a processing budget, bigger doesn’t necessarily mean better.
Tech & Society
When Safety Becomes Surveillance: Colleges Track Students Using AI Designed to Monitor Mental Health
United States colleges tracked activists using a natural language processing system intended to monitor their mental health.
Machine Learning Research
Massively Multilingual Translation: Machine Learning Model Trained to Translate 1,000 Languages
Recent work showed that models for multilingual machine translation can increase the number of languages they translate by scraping the web for pairs of equivalent sentences in different languages. A new study radically expanded the language repertoire through training on untranslated web text.
Business
Generating Investment: Generative AI Startups Raise Hundreds of Millions in Funding
The generative gold rush is on. Venture capitalists are betting hundreds of millions of dollars on startups that use AI to generate images, text, and more, Wired reported.
Tech & Society
The Black Box Awakens: Confronting the Fear of Self-Aware AI in 2022
AI researchers are starting to see ghosts in their machines. Are they hallucinations, or does a dawning consciousness haunt the trained weights?