Interviews & Essays
Noam Shazeer helped spark the latest NLP revolution. He developed the multi-headed self-attention mechanism described in “Attention Is All You Need,” the 2017 paper that introduced the transformer network.
Tech & Society
Facebook uses automated systems to block hate speech, but hateful posts can slip through when seemingly benign words and pictures combine to create a nasty message. The social network is tackling this problem by enhancing AI’s ability to recognize context.
Machine Learning Research
Conversational agents have a tough job following the zigs and zags of human conversation. They’re getting better at it — thanks to yesterday’s technology. Amazon recently improved the Alexa chatbot’s ability to identify the current topic of conversation.
Machine Learning Research
The words “big” and “large” have similar meanings, but they aren’t always interchangeable: You wouldn’t refer to an older, male sibling as your “large brother” (unless you meant to be cheeky). Choosing among words with similar meanings is critical in language tasks like translation.
Machine Learning Research
AI learns human biases: In word vector space, “man is to computer programmer as woman is to homemaker,” as one paper put it. New research helps language models unlearn such prejudices.
Machine Learning Research
Language models can’t correct your misspellings or suggest the next word in a text without knowing what language you’re using. For instance, if you type “tac-,” are you aiming for “taco,” a hand-held meal in Spanish, or “taca,” a crown in Turkish?
Machine Learning Research
Natural language processing lately has come to resemble an arms race, as the big AI companies build models that encompass ever larger numbers of parameters. Microsoft recently held the record — but not for long.
The Batch Newsletter
Last week, I wrote about the diversity problem in AI and why we need to fix it. I asked you to tell us about your experiences as a Black person in AI or share the names of Black colleagues you admire.
Tech & Society
Vendors of face recognition are updating their tech as people don masks to protect against Covid-19. Police are bound to take notice. Companies that provide computer vision systems, including at least one that supplies law enforcement agencies, are training models to recognize obscured faces.
Science
The ocean contains distinct ecosystems, but they’re much harder to see than terrestrial forests or savannas. A new model helps scientists better understand patterns of undersea life, which is threatened by pollution, invasive species, and warming temperatures.
Tech & Society
Machine learning researchers report better and better results, but some of that progress may be illusory. Some models that appear to set a new state of the art haven’t been compared properly to their predecessors, Science News reports based on several published surveys.
Machine Learning Research
Reinforcement learning can beat humans at video games, but humans are better at coming up with strategies to master more complex tasks. New work enables neural networks to connect the dots.