Amazon today announced that nine teams from around the globe have been selected to participate in the Alexa Prize SocialBot Grand Challenge 5 (SGC 5), a university challenge focused on advancing human-computer interaction and open dialogue conversation.
Alexa Prize is a flagship industry-academic partnership program dedicated to accelerating the science of conversational AI and multimodal human-AI interactions.
The teams selected for the challenge, which began this month, include five returning competitors — including the top two finishers in the most recent challenge — and four new universities.
|Czech Technical University, Prague
|University of California, Santa Cruz
|Universidad Politécnica de Madrid
|Luis Fernando D’Haro
|Carnegie Mellon University
|Stevens Institute of Technology
|University of California, Santa Barbara
|University of Illinois, Urbana-Champaign
The Alexa Prize, launched in 2016, challenges university teams to design socialbots that Alexa customers can interact with via Alexa-enabled devices.
The ultimate goal is to meet the Grand Challenge: earn a composite score of 4.0 or higher (out of 5) from a panel of judges, and have those judges find that at least two-thirds of their conversations with the socialbot in the final round of judging remain coherent and engaging for 20 minutes. The first team to meet the Grand Challenge will win a $1 million research grant for their university.
“Prize competitions provide an agile experimentation framework for researchers and students, encouraging them to explore transformational ideas at the boundaries of what is achievable,” said Reza Ghanadan, senior principal scientist with Alexa AI and head of Alexa Prize. “We have developed the CoBot platform and tools to lower the barriers to AI innovation for academic research community and students interested in conversational AI assistants. These tools allow students to quickly deploy their solutions in the real world with Alexa, then observe, evaluate and enhance their research results using feedback from Alexa customers.”
Updates to this year’s competition
This will be the first iteration of the SocialBot Grand Challenge to incorporate multimodal (voice and vision) customer experiences. In addition to verbal conversations, customers with Echo screen devices or a Fire TV may be presented with images or text on screen that enhance the conversational experience. Teams will have the opportunity to improve their customer interactions by including additional text and images that provide more diverse and meaningful information.
“Creating a socially adept AI is a hard problem. This is because human-like social conversation is remarkably delicate and complex, and the open domain nature of the SocialBot dialogues makes it extremely challenging,” Ghanadan said. “You need to provide relevant and deep responses to a wide range of topics users may ask, maintain a natural and coherent exchange throughout a potentially long conversation, and accurately interpret the intent of the user by correctly picking up on names, topics, places and products while taking into account the context of each conversation turn. You also need to make the interactions with users lively and engaging, which is challenging given the diversity of topics and users interacting with Alexa.”
This year there are two sets of awards: one set for overall performance and one set for scientific impact. Prizes for overall performance in the competition will be $250,000 for the first-place team, $50,000 for second, and $25,000 for third.
Alexa Prize Socialbot Grand Challenge 4 Finals | Amazon Science
In an effort to sharpen the focus on scientific contribution and advance the science behind conversational AI, this year’s university teams will not only compete for prizes related to overall performance, but also for a separate award for scientific invention and innovation.
The new award allows teams to focus on advancing the field of conversational AI through a deeper study of the fundamentals of open dialogue conversations. Cash prizes for scientific contribution will be awarded to students on the winning teams — $250,000 for the first-place team, $50,000 for second, and $25,000 for third.
A unique challenge
The SocialBot Challenge represents a unique opportunity for student researchers to experience and learn how their ideas work within a real-world environment.
“We have learned that success requires researchers to create generalizable AI techniques and incorporate knowledge in appropriate and engaging ways,” Ghanadan said. “It also involves addressing open research problems in natural language understanding and multimodal language processing, contextual understanding, natural response generation, empathy, and commonsense reasoning, to understanding social norms, and dialogue management.”
Each university selected for the challenge receives a research grant of up to $250,000, Alexa-enabled devices, free Amazon Web Services (AWS) cloud computing services to support their research and development efforts, access to Amazon scientists, the Cobot (conversational bot) toolkit, and other tools such as automated speech recognition through Alexa, neural detection and generation models, conversational data sets, and design guidance and development support from the Alexa Prize team.
In previous challenges, participating teams have improved the state of the art for open domain dialogue systems by developing improved natural language understanding (NLU) systems, neural response generation models, common sense knowledge modeling, and dialogue policies leading to smoother, and more engaging conversations.
The teams’ socialbots will be available for Alexa customers to engage with in March 2023, a finals competition will be held in early August, and winners announced later that month.
“Alexa, let’s chat”
As with the previous challenges, Alexa customers will be able to engage in conversation with teams’ socialbots when they become available in March 2023 by saying, “Alexa, let’s chat.”
Alexa customers will then receive a brief message informing them that they are interacting with an Alexa Prize university socialbot before being randomly connected to one of the participating socialbots. After exiting the conversation with the socialbot, which customers can do at any time, customers are prompted for a verbal rating, followed by an option to provide additional feedback. Over the course of the fourth challenge which concluded in August 2021, Alexa customers held more than 1 million conversations with the student teams’ socialbots, spanning tens of millions of interactions. The interactions, ratings and feedback are shared with the teams to help them improve their socialbots.
Before the new socialbots launch in March 2023, “Alexa, let’s chat” will direct you to conversations with other previously developed socialbots.
The “Alquist” team from Czech Technical University won the fourth challenge, with teams from Stanford and the University of Buffalo earning second- and third-place prizes, respectively. The publications from that challenge can be found here.
Winning teams from previous years include Emory University, the University of Washington, and the University of California, Davis.