This Robot Uses Facial Recognition to Identify Masks

Pepper, the helpful robot manufactured by SoftBank Robotics that provides services like answering guest questions and taking F&B orders, will soon be imbued with a new ability: scanning faces for masks.

Here and overseas, the four-foot-tall humanoid robot is used in places like restaurants and hotels to “welcome, inform and entertain people.” Guests can engage with Pepper through conversation and a large touchscreen located underneath its head.

Living in Las Vegas, I’ve had the opportunity to interact with a Pepper robot at the Waldorf Astoria. For the most part, my friends and I selected entertainment options from the touchscreen. Pepper performed a dance and told jokes, but also could have answered questions about hotel amenities, offered recommendations, and told us where to go to check in.

For operators, Pepper automates mundane, repetitive tasks. That functionality allows team members to focus on tasks that are more inline with their talents. Beyond “charming” guests and offering an entertaining distraction, the SoftBank robot is a marketing machine—literally. Not only does an interactive robot encourage guests to seek it out for social media posts (which in turn promote the venue in which it operates), Pepper can tell guests about promotions and new products.

Using facial recognition, Pepper is able to collect data about guests and share that information with operators and their teams. Doing so helps a business understand guest wants and needs, streamlining operations and leveraging profitable trends. And now Pepper’s facial recognition capabilities have been upgraded for today’s “new normal.”

Pepper will soon receive the ability to scan human faces for masks. This feature will first be launched throughout Europe. Using another AI-driven feature called Single Shot Detector, Pepper can recognize and scan five human faces at once. That will help venues manage small groups in mere seconds.

After a person is scanned by Pepper, their picture will appear on the robot’s touchscreen. A green circle will accompany a person’s “screenshot,” and the guest will be thanked verbally for wearing a mask by Pepper. If someone isn’t wearing a mask, a red circle will appear on their photo. Pepper will then request that the guest put on their mask.

According to SoftBank, Pepper won’t collect personal data regarding a specific guest’s mask wearing habits. The basic data that will be collected—daily percentage of guests wearing a mask versus those who aren’t—will be accessible by operators.

Obviously, Pepper isn’t cheap. There’s an initial layout of nearly $2,000 to obtain a Pepper, and there are monthly maintenance and insurance fees associated with the robot. A few years ago, The Robot Report calculated that ownership costs over 36 months can exceed $14,000.

With the hospitality industry being decimated on a daily basis, it’s unreasonable to expect many—if any—chain and independent restaurant operators to acquire new Pepper robots to assist with mask compliance (along with many other functions). The upgrade will likely be welcomed and appreciated by operations that already use Pepper; identifying possible health risks before a team member engages with a guest could prove attractive to many operators.

Several publications and health experts have predicted that wearing face coverings will extend well into 2021, whether by official mandate or habit. Many businesses may continue to require masks past the end of this year, particularly if they’ll be held liable for Covid-19 transmission.

If SoftBank’s mask recognition technology trickles down to more affordable platforms, operators may adopt it more widely. And if Pepper or a camera system can add reading people’s temperatures to its repertoire in the near future, health risks may be identified more quickly and efficiently.

Whether such features would be embraced by guests or met with skepticism due to data collection is anyone’s guess.

Neither the author nor Hospitality Villains received compensation, monetary or otherwise, from SoftBank Robotics or any other entity in exchange for this post.