Microsoft created a bot to auto-caption photos and it's going hilariously wrong

It's sometimes amazingly accurate, too.
 By 
Stan Schroeder
 on 
Original image replaced with Mashable logo
Original image has been replaced. Credit: Mashable

Mere weeks after Microsoft had to shut down its Tay chatbot for sending offensive tweets, the company might be in for another AI-related embarrassment - albeit probably not as serious. 

This time, the culprit is Captionbot, an AI experiment that tries to recognize what's on a photo and come up with a text caption. Launched in late March, Captionbot actually does a pretty good job in many examples, but it's also horridly wrong in others, and when that happens, social media can be unforgiving.


You May Also Like

In one example, Captionbot mistakes the First Lady Michelle Obama for a cellphone. In another, it thinks the famous blue/black/white/gold dress is a cat wearing a tie (or a suitcase, see image above). 


It's not just that Captionbot is sometimes wrong -- it'd be absurd to expect it to always be right in a world where even humans struggle to understand what's in a CAPTCHA image. But things get awkward when it's not only wrong, but weird. Check out some examples below. 



As funny as Captionbot's failures are, we have to hand it to Microsoft for sometimes getting things amazingly accurate as well. Check out some of the things we tested, below. 

Original image replaced with Mashable logo
Original image has been replaced. Credit: Mashable


Original image replaced with Mashable logo
Original image has been replaced. Credit: Mashable

Have you seen any particularly bad or good examples of Captionbot in action? Share them in the comments. 

Stan Schroeder
Stan Schroeder
Senior Editor

Stan is a Senior Editor at Mashable, where he has worked since 2007. He's got more battery-powered gadgets and band t-shirts than you. He writes about the next groundbreaking thing. Typically, this is a phone, a coin, or a car. His ultimate goal is to know something about everything.

Mashable Potato

Recommended For You
Microsoft 365 Outlook down: Microsoft breaks silence on outage
Microsoft logo

Google Chrome unveils Gemini-powered auto-browsing feature
Chrome auto browse

Samsung to auto-tag AI-generated photos in Galaxy S26
galaxy s26 ultra smartphone

Microsoft 365 outage cause revealed
microsoft 365 logo on phone

Bring Microsoft Office staples to your Mac for less than $9 each
MacBook keyboard

More in Life

Trending on Mashable
NYT Connections hints today: Clues, answers for April 3, 2026
Connections game on a smartphone

Wordle today: Answer, hints for April 3, 2026
Wordle game on a smartphone

What's new to streaming this week? (April 3, 2026)
A composite of images from film and TV streaming this week.


NYT Strands hints, answers for April 3, 2026
A game being played on a smartphone.
The biggest stories of the day delivered to your inbox.
These newsletters may contain advertising, deals, or affiliate links. By clicking Subscribe, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.
Thanks for signing up. See you at your inbox!