140th Day Of Lockdown

Maharashtra53560136843518306 Tamil Nadu3086492506805159 Andhra Pradesh2445491547492203 Karnataka1886111055993398 Delhi1461341316574131 Uttar Pradesh126722767212120 West Bengal98459671202059 Bihar8274154139450 Telangana8075157586637 Gujarat71064542382652 Assam5883842326145 Rajasthan5249738235789 Odisha4592731785321 Haryana4163534781483 Madhya Pradesh3902529020996 Kerala3433121832109 Jammu and Kashmir2489717003472 Punjab2390315319586 Jharkhand185168998177 Chhatisgarh12148880996 Uttarakhand96326134125 Goa871259575 Tripura6161417641 Puducherry5382320187 Manipur3752204411 Himachal Pradesh3371218114 Nagaland30119738 Arunachal Pradesh223115923 Chandigarh1595100425 Meghalaya11154986 Sikkim9105101 Mizoram6203230
Technology Other News 02 Dec 2019 This wholesome new t ...

This wholesome new tool makes memes enjoyable for the visually impaired

Published Dec 2, 2019, 11:06 am IST
Updated Dec 2, 2019, 11:22 am IST
Researchers at Carnegie Mellon University have made a tool to automatically identify memes & apply prewritten templates, adding alt text.
Twitter allows people to add alt text to their images, but that feature isn't always easy to find.
 Twitter allows people to add alt text to their images, but that feature isn't always easy to find.

In a new study researchers have developed a tool that will enable visually impaired people who find it difficult to understand memes, to enjoy them.

Researchers at Carnegie Mellon University have developed a tool to automatically identify memes and apply prewritten templates to add descriptive alt text, making them intelligible via existing assistive technologies.


"Memes are images that are copied and then overlaid with slight variations of text. They are often humorous and convey a shared experience, but if you're blind, you miss that part of the conversation," said Cole Gleason, a PhD student at CMU's Human-Computer Interaction Institute (HCII).

"Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention," said Jeff Bigham, associate professor at the HCII. "Many people use memes, and so they should be made accessible."


Memes largely live within social media platforms that have barriers to adding alt text.

Twitter, for example, allows people to add alt text to their images, but that feature isn't always easy to find. Of 9 million tweets the CMU researchers examined, one million included images and, of those, just 0.1 per cent included alt text.

Gleason said basic computer vision techniques make it possible to describe the images underlying each meme, whether it be a celebrity, a crying baby, a cartoon character or a scene such as a bus upended in a sinkhole.


Optical character recognition techniques are used to decipher the overlaid text, which can change with each iteration of the meme, reported the study published in - Access.

For each meme type, it's only necessary to make one template describing the image, and the overlaid text can be added for each iteration of that meme. But writing out what the meme is intended to convey proved difficult.

"It depends on the meme if the humour translated. Some of the visuals are more nuanced," Gleason said. "And sometimes it's explicit and you can just describe it."


"For example, the complete alt text for the so-called 'success kid' meme states "Toddler clenching fist in front of the smug face. Overlaid text on top: Was a bad boy all year. Overlaid text on bottom: Still got awesome presents from Santa."

The team also created a platform to translate memes into sound rather than text. Users search through a sound library and drag and drop elements into a template. This system was made to translate existing memes and convey the sentiment through music and sound effects.

Click on Deccan Chronicle Technology and Science for the latest news and reviews. Follow us on Facebook, Twitter