High-tech solutions to take manga around the world

SOLUTION

Mantra uses computer vision and natural language processing technologies to translate any manga from one language to another.

© Ken Akamatsu

Speech Bubble Detection

First, Mantra detects speech bubbles that contain text to be translated into another language.

© Ken Akamatsu

Text Recognition

Next, Mantra captures the text inside the speech bubble. Mantra's robust recognition algorithm helps detection from noisy images.

© Ken Akamatsu

Machine Translation

Finally, Mantra translates the detected texts into target languages and embeds them into the speech bubbles.

Watch our video to check out how Mantra works!

TECHNOLOGY

Mantra achieves state-of-the-art translation performance with novel machine translation and text detection techniques.

End-to-End Translation Pipeline

Mantra provides an end-to-end pipeline for manga translation. The fully automated processing is much faster than human translation.

Automatic Extraction for Training Data

An accurate machine translation system requires huge amount of training data. Mantra automatically extracts the training data from the published/translated manga.

Robust Text Recognition

Mantra recognizes texts on manga accurately even if the input images include varying fonts or noise.

DEMO

Mantra translates any uploaded manga into English.

Upload image here!

Translating...

or

Examples:

images are from Manga109 dataset http://www.manga109.org
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu
© Ken Akamatsu

MEMBERS

Introducing members of Mantra.

Shonosuke Ishiwatari

Ph.D. candidate at The University of Tokyo / JSPS Research Fellow (DC2). Studying natural language processing, especially interested in machine translation and description generation.

Ryota Hinami

Ph.D. candidate at The University of Tokyo / JSPS Research Fellow (DC2). My research interests include computer vision and multimedia.

Kazuhiko Yasuda

Master's student at The University of Tokyo. 2 years of experience in web application development. Qualified network specialist.

Yusuke Matsui

Yusuke Matsui received his Ph.D. degree in EECS from the University of Tokyo in 2016. He is currently an assistant professor of Institute of Industrial Science at the University of Tokyo, Japan. His research interests lie in computer vision and multimedia processing, with particular interest in comic image processing.

CONTACT