Artificial intelligence technology develops so fast that it has almost been a part of our daily life. It seems AI can do a lot of things from composing music to unmanned driving, from playing games to da Vinci surgical system. While for anime lovers, AI can help them turn their selfie into anime character, just the same as other anime characters that they’ve seen in the manga.
Want to enlarge your anime images? Check out our list of Anime Image Enlargers Review.
How does AI turn selfie into anime?
Generative adversarial networks (GANs), created in 2014, is believed to lay the foundation for a variety of image synthesis applications. Researchers can either use supervised learning to learn a mapping model from paired data samples, or use unsupervised learning to learn a shared latent space and cycle consistency assumptions.
A generative adversarial network (GAN) is a class of machine learning systems invented in 2014. GAN refers to two neural networks contesting with each other in a game. One is “generator” which is to generate candidates and the other is “discriminator” which is to evaluate them.
The training objective of the generative network is to “fool” the discriminator by generating novel candidates that the discriminator thinks are not synthesized. Provided with a known dataset, the discriminator is trained to achieve acceptable accuracy.
Researchers from NCSoft, a South Korean company, proposed the “U-GAT-IT” method in a paper titled Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation. This novel method can transform a selfie into a Japanese-style anime face and generate visually superior results compared with previous techniques, such as CycleGan, MUNIT, and DRIT.
This innovative method improves GAN model performance in translating between images that are significantly different in shape, such as from cats to dogs, or from selfies to anime characters. It incorporates a new attention module in both the generator and the discriminator to identify discriminative image regions in source and target images. Then researchers trained an auxiliary classifier to guide the model to focus on important regions (eg. eyes and mouths) by generating a set of attention maps.
Adaptive Layer-Instance Normalization (AdaLIN), applied to the decoder of the generator, is to help the method flexibly control degrees of change in shape and texture without adjusting the model architecture or hyperparameters.
NCSoft is a South Korean video game giant. It has developed much well-known role-playing game series, like lineage, City of Heroes, WildStar, Guild War, and Blade & Soul. This company has been doubling down on AI since it launched its AI and Natural Language Processing Center in 2011. Then it introduced an AI system powered by reinforcement learning to battle down human professional players in its game Blade & Soul.
What programs are available?
Then what programs can you use to transform your selfie into anime character? We list the four most popular products on the market and give you a brief introduction about their performance and features. Selfie2Anime and Petalica Paint are online products while Meitu and TwinFace are mobile apps that are available both on Android and iPhone.
To find out more tools about Anime Character Creators and check out our list.
This program is an official image dataset for the U-GAT-IT model. It is totally free now. Therefore, it performs pretty well in generating anime-style characters based on real people. After you upload your portrait, this program requires you to crop the image to avatar size. Then you need to enter your email address to receive the result.
Check out how to use Selfie2Anime in this video here.
Although it performs better than many other similar products, it is far from satisfactory. The output is quite blurry since many features of the original are not properly identified and generated, such as nose, hair, eyes and so on.
It distinguishes the foreground and background. The good news is that there is not too much distortion on the whole. The outline of the figure is clear and the color is closed.
Although the stroke, color, and the whole style look like anime, there is a large gap between the result and real anime characters.
PaintsChainer is developed by PFN in January 2017. You can upload a human portrait to turn it into an anime character. Unlike Selfie2Anime, this one doesn’t require you to crop your image. But the output is a little bit scary since the color, outline, and layout are quite bizarre.
It seems it can’t handle the blurry background well and mix up the background and foreground. It identifies the human face but there is not any change.
Another function works better. You can upload your sketch image and have it colored automatically through this program. Moreover, it is possible for you to add color hints to control the colorization results.
This Chinese photo-editing app has finally spread to the West. It is available on Android and iPhone. If you want to have your own anime character, then you can choose the “anime” option and take a photo of your face. The camera will scan your face to identify your facial features, like eyebrow, hairstyle, and face shape. Then it will match up with their datasets and generate an anime character that has similar features with you. But to be frank, the generated anime character is a little bit exaggerated. You can even tune the details, like hair color, completion, and decoration. The most interesting part is that after you finish and move to the camera mode, your head will be replaced by the anime avatar that you’ve just created. It will exactly copy your moves, like blinking, raising the head and turning your head.
Besides that, you can also find a great number of filters in this app, like watery, creamy, lively, carbonic, normcore, peach, rosy, INS, Ballet, and so on. You can also upload your photos to beautify them through this app. Its AI portrait function can automatically enhance your images while taking a photo.
TwinFace applies the open-source U-GAT-IT to create an anime picture based on the input. But the results are not that satisfactory. It’s only got a 2.5-star rating on the Google Play store by 256 users. A lot of features would be distorted during your use, such as glasses, hairstyles, beards, and eye color. According to user reviews, sometimes it even produces terrible and even scary anime portraits that loos nothing like the original one. Some also complain that it takes much time to install and operate because it is too slow.
Speaking of the output, it still needs to be largely improved, especially for the details. These products either generate a good-looking anime character that doesn’t look like you at all or produce a blurry sometimes scary one that distorts a lot of features. In this regard, these products fall short of expectations for now. But they are believed to perform better as AI technology rapidly moves forward, and it will not take long.
If you want to find out more useful products to generate your portrait, check out our list of Portrait Generators Review.