After seeing @bwasti’s creative image-to-emoji project, I created my own PyTorch implementation that transforms photos into mosaics made of 10x10 pixel emoji tiles.
I started by building a dataset from the standard emoji set on unicode.org, using their 14x14 pixel PNG versions. To give the model more to train on, I added light augmentation — Gaussian blur and a small random rotation.
For the model itself, I first overfitted torchvision.models.resnet18(), then tried the much smaller homegrown network below. Both worked, but the ResNet18 output looked cleaner to me.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class EmojiNet(nn.Module):
def __init__(self, num_classes):
super(EmojiNet, self).__init__()
self.feature_extractor = nn.Sequential(
nn.Conv2d(in_channels=3, out_channels=8, kernel_size=3, stride=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2),
nn.Conv2d(in_channels=8, out_channels=32, kernel_size=3, stride=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2)
)
self.clasifier = nn.Sequential(
nn.Linear(in_features=32, out_features=256),
nn.ReLU(),
nn.Linear(in_features=256, out_features=num_classes),
)
def forward(self,x):
x = self.feature_extractor(x)
x = torch.flatten(x, 1)
x = self.clasifier(x)
return x
Resnet18 output:
EmojiNet output:
One day I’ll have enough time to re-train a simpler model and port it to C++ with every tip and trick I can find to make it run in real time. Then I’ll be enough.
