Pixelate me: creating a telegram bot to make beautiful pixelated profile pictures

Recently, the quality of the photos has been growing rapidly. However, many of us would like to add some pixel-style images to our lives.

Just to solve this problem, we will create a bot for Telegram, which will process our photos and all this in 15 minutes using Google Colab.

We will use Google Colab because it is providing GPU instances for free. After 15 minutes of coding you will be able to generate images like this:

Algorithm’s result

Creating Google Colab file

Google Colab Notebooks provide free GPU instances to accelerate image processing using known deep learning libraries such as PyTorch or TensorFlow. To create Google Colab notebook go to colab.research.google.com → “New Notebook”

Fig. 1. Creating new Google Colab notebook

Face detection & Segmentation

To detect the area that we are interested in the photo, we will use face detection algorithms — face_recognition.

Face recogniton

To install this package simply run !pip3 install face_recognition. Let’s define a function which takes an image as input and returns the largest face on the picture cropped with some padding around:

One can detect the largest face and crop image using get_cropped_image function

Note, that we explicitly added padding around detected bounding box because face detection algorithms usually ignore hair, neck, ears etc.

As for segmentation part we will use Grapy-ML which may installed by using the following code:

!git clone https://github.com/Charleshhy/Grapy-ML > /dev/null

To use the segmentation network, we need to download weights using gdown package. Run the following code in your notebook:

Downloading weights for the segmentation model

Let’s define initialization for the segmentation model:

Helper function to initialize a segmentation network

Main algorithm

The algorithm consists of the following steps:

• Detect faces and choose the largest one
• Crop an image with padding
• Segment person on the cropped image
• Pixelate
• Find and add contours using convolution

Let’s define pixelate function:

The function takes an image (np.ndarray) or path (str) as an input and returns a dict of images

Telegram Bot

There are several telegram API wrappers, we will use pyTelegramBotAPI.

!pip3 install requests > /dev/null
!pip3 install pyTelegramBotAPI > /dev/null

Now we can import all we need:

import logging
from io import BytesIO
import numpy as np
import requests
from PIL import Image
from skimage.transform import rescale, resize
import telebot

To use Telegram API you need to obtain your bot’s token from @BotFather. Write a message to the bot and follow the instructions. Add token variable:

TOKEN = '4346%%%%%:AAH7TebGbSKim6cPPC_um6Tt3yKzy%%%%%%'

Finally, we can initialize a bot with a message handler, that returns the text of a sent message:

Bot initialization with two simple handlers

However, we need to somehow check if a user sends an image, process it, and send the result back. We can do that with one more handler:

Now, run the bot using bot.polling()and see results

Constructing DeepLabv3+ model...
Number of classes: 21
Output stride: 16
Number of Input Channels: 3
2020-05-09 18:15:05,687 (__init__.py:420 MainThread) INFO - TeleBot: "Started polling."
2020-05-09 18:15:05,688 (util.py:59 PollingThread) DEBUG - TeleBot: "Received task"
2020-05-09 18:15:05,691 (apihelper.py:54 PollingThread) DEBUG - TeleBot: "Request: method=get url=https://api.telegram.org/bot43xxxxxxx:xxxxTebGbSKim6cPPC_um6Tt3yKzy1frPx0/getUpdates params={'offset': 1, 'timeout': 20} files=None"
2020-05-09 18:15:25,725 (apihelper.py:64 PollingThread) DEBUG - TeleBot: "The server returned: 'b'{"ok":true,"result":[]}''"
2020-05-09 18:15:25,726 (__init__.py:323 PollingThread) DEBUG - TeleBot: "Received 0 new updates"
2020-05-09 18:15:25,727 (util.py:63 PollingThread) DEBUG - TeleBot: "Task complete"

Our bot is working!

That’s it! Code is available at GitHub or at Google Colab