Neural networks have found wide application in the field of image processing. However, high-resolution image processing algorithms using architectures such as generative adversarial network impose limitations on the execution environment because require significant computing power.
The goal of Netapix project was to develop a neural network architecture, as well as an algorithm for its training, capable of processing high-resolution images in a given style, with the ability to execute on the user’s device. This was achieved with the help of CoreML framework which was designed to work with machine learning technologies that significantly expanded the possibilities for using neural networks directly on user devices.
Phase / 01.
Deep neural networks showcase impressive results in the field of image processing. Our task comes down to finding a set of parameters sufficient to perform image processing transformation.
At the same time, each preset can be a collection of different filters, which means that there should be enough parameters to maintain variability. On the other hand, the number of parameters is limited by the amount of RAM on mobile devices and processing time.
We chose Convolutional neural networks. CNNs demonstrate good results in the field of image recognition, and also make it possible to significantly reduce the parameter space, which has a positive effect on processing speed.
Phase / 02.
To train the neural network, we've collected both:
- a huge data set of photos taken on the phone, and
- a set of presets created by photographers.
The core goal of our experiments was to come up with a one-box set of parameters that would allow to perform transformation in a uniform way for each unique preset without significant loss of quality.
As a result of fine-tuning various training parameters as well as testing various architectures, we have managed to obtain a CNN model, which was successfully replicating all filters from an in advance prepared set with high accuracy.
Phase / 03.
To demonstrate our custom technology, we've developed an iOS mobile application with functionality of a typical photo editor, namely:
- Photo editing tools
- Photo sharing
Our mobile application refers to more than 100 pre-trained neural network models as filters, implementing the functionality of unique presets created by photographers in a uniform way.