Microsoft Bing announced a new AI technology that will bring 4K image experience to websites through Microsoft Edge, automatically enhancing website images. The technology, called Turing Image Super-Resolution, makes images display at a high resolution, no matter how poor the original image is.
The new technology was developed by Microsoft’s Project Turing AI development team.
Already Used in Bing Maps
The new technology is already in use in Bing Maps to sharpen the quality of their sattelite aerial imagery.
Below is a comparison of aerial imagery of Google’s headquarters in Mountain View, CA.
The screenshot of Bing Maps is on the left and the corresponding image from Google Maps is on the right:
Bing Maps vs Google Maps
How Microsoft Built the Technology
There were four important insights that led to the success of the model.
- Human Raters
- Noise Modeling
- Perceptual and GAN Loss
- Transformers for Vision: Enhance and Zoom
Human Raters
Microsoft realized that metrics used to measure success of image-related models didn’t align with human visual perception. So they created a side-by-side visual comparison tool that used human raters to help evaluate the success of the model.
Noise Modeling
Microsoft took the approach of starting with high quality images and then degrading them by adding noise to them and then teaching the model to get the image back to the original high quality state of the image.
Perceptual and GAN Loss
This was part of the effort to align the results to human vision.
The Microsoft announcement stated:
“… we found that optimizing our models solely using pixel loss between the output images and ground truth images was not enough to produce the optimal output that aligned with a human eye’s perception.
In response, we also introduced perceptual and GAN loss and tuned an optimal weighted combination of the three losses as an objective function.”
Transformers for Vision
Microsoft leveraged the power of Transformers which were used in language models, focusing on enhance and zoom.
What that means is enhancing the image and also focusing on scaling the image up, which is a difficult thing to do.
Typically it’s easy to shrink an image. But to take a small image and scale it up generally ends up maginfying the low resolution artifacts of the original image.
So what the researchers did was create a system that can calculate and “recover” the missing image data from the lower resolution image and bring it to a higher resolution.
Microsoft calls the process of scaling an image up, DeepZoom.
Edge: 4K TV of Web Browsers
Microsoft envisions this new AI feature as a way to bring a 4K visual experience to surfing the web, as well as enhancing video meetings and family photos uploaded to the web.
The technology is already available in the experimental version of Edge called Edge Canary.
The new feature will be rolling out to the mainstream version of Edge browser over the coming months.
Citation
Read Microsoft’s Announcement