Bing announced that it is changing the user agent string that identifies itself as Bingbot. Now there will be two user agents, one for desktop and another for the mobile crawler.
The new user agent provides more information, including the latest version of Edge.
User Agents
A user agent is a message from a software that tells the website server what kind of software device is visiting it.
A user agent can tell a website that it’s a browser or a search engine bot.
Some website software and SaaS applications rely on the user agent in order to identify search bots and do something like keep them from visiting the search or registration pages.
Bing User Agent Change is a Transition
Microsoft will continue to use the old bingbot crawler user agent until Fall 2022, which is a vague date.
However, the Bing Webmaster Tools URL inspection tool has already switched over to the new bingbot user agent.
There are apps and software that may need updating, so this gives them several months to update their software so that they can accurately identify bingbot.
But there is no need to update a robots.txt because the new version can continue to be addressed in the robots.txt as bingbot.
Robots.txt will not need to be updated to accommodate the new bingbot user agent string.
Nevertheless, for software and services that do partial matches, such as phpBB, updating code to accommodate the new user agent will not be necessary.
If a publisher, app or software uses bingbot/ to identify bingbot with a partial map then that code will continue to work.
Old and New User Agents
This is the old bingbot user agent:
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
The new bingbot user agents are an improvement because they identify between the desktop and mobile versions of the Microsoft Bing web crawler.
These are the new bingbot user agents:
Desktop
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
Mobile
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
According to the official Microsoft Bing announcement:
“We will carefully test websites before switching them to our new user-agent Bing Webmaster Tools URL Inspection has already started using the new desktop user-agent for the Live URL Test to help you investigate potential issues.
In case you face any challenges feel free to contact us.”