Advertising disclosure

Hosting Canada is community-supported. We may earn a commission when you make a purchase through one of our links. Read Disclosure.

What are “User Agents”?

The following is a snapshot of Chinese internet usage as of early 2018. For Western online marketers looking to access buying power on the OTHER side of the Great Firewall of China, you’re welcome…

(The statistics reflected below were taken from the 39th Survey Report produced by the China Internet Network Information Center – CNNIC. You can read the full report if you like poring over 100+ pages of statistics or just read our opinion on the good stuff below.)

The Definition of a User Agent

User Agent FlowchartAnyone who browses the internet today will have a user agent.

What is it?

Basically, this works as the software that bridges the gap between the internet, the user and you.

You might have an easier time understanding how user agents backtrack to look at the evolution of it.

In the past, the internet was an entirely text-based system at the beginning. Users were required to type in the commands and navigate and send messages.

Today, we have browsers that we can use to do this for us. All that we have to do is point and click. Basically, the browser acts as the agent, turning the actions into the commands.

Whenever the browser loads the website, it will identify the agent for retrieving the content that has been requested. This puts the user agent identification and the browser in with a host of information. It will send the host information for the network and device that it’s on.

Why User Agents Matter

User agents have importance because of how it controls the search engine robots for the robots.txt to crawl and file on the server. You should always understand, however, how user agents aren’t the perfect system, and they won’t function to perfection in every scenario.

In some cases, the information from the user agent can be faked. Before you can leverage such information usefully, you will need to have a deep understanding of the components and the user agent strings.

You will also have to look at the risks that come with depending on a method like this for delivering solid content to people.

User-Agent String

What are User Agents Intended For?

Ultimately, you could compare this set of information as data made for web developers to use. This lets them customize the experience based on the user agent that loads the page.

The Different Types of User Agents

Normally, browsers give us a straightforward example of what a user agent is. Still, you have other tools that could serve the purpose of the agent as well. Not every user agent will be instructed or controlled by the human in real time.

The search engine crawlers give us a fine example of one of the user agents that becomes mostly automated. This robot will trawl the web and not have a user next to the helm.

User-Agent Switcher

Here are some of the potential user agents that you could list:

Some of the crawlers that are user agents include:

  • Slurp
  • Google
  • Google Images
  • Yahoo!

Believe it or not, consoles can also act as user agents. These include the Playstation 3, Playstation Portable, Bunjalloo and the Wii.

Looking at It in Greater Detail

After the user agent identifies itself with your web server, the process that follows is known as “content negotiation.”

This lets the website serve even when you have different versions of it. This bases itself on the user agent string. The agent will take its ID card and pass it along over to the server that will read it.

HTTP Content Negotiation

Once this happens, the server will negotiate the scripts, suitable files and the media. When the web first began, user agents were intended as a programming design that could help the computer to distinguish between Mozilla from Mosaic.

The key reason for this was because of how Mozilla supported numerous frames, but Mosaic didn’t support as many of them.

How Do You Read the HTTP User Agent Strings?

If you wanted to see the user agent string with greater detail, you should look at this example. This has been generated by the WhoIsHostingThis User Agent Tool.

The results that you get with this will have unique results based on your individual network, computer and device. Let’s take the example of one from an office. The user agent application that is being used is known as Mozilla version 5.0. This could also be a software piece that acts compatibly with the system.

Whoishostingthis User Agent Example

When you look at its operating system, you will see how you have an OS X version 10.2.2. This runs on the Mac.

When we look at the client, you have Chrome version 51.0.2704.84. When it comes to the engine that displays the content, the device is referred to as AppleWebKit version 537.36.

Unfortunately, dissecting the user agent strings can be somewhat complicated because of how you don’t have a standard format with this system. Still, you have guides as well as analytic tools that you can find all across the web.

Robots TXT File User Agent

Looking at the file on the web server, you will see a robots.txt file. This controls part of how the user agents will behave. In most of the cases, you will use the robots.txt for giving the commands to the robot crawlers. Crawlers are a specific type of user agent, and the information from the robots.txt file will only apply to the crawlers.

It remains up to the crawlers to interpret this information as we have intended. Before we ever start, we have to realize that the user agent will never be the fool-proof method for identifying devices, users and browsers.

People can spoof the information, and the user also has the choice to hide this information if he wants. For most of the cases, however, user agents have become one of the useful things that developers can use.

You May Also Like:

References and image credits: