Web Performance Terminology
Web metrics are measures of software characteristics which are quantifiable or countable. They are important for several reasons, including demonstrating how well (or not) a website is performing, planning work projects, and measuring productivity. The goal of tracking and analyzing web metrics is to determine the quality of a website or process, improve that quality, and predict the quality once the software development project is complete.
Types of Web Metrics
Web metrics exist on two levels - direct metrics and indirect metrics. As the term suggests, direct metrics relate more directly to the way a user perceives a website’s performance. They measure the user’s experience of a live site and are used to diagnose site-related problems. Indirect metrics, on the other hand, are used to provide explanations of what is being observed by direct metrics. Both kinds of metrics can be subdivided into primary and secondary metrics.
Primary Direct Metrics
The direct metrics that most accurately measure the user’s experience with a site are known as primary direct metrics and include the following:
-Custom timing events.
-Input latency.
-Meaningful first paint.
-Speed index.
-Time to interactive.
Secondary Direct Metrics
Secondary direct metrics measure the individual parts that comprise the whole user experience. They help to link a problem to the experience and the website’s performance in general. Some useful secondary direct metrics include:
-First paint.
-Onload.
-Render start.
-Total backend time or time to first byte.
Indirect Metrics
Indirect metrics help to provide explanations of the specific technical reasons for performance problems affecting a site. They can be provided as guidance to a development team that is designing a feature or mockup. They also assist in the building of a user interface (UI) and act as indicators of how it will perform.
Primary Indirect Metrics
Primary indirect metrics are related to page performance. They are not perfect indicators of website speed, but they do tie in closely with direct metrics. Therefore, a web designer can look at a direct metric and get an idea of how a user might perform when one of the primary indirect metrics has changed. The important primary indirect metrics to keep track of are:
-Document Object Model (DOM) weight or total DOM nodes.
-Number of requests.
-Page weight or total size of requests.
Secondary Indirect Metrics
The purpose of secondary indirect metrics is to diagnose performance problems or to check that the website does not have any obvious performance anti-patterns. Some secondary indirect metrics, such as domain name system (DNS) requests, can be used to check that the website architecture does not go over a normal amount. Others, e.g., the number of DOM nodes, can help to pinpoint a difficult-to-find performance problem.
Types of Monitoring Methods
Metrics that measure performance differ in how they can be monitored. The monitoring method establishes how and where certain metrics can be used, and will, therefore, play a role in which metrics to choose.
Real-Time Monitoring
As the name implies, real-time monitoring looks at the performance of actual users while they are interacting with a live site. Metrics that can be monitored in real time include: backend, custom events, DOM complete events, number of DOM elements, number of requests, onload, and render start. Real-time monitoring does require a live site with enough traffic to yield statistically significant results.
Real-time monitoring may need to take into account any fluctuations and differences in performance figures. For example, if a section of your audience’s network is slow for several hours, then the performance numbers for that day will be off. This makes it potentially more difficult to associate releases or code changes with changes in performance.
Synthetic Monitoring
Synthetic monitoring involves testing a site’s performance in an environment that is both simulated and consistent. This might mean running a performance tool as part of the build or using a tool to run automated speed tests. Speed index, a metric that can only be used in a simulated environment, is a widely-used synthetic monitoring metric. Any of the real-time monitoring metrics can also be used in a synthetic environment.
Because test runs can be made invariable, synthetic monitoring allows you to isolate performance changes. It is the best way to compare a site’s performance against the sites of competitors or similar sites. It also allows for consistent monitoring of a site’s performance when code/design changes are made.
Pros and Cons of Various Metrics
The following provides a brief overview of a selection of metrics (they are listed in alphabetical order).
Custom Timing Events
Custom timing metrics are timings (in milliseconds or seconds) of how long it takes a specific feature to become fully visible to the user. This metric is sometimes known as “hero image timing,” as testers often use the image of a hero as the custom event to test.
Pros: Very closely related to the designer’s goals for the site. Does not require a live URL or a complex testing setup.
Cons: Presents a challenge where the timed object changes frequently (e.g., a blog). Difficult to manage on websites that have a variety of page templates.
DOM Nodes
The number of DOM nodes is an approximate measure of how many HTML elements or tags are on the page. A browser will take longer to process and render a page if it contains more nodes.
Pros: Easy to test.
Cons: Reducing the number of DOM nodes does not generally have a significant impact on performance.
First Byte
First byte is the time from when the initial request went out from the browser to the server to when the first byte from the server comes back to the browser. Because it measures the time it takes the browser to respond, it is sometimes known as “backend time.”
Pros: Discovers performance-related problems with backend systems. Available in most testing tools and easy to test.
Cons: Does not provide the big picture, like meaningful first paint or speed index. Does not give inclusive details of backend response.
First Paint
First paint is a browser-based metric that indicates the amount of time from first byte to the first pixel rendered. The main reason to use first paint over the more accurate render start is its availability in most browsers
Pros: Easy to test and can be used for real-time user monitoring.
Cons: Inaccurate in some browsers
First Meaningful Paint
First meaningful paint is a browser-supplied metric that measures the time it takes for the most significant content to be fully rendered on the site. Measurement involves these heuristics: watching all layout events as the page is loading; filtering by events for new objects above the page fold; accounting for web font loading.
Pros: Similar to speed index, this is a very accurate measurement of how the user perceives the performance of a site. It does not require a live URL or a complex testing setup.
Cons: Only available in Google Chrome.
Input Latency
Input latency refers to the amount of time it takes for an app to respond to users as they interact with it. It differs from other metrics in that it does not relate to the initial loading and displaying of a page. Instead, it is a metric that is continuously tracked over time as the user interacts with the site. It is also possible to test synthetically. For more information, see Lighthouse’s input latency documentation.
Pros: The only metric that allows you to examine how the site responds over time as the user interacts with it.
Cons: Harder to quantify with other metrics and difficult to measure accurately.
Number of Requests
A metric that measures the total number of requests that a page makes while loading resources. Note that the HTTP/2 protocol does not limit the number of concurrent requests. When measuring total number of requests, it is important to know if the site is requesting resources from multiple domains - the number of domains can have an impact on how the browser processes requests in parallel.
Pros: Straightforward to test.
Cons: Obsolete for any site served over HTTP/2. Does not account for resources being served over multiple domains.
Onload
The load event fires when a document has finished loading and all of the objects in the document have been loaded into the DOM.
Pros: Easy to calculate for most pages and can be calculated on almost any browser in any environment.
Cons: It does not provide a metric for the validity of perceived user performance as the page may have become fully rendered and active before the firing of the load event. For more detailed information read Moving Beyond Window Onload.
Render Start
Render start is a synthetic metric that measures the time from the first byte to when the browser draws the first pixel on the screen. It is similar to first paint but provides more accuracy because it uses visual cues to calculate. It will bring up problems occurring in the request pipeline, e.g., not placing or deferring scripts at the end of a document, or the requirement of too many blocking CSS resources. See this WebPagetest discussion for more information.
Pros: It is very accurate.
Cons: Requires complex testing setups such as WebPagetest. Not as thorough as speed index or as meaningful as first paint.
Speed index
By the use of video capture, speed index measures (in milliseconds) how long it takes a page to become completely visible to the user. The score ranges from 0 to infinity, with a lower score being better.
Pros: Accurately measures a user’s perceived performance.
Cons: Requires a live URL to test, or a WebPagetest instance or service.
Time to Interactive
Time to interactive measures how long it takes for a site to become available for user interactions. The purpose and content of the site will determine whether to use this metric. For a site where interaction is the goal, this is a crucial metric.
Pros: Easy to test using Lighthouse.
Cons: May not provide useful data for content-heavy sites without interactive elements. May not be possible to test in browsers other than Chrome.
Total Page Weight
This metric is also called “total requests.” It represents an accrual of all the resource weights of a site (including the HTML of a page) and is measured in kilobytes or megabytes.
Pros: Easy to test and understand at all stages in site development.
Cons: Does not tell everything about site performance because how resources are loaded needs to be taken into account.
Which Metrics to Choose
It can be confusing to know which are the most suitable metrics to use to evaluate your site. If you are looking for more information and some guidance on which metrics to choose, read this how-to guide on performance metrics.
And if you're looking for help on building an effective measurement strategy, contact us today.
