Analytic Performance Models

An analytic performance model is a mathematical construct that represents key aspects of a computer system. Analytic performance models are an excellent tool for quickly evaluating the performance of new or modified product. They are also well suited to comparing the performance of several alternative designs. There are several analytic modeling techniques that useful for computer systems performance analysis, as discussed below:

Analytic Techniques

Bounding Calculations

These are simple "back of the envelope" calculations that are done to quickly assess the maximum throughput of a single system component. For example, a web site that is serving one KB web pages over a 10 Mbit/sec Ethernet can provide at most 1220 pages/second.

Bounding calculations are quick and easy to understand, but they have several significant limitations. In almost all cases, they yield optimistic predictions: as the system is considered in more detail, predicted performance decreases. For the example above, the Ethernet, TCP/IP and HTTP protocols all add headers to the served pages that use part of the network's capacity. Simple bounds can not predict latency, nor can they model any interaction between components of the system. For these reasons, the best uses of bounding calculations are to quickly eliminate unworkable design options from consideration and to sanity check a more detailed model.

Queueing Network Models

A queuing network model represents a system as a set of service centers that process customers (also known as jobs, transactions, or arrivals). Service centers can be any active resource such as a CPU, disk or network link. Queueing network models can predict the latency, throughput and utilization of a system and its components. One leading text (Quantitative System Performance) reports that, in most cases, throughput and utilization are predicted within 10% of measured values, while latency predictions are within 30%.There are several types of queueing network models, as described in the following sections:

Asymptotic Bounds

Asymptotic models are the simplest application of queuing network models. They are used to predict the best-case and worst-case latency and throughput of a single service center. A typical use of asymptotic bounds is to examine the behavior of a bottleneck resource under various loads. These calculations can be done by hand.

Open Network Models

In an open network model, customers pass through a system once. For example, a network device processing a stream of packets would be modeled as an open system. The number of customers in the system is calculated from the customer arrival rate and the characteristics of the service centers. If there are several distinct types of customers, a multi-class open model is used. While it's feasible to analyze a small open system by hand, in most cases a spreadsheet or program would be used.

Closed Network Models

In a closed network model, the number of customers is fixed. The classic example is a time-sharing computer system with a fixed number of user terminals. In a closed model, the number of customers is known and the throughput is calculated. An iterative method known as Mean Value Analysis (MVA) is used to calculate throughput and latency.

As with open models, closed models may be either single-class or multiple-class. There are two variations on the MVA algorithm: exact solution and approximate solution. Note that "exact" and "approximate" refer to the calculations, not to how closely the queuing network model reflects the real system. The approximate model is typically used for multiple-class models with large numbers of customers. Due to the iterative nature of MVA, close models are best solved by being programmed.

Limitations of Queuing Network Models

Although queueing network models are a versatile and powerful way to model system performance, they do have some limitations. All of the solution techniques require that model meets certain conditions that make it a separable model; in some cases this could make the model deviate significantly from the real system. Also, queuing models are not well-suited to modeling passive resources such as buffers, although some approximations do exist

Markov Chains

Markov chains can be used to model systems as a finite set of states with known rates of transition between states. They are well suited to predicting system availability given component failure rates and repair (recovery) rates. They can also be used to model buffers and queues that hold a small number of identical items.

Resources for Analytic Modeling

This section contains pointer to more information about analytic performance modeling as well as tools to aid the modeling process. Please note that:

Information about Analytic Modeling

The definitive introduction to queueing network models is Quantitative System Performance: Computer System Analysis Using Queueing Network Models, by Edward D. Lazowska, John Zahorjan, G. Scott Graham, and Kenneth C. Sevcik. Originally published in 1984 by Prentice-Hall, the complete text is now available (as PDF files) on the web for viewing or download.

Another good resource is part VI of The Art of Computer Systems Performance Analysis by Raj Jain. This covers a wider scope of topics than the Quantitative Systems Performance but in less depth.

Tools for Analytic Modeling

There are several types of tools that are useful for evaluating queuing network models, markov chains, or other analytic performance models. Spreadsheets can be used for many simple calculations.

Wolfram Research's Mathematica^® is an versatile and powerful tool for both symbolic and numerical calculations. The FrontRunner staff uses it extensively to evaluate queueing network models and markov chains.

Specialized software to evaluate queueing network models is limited, and most appears to open-source or academic work that may have limited support. Professor Myron Hlynka of the University of Windsor maintains a list of queueing software that includes some tools for queuing network models.

PDQ is open source software (written in C) for evaluating queueing network models of computer systems. It is maintained by Dr. Neal J. Gunther of Performance Dynamics Company^TM. [Disclaimer: FrontRunner CPC has no experience with this tool].

Return to the Understanding Computer Performance Analysis page.
Return to the FrontRunner Computer Performance Consulting home page.