Simulation Performance Models

Conceptually, simulation models a real-world system as a computer program. Simulation allows a system to modeled at any level of detail: from a direct translation of a queuing network model to capturing every aspect of the system's behavior. Simulation also supports the collection of any performance metric that can be defined and programmed.

Architecture of simulations

Simulations of computer systems written for the purpose of performance analysis are generally discrete event simulations. In a discrete event simulation, the representation of time is quantified, and the system state only changes when an event occurs. By comparison, a continuous simulation models time as a continuous progression.

Discrete event simulation of computer system may be further categorized as event-driven or cycle-based. An event-driven simulation models system activity as a series of events that occur asynchronously and at irregular intervals. For example, a simulation of network file server would include events such as the arrival of Ethernet packets or the completion of disk writes. Event-driven simulations can model a wide variety of systems. In cycle-based simulations, all changes in system state are synchronized to a single clock. The simulation is essentially a large state machine that changes state at each clock. Cycle-based simulation are typically used to model processor cores or other digital logic with a single clock frequency. Note that the two approaches can be combined in a simulation that includes clock ticks as a simulation event.

Simulation of Workloads

There are three basic techniques to generate workloads for simulation: stochastic, trace-driven, or execution-based. Stochastic simulations describe the arrival patterns of customers and other aspects of the workload by sampling from a probability distribution. Many workloads can be described accurately by using the appropriate distribution. Stochastic workloads are a good choice when detailed information about the workload isn't available, or where there is a need to vary workload characteristics. Workload generation is efficient and does not require large data files.

Trace-driven simulation represents the workload as a sequence of operations or requests. For a web server simulation, a sequence of HTTP requests would be a suitable trace, while a simulation of next-generation x86 CPU might use a trace of micro-operations. If trace data that accurately represents the workload is available, it can lead to good simulation results while avoiding the need to write simulation code to model the workload. The drawbacks of using traces are that trace collection is a nontrivial effort and that the data files are very large.

Both stochastic and trace-driven simulations can be written at many levels of detail. The third simulation method, execution-based simulation, is used to model processors in detail. The input to an execution based simulation is the same executable code that the real system will run. Although this method requires writing and validating a detailed simulation, it leverages existing compilers and permits a wide-variety of workloads to be modeled with little effort.

The principal drawbacks to using any simulation are the effort required to write and validate the simulation program, and often considerable computing requirements (CPU time for all simulations, disk space for traces). In general, a simulation will run many times slower than the real system: an hour of simulation may only yield a few seconds of real time.

Simulation Tools and Resources

Please note that:

There is a vast array of simulation software available, both commercial and open source. The two packages that the FrontRunner staff has the most experience with are:

• OPNET Modeler^® is the leading commercial software for network simulation. Perhaps its strongest point is the extensive library of link types, protocols, and specific devices. An excellent choice for projects that need to including existing network technology in a detailed model. Modeler could also be used for many non-network simulations.
• Hyperformix (formerly SES) Workbench^TM is another comprehensive simulation tool. Compared to OPNET Modeler, it supports a more generic and abstract approach to simulating computer systems. A good choice for modeling systems where both active and passive resources are significant.

Both of the above packages offer impressive capabilities at an impressive price. If you are looking for something more economical or perhaps just a different modeling paradigm, there are many other options available. Here are some ideas for starting your search for simulation software:

• Mesquite Software offers CSIM18, a processing oriented modeling library for C/C++ programs. This is commercial software, but it's about an order of magnitude less than either Modeler or Workbench.
• ns2 is an open source network simulation that's popular with researchers. It has good support for the TCP/IP protocol stack, the core protocol of the Internet.
• The Open Directory Project and Google both offer directory pages devoted to simulation.

Return to the Understanding Computer Performance Analysis page.
Return to the FrontRunner Computer Performance Consulting home page.