Go to ...

RSS Feed

July 16, 2020

Toolkit Item: LatencyMon

LatencyMon, from Resplendence Software Projects, describes itself as a “real-time audio suitability checker.” Essentially, the software works by emulating an active audio playback session in Windows Vista (and newer versions, through Windows 10 and Server 2019). While it’s running the program measures various kinds of delays that occur on its host system and reports back through its UI. This program is an essential tool for system administrators, audiophiles, and audio professionals who want to play the highest-possible quality sound on a Windows PC. It measures various kinds of process latency on such systems — hence the name for the program — and is sensitive to delay thresholds that will likely (or definitely) impact the system’s ability to play back audio accurately and correctly.

Looking Good with LatencyMon

Color gradations in the LatencyMon GUI range from bright green (best) into darker shades (OK, but not as good) into a yellowish-brown (questionable) and on into red shades (unacceptable). This provides quick, intelligible visuals to help users gauge the suitability of the PC under test for audio playback. Here’s program output for a healthy system with no apparent (or actual) audio playback issues:

Toolkit Item: LatencyMon.OKmonitor

Even an older system (a Lenovo T520, in this case) is able to play back audio nicely, as this LatencyMon output clearly shows.
[Click image for full-sized view.]

The preceding graph makes mention of various causes of latency on Windows PCs. These include:

  • Interrupt to process latency: the time it takes the system to field an interrupt (switch over from the current active process to another active process). This switchover can occur for any number of reasons. It might mean that the time slice allocated to the current process has expired, so it’s rotating out to be replacing by the next item in the process queue. It might mean that the current active process has become blocked (usually, that means it’s waiting on some system resource, often I/O reads or writes) so another process needs to swap in to keep the CPU busy. Or it might mean a higher-priority process has announced itself, and is going to interrupt the current process and take over. The “interrupt to process” latency measures the time it takes to stop doing one thing (the current active process) and start doing another (the new active process that’s taking over the CPU).
  • ISR stands for “Interrupt Service Routine” and is a special short program that helps to orchestrate CPU activitiy. Any active thread running on a CPU is temporarily halted when an Interrupt gets serviced — and that’s precisely what an ISR does. If a higher priority process (or even some other process) needs to take over, an ISR can scheduled a deferred procedure call (DPC) to offload the activity underway so that the CPU can turn its attention to something else. Exceptionally high ISR execution times means it’s taking a (relatively) long time to switch from one active task to another active task. This can be an issue when real-time outputs (like audio or video) are active on a PC.
  • DPC stands for “deferred procedure call” and refers to what happens when an ISR decides that the current active process needs to relinquish the CPU and get back into the “awaiting service” queue to get another turn sometime in the (hopefully very near) future. DPCs usually run on the same processor on which they’ve running. Thus, they have to wait until the ISR is finished before they can resume. In the case of an audio or video program that’s running, it has to wait for the ISR to finish, and then for the DPC to run to completion to once again take over the CPU. As the program documentation says “If execution time gets too high, the audio program may be unable to deliver audio buffers to the hardware in a timely manner.” This is what causes stuttering (which actually means audio is starting and stopping quickly and often rather than playing continuously). It can also cause clicks and pops because sounds get interrupted only partway through, and sound more like noise than audio playback.
Working Best with LatencyMon

Using LatencyMon is a performance measurement  and management exercise. Thus, a few important caveats are worth sharing for those who may wish to put this useful too to work:

  • You don’t need to run one or more audio processes while using LatencyMon. It already emulates an audio process and is able to gauge system performance without actually playing sound. That said, the LatencyMon FAQ (a document worth at least skimming over, if not reading in detail) says that the only reason one might run audio while using LatencyMon is to get an accurate count of hard pagefaults from the audio process itself. (A hard page fault requires disk access, because it means the page reference is not already resident in memory, and represents a processing event that takes several orders of magnitude longer — milliseconds instead of microseconds — for Windows to satisfy.) In some cases, this may actually be significant — and might indicate that a different driver or audio program would work better on the test machine.
  • When using LatencyMon to examine a system, be sure to run a characteristic workload. You don’t want to let the system idle along doing nothing (seldom representative of real usage scenarios, except perhaps for home theater or strictly audio playback PCs). But you don’t want to overload your system, either. I was able to turn a system that LatencyMon found adequate for audio playback (see preceding screen capture) into one that was rated as problematic by running compute- and I/0-intensive tasks (e.g. Disk Cleanup, sfc /scannow, or any of the DISM /online /cleanup-image commands). Do your monitoring when the sytem is in normal use, when its user also wants to play back audio.
  • I usually run LatencyMon for 60 -180 seconds to get a sense of general latency characteristics. Over time, scheduled Windows jobs will inevitably run and may adversely (and unfairly) affect the program’s estimates of system suitability. Again, you want to base your measurements on real-world situations and similitude as much as you can.
When LatencyMon Finds Issues or Problems

You’ll see that things are not going so well, latency-wise, just from the color spectrum of the output. Here’s what the same system shows in LatencyMon if I run one of the afore-mentioned compute- or I/O-intensive activities while it’s monitoring:

Toolkit Item: LatencyMon.trouble

If LatencyMon shows red (and provides red-text status info) things are not going well.
[Click image for full-sized view.]

What to do About LatencyMon Issues or Problems

LatencyMon offers a variety of tabs, in addition to the Main graphical display (appears in both prior screen caps). If a host PC appears to manifest issues, check the other tabs for more guidance.

Here’s a brief summary of what you’ll find on each of them, including info that is likely to help with remediation or workarounds to improve overall latency characteristics:

1. Stats: You can’t read the info for this tab until you stop monitoring (otherwise, the program is continuously updating this info, and won’t let you scroll through it to read its findings). It tells you such things as the process with the highest pagefault count, total pagefaults, hard pagefault count from hardest hit process, and number of processes that experienced pagefaults. You also get per CPU information here including interrupt cycle times, plus ISR and DPC highest execution times and counts. These might be helpful in diagosing an overloaded or overtaxed CPU.
2. Processes: lists all processes active in Windows during the monitoring period, with information about hard pagefaults. If you sort on that column (click the “Hard pagefaults” header item) it will show you the processes that experience hard pagefaults in ascending (or better yet, descending, order of occurrence). You’ll be interested in those processes where the most pagefaults occur. For example, on my T520, I had over 200 processes active, but only 38 experienced page faults, and only 5 had more than 100 page faults, while 30 had fewer than 10 page faults, and 20 had 5 page faults or fewer). My top offenders included programs I didn’t need to leave open (e.g. Settings and Snipping Tool) which suggests some easy ways to reduce system load.
3. Drivers: shows all drivers active in Windows during the monitoring period, with infomration about ISR and DPC counts, and highest/overall execution times. Sorting on ISR and DPC counts is both helpful and informative and shows where system delays originate. I was amazed, for example, to see that the USBPORT.SYS driver was responsible for the vast majority of ISRs on the T520 PC (57%). Factoring out LatencyMon itself (the biggest cause of DPCs), OS components accounted for the vast majority of DPCs — just as you’d expect on a healthy system with good audio playback characteristics.
4. CPUs: shows per processor stats, including ISR and DPC counts, execution times, and most active drivers. Nothing too exciting or informative here (except perhaps on systems with failing CPUs/processor components).

All in all, LatencyMon is an informative tool for examining and troubleshooting PC audio playback issues. Though the free version is readily downloaded from Resplendence.com, IT pros should be aware its licensed only for personal or home (non-commercial) use. If you want to make use of this excellent tool at work — and you probably will — you’ll need to shell out the approximately US$120 or €100 that the company charges to use this tool in a commercial setting. IMHO, it’s definitely worth the cost!

Author: Ed Tittel

Ed Tittel is a 30-plus-year computer industry veteran. He’s a Princeton and multiple University of Texas graduate who’s worked in IT since 1981 when he started his first programming job. Over the past three decades he’s also worked as a manager, technical evangelist, consultant, trainer, and an expert witness. See his professional bio for all the details.

Leave a Reply

More Stories From Admin Tools