Guider Daemon Automates Linux Performance Monitoring

SEATTLE–By the time he finished his presentation on the Guider Linux performance monitoring daemon, Peace Lee got three (3!) separate rounds of applause from the audience.
The Linux greybeards in attendance were impressed with the program, that he wrote the program himself (over a period of nine years) and that he did so without any help from system libraries.

Logo
Indeed, Guider is an impressive piece of work. Written in Python and run from the command line, Guider is a full observability suite for Linux systems, with over 150 commands for analysis and experimentation, capable of all manner of monitoring, profiling, tracing, and logging functions.
Initially released in 2021 under a GPL 2 license, Guider has thus far amassed 562 stars on GitHub, with 85 forks.
Lee built Guider for his job as a performance architect at Hyundai Motor Corp.
He needed an extremely accurate and low-footprint real-time monitoring tool, one with the ability to detect issues and file reports required to debug increasingly complex in-car software systems.
He found the existing set of Linux performance monitoring and profiling command line utilities too difficult to work with, at least for what he needed.
“Most platforms provide good debugging and analysis apps. However, when the scope of the problem extends to the service or platform level [such as for automotive systems], analyzing and improving it becomes much more difficult,” he told the audience.
What Can Guider Do?
The code is available from GitHub and can be downloaded in your favorite Python environment via Pip.
Once installed, Guider can be set upon CPU and GPU, memory, storage and other drivers, interrupt requests and inter-process communications (IPC).

Guider features.
Guider measures everything in precise increments of time (microseconds), and size (bytes) and can write everything to a file for later analysis.
It can measure the resource usage of systems and tasks. It can snapshot data. It can count occurrences and send notices when thresholds have been reached.
In the presentation, Lee used the memtrace command to track the memory usage of a 100MB application.
Many of Guider’s functions resemble those of the built-in Linux command line tools, such as Top, which shows a top-level view of a system’s performance.
If you are wondering why your app is slow to boot, why your apps quit, your video stutters, or your system suddenly resets itself, fire up Guider.
Automated Reporting
And if these 150+ commands aren’t impressive enough on their own, the software also comes with what looks to be a very robust set of reporting and visualization tools.
It can visualize results with SVG, which can be easily rendered via browser. The familiar Flame Graph, heatmaps other familiar observability formats are used in many cases.
Guider can also be configured to check for events, watch when thresholds are met, file reports on the anomalies and even execute other commands in response.

A Guider configuration file for automating CPU usage-based actions).
Lee also discussed the software’s bulk data analysis capabilities, in which performance data from multiple sources (such as Lidar data from automobile object detection systems) can be captured and analyzed for long-term trends.