Phoronix Test Suite

Phoronix Test Suite

The Leading Software For Automated, Open-Source Testing & Benchmarking

Latest Release
Phoronix Test Suite 5.2.1-Khanino
11 July 2014

About

Phoronix The Phoronix Test Suite was developed out of the tools devleoped at Phoronix.com since its founding in 2004. The Phoronix Test Suite is designed for Linux benchmarking but also supports other operating systems. Read more about the Phoronix Test Suite.

OpenBenchmarking.org

OpenBenchmarking.org OpenBenchmarking.org was launched with Phoronix Test Suite 3.0 as a means of sharing benchmark results with the greater community, collectively collaborate over test results, store test profiles and test suites within a centralized cloud repository, and carry out other tests. Visit OpenBenchmarking.org to learn more.

Phoromatic

Phoromatic Phoromatic is an enterprise-designed component to the Phoronix Test Suite to centrally manage entire test farms/clusters of systems, carry out tests on a time or triggered basis, and provide support for other centralized test management purposes. Learn more at Phoromatic.com.

Commercial Support

Commercial Support The Phoronix Test Suite's future development is dependent upon the continued support from organizations and other stakeholders taking advantage of this long-standing, open-source benchmarking software. We are happy to offer custom engineering services, commercial support, sponsorship of new open-source software features, custom test profiles, and other commercial support options for those interested in helping advance this automated Linux benchmarking software. Contact us to learn more.

Phoronix Test Suite 2.2 Changes

Phoronix Test Suite 2.2.0 "Bardu"
November 16, 2009

- pts-core: Various code path optimizations and cleaning
- pts-core: Add dump-system-sensors option to display a readout of all supported Phodevi sensors and their values and print unsupported sensors
- pts-core: Fix TEST_RESULTS_NAME environmental variable not working
- pts-core: Fix network-setup network testing check actually use the proxy, hook in the HTTP proxy override properly
- pts-core: Clean-up download cache directories code
- pts-core: Add "installed-tests" virtual suite of tests that are currently installed
- pts-core: On GTK2 GUI Don't show buttons/header for reference system comparisons for results when there are actually no results to compare
- pts-core: Small OpenSolaris 2010.02 support updates
- pts-core: Various small bug fixes
- phodevi: Support for the GPU fan-speed sensor with ATI graphics driver
- phodevi: Add support for reading the stock GPU engine/memory clock when using the open-source Radeon driver
- phodevi: Fixed detection of the xf86-video-radeonhd display driver
- phodevi: Fix throwing SUSE Linux 11.2 into an infinite loop when running "X -version" on this OS, "Xorg -version" works fine instead
- pts: Update darwin test suite
- pts: Support using zypper to install external dependencies on SuSE/OpenSuSE since it can ignore missing / removed packages without failing
- documentation: Update documentation for Phoronix Test Suite 2.2

Phoronix Test Suite 2.2 Beta 3
November 9, 2009

- pts-core: Fix API breakage for installing dependencies from install-dependencies option call
- pts-core: Report the disk scheduler to the test notes area when a disk test is run
- pts-core: Add SelectSuitesOrTests tag to user-config.xml for selecting between tests / suites showing by default in GUI
- pts-core: Add initial analyze-result-file command for showing basic analytical / stats for a given result file
- pts-core: Support showing information from info command when supplying a virtual suite
- pts-core: Don't contain tests that are of license RETAIL or RESTRICTED in the "all" virtual suite
- pts-core: Fix where recover-run would not recognize the result file if there was not at least one successful one previously
- pts-core: When using recover-run, put back in batch mode state if the test was originally run in batch mode
- pts-core: Validate test installations earlier in the test run process
- pts-core: When clicking "View Results" from GTK2 GUI, generate graphs if needed
- pts-core: Fix regression that broke reference comparisons and removing identifiers from result files
- phodevi: Add support for GPU fan speed monitoring (currently only implemented for NVIDIA's binary display driver)
- phodevi: Add property to disk object for reading the scheduler in use
- phodevi: DMI parsing support on Linux through /sys/class/dmi/id/ interface, for now implement motherboard way
- phodevi: Detect and support reading the X.Org DDX driver module version for Cirrus and other obscure drivers/hardware
- pts: Update for Fedora external dependencies XML
- pts: Start adding new reference comparison systems with runs against Phoronix Test Suite 2.2

Phoronix Test Suite 2.2 Beta 2
November 3, 2009

- pts-core: With toggle_screensaver module, disable the display(s) from sleeping when on AC power in GNOME during testing
- pts-core: Clean-up more code used during the test execution process
- pts-core: Rename SaveSystemDetails user-config.xml option to SaveSystemLogs
- pts-core: Add option to archive a test's installation log to a sub-directory of the results file
- pts-core: Check for user-config-template.xml template in pts-core/static/
- pts-core: Add PTS Desktop Live user-config.xml template
- pts-core: Move txt file lists from pts-core/static/ to pts-core/static/lists/
- pts-core: Allow test options to be specified externally via PRESET_OPTIONS environmental variable (see commit for example)
- pts-core: Fix bug where from GTK2 GUI selecting to not save test results would prompt you from CLI to save results or not
- pts-core: Vastly re-work the pts-includes-run_options code
- pts-core: Move images to pts-core/static/images/
- pts-core: Create modules tag section in user-config.xml, move default modules to load into the LoadModules tag instead of text file
- pts-core: Start adding user-interface options to the user-config.xml
- pts-core: Updates for library / Phoromatic server usage
- pts-core: Don't ask user whether to try again on a failed test download when in automated/batch mode
- pts-core: When installing external dependencies fails, continue installing just the tests where the dependencies are satisfied]
- pts-core: Add __event_user_error hook to PTS module API
- phoromatic: Prevent overloading the Phoromatic server if the client module keeps trying to communicate too many times per minute
- phoromatic: Support for reporting system/PTS warnings to Phoromatic server
- phoromatic: Support for setting the default system title on the Phoromatic server as the test system's hostname
- phoromatic: Report new system software / hardware for test system to Phoromatic server when appropriate
- phoromatic: Add support for uploading Phoromatic-sanctioned test results to Phoronix Global, if Phoromatic server tells it to
- phoromatic: Add support for Phoromatic server to tell client whether or not to archive the test results locally upon completion
- phodevi: Improve phodevi smart caching, bring actual caching into pts-core and place the cache_store in the core.pt2so
- phodevi: Fix possible space when getting resolutions from xrandr
- phodevi: Don't report GPU memory clock in GPU string when GPU memory clock is 0MHz (NVIDIA reports this for at least some IGPs)
- pts: Add workstation test suite
- pts: Add disk-ssd test suite
- pts: Revamp server test suite
- documentation: Add a basic reviewer's guide

Phoronix Test Suite 2.2 Beta 1
October 27, 2009

- pts-core: Fix for running result files that have forced AA/AF levels and then fail to merge
- pts-core: On missing timezone setting, set to UTC if the PHP version is 5.3.0 or newer, emits a warning otherwise
- pts-core: Rename change-results-display-order option command to reorder-result-file
- pts-core: Add edit-result-file meta option
- pts-core: Add MinimumLength tag to XML test profile specification
- pts-core: Add MaximumLength tag to XML test profile specification
- pts-core: Few fixes/changes to test execution code
- pts-core: When dealing with the stats support for validating test results, don't increase the run count prematurely
- pts-core: Fend off divide by zero errors in the standard deviation code
- pts-core: Improve performance of PTS assignment functions
- pts-core: Don't remove the PTS run_lock when running a secondary instance of PTS
- pts-core: Add debug-run option to provide more run-time information to assist test profile writers in debugging / verifying profiles
- pts-core: Drop central pts-core debugging code since it was very rarely used and not too helpful
- pts-core: Add in additional LC_NUMERIC override to eliminate non-US decimal problems with test results
- pts-core: Various test run code clean-ups
- pts-core: With make-download-cache option only check for downloads on installed tests
- pts-core: Support "all" as an argument with remove-installed-test option
- pts-core: Improve object security
- pts-core: Improve desktop responsiveness of GTK2 GUI
- pts-core: Add automated Internet connectivity check on start
- pts-core: More efficient paths for determining reference system comparisons
- pts-core: Add __pre_test_download, __interim_test_download, and __post_test_download hooks to the PTS module API
- pts-core: Start work on GTK2 GUI interface for during the test installation and run processes
- pts-core: Rewrite web-browser launching code
- pts-core: Rework list-tests option code
- pts-core: Build select caches on start-up when running GTK2 GUI
- pts-core: New bash completion support
- pts-core: Support using the update_checker module from the GTK2 GUI for notifying of new PTS releases
- pts-core: Add initial bisect module to allow users to hook PTS into any Git repository for bisecting with external script
- pts-core: Use pts_test_profile within pts-includes-run
- pts-core: Check for allow_url_fopen php.ini setting if it's disabled rather than letting it produce a warning
- pts-core: Rename the pts_standard_display_mode to pts_basic_display_mode
- pts-core: When running in batch mode or with debug-run, at end of testing report any tests/options that failed to properly run
- pts-core: When tests are not installed when called in run-test option, prompt the user to automatically install the tests from there
- pts-core: At end of the install process, list all tests that failed to properly install
- pts-core: Various bug fixes
- phodevi: Fend off possible infinite loop on OpenBSD when phodevi_parser::read_sysctl() fails
- phodevi: Fix for reporting fglrx version string with newer versions of ATI Catalyst Linux driver
- phodevi: Improve motherboard detection on BSD operating systems
- phodevi: Improve CPU temperature detection on BSD operating systems
- tandem_Xml: XML parsing fix when reading
- pts: Fix for prey test profile
- pts: Update openal external dependency in fedora-packages.xml
- pts: Update compress-7zip test profile against p7zip 9.04 version
- pts: Update sqlite test profile against SQLite 3.6.19
- pts: Update wine-unstable base test profile against Wine 1.1.32
- pts: Update graphics-magick test profile against GraphicsMagick 1.3.7
- pts: Use fedora-packages.xml external dependencies on CentOS instead of centos-packages.xml that is the same but less maintained
- pts: Upstream request regarding cmake change for minion test profile

Phoronix Test Suite 2.2 Alpha 4
October 12, 2009

- pts-core: Fix regression that caused running/installing suites to fail from Alpha 3
- pts-core: Add support for stopping/skipping the current test by touching ~/.phoronix-test-suite/skip-test during the process
- pts-core: Remember radio and check button selects in menu on GTK2 GUI for the duration of the program
- pts-core: Optimize a few common code paths
- pts-core: Improve detection of Catalyst driver with newer releases
- pts-core: Don't go awry when computing standard deviation on an array that may also contain errors from a test run due to parsing failure
- pts-core: Fix fatal error message with GtkWindow missing
- pts-core: Fend off possible warnings on pts_test_option
- pts-core: Rename pts_batch_display_mode to pts_concise_display_mode
- pts-core: Fix in subtitle reporting on graphs for tests with no options under certain conditions where it reported ":"
- pts-core: When using rename-identifier-in-result-file option, rename the benchmark-logs and system-logs folders too for that identifier
- pts-core: When using remove-from-result-file, remove the benchmark-logs and system-logs folders for that identifier if present
- pts-core: Move system parsing functions out of pts-core and into phodevi_parser object in Phodevi
- pts-core: Support exporting some functions from pts-core to the parse-results process during test runs
- phodevi: Don't try using aticonfig when reading the AMDPCSDB on an unsupported graphics adapter, use the internal parser instead
- phodevi: Support reading OpenGL vendor string
- tandem_Xml: Optimize the XML string parser
- pts: Updates to hpcc test profile
- pts: Add initial opencl-ati test profile
- pts: Add python-boost-development, boost-thread-development, and python-numpy external dependencies
- pts: Add initial pyopencl test profile
- pts: Update Nexuiz test profile against Nexuiz 2.5.2
- pts: Update netbook test suite
- pts: Change over relevant test profiles to using pts_report_numeric_result() rather than print statements

Phoronix Test Suite 2.2 Alpha 3
September 28, 2009

- pts-core: Add support for HTTP communication through a proxy server
- pts-core: Add network-setup option for configuring network settings
- pts-core: Add NoNetworkCommunication boolean option to user-config.xml for disabling network access
- pts-core: Set timeout on HTTP connections to eight seconds
- pts-core: Fix for results not showing up if renaming the first result identifier using rename-identifier-in-result-file option
- pts-core: Fixes for FreeBSD 8.0 support
- pts-core: Fix for reporting "Already Installed" tests with install option
- pts-core: Better handle toggle_screensaver module when xdg-screensaver is missing
- pts-core: Greater fallback disk support, should help with some BSD operating systems
- pts-core: Add support for pts-core to dynamically increase a test's run count if the standard deviation exceeds a predefined threshold
- pts-core: Add DynamicRunCount and LimitDynamicToTestLength for controlling the statistics / standard deviation settings
- pts-core: Add StandardDeviationThreshold to user-config.xml for setting the standard deviation threshold percentage
- pts-core: Better support floating values within the preferences window of the GTK2 GUI
- pts-core: Add batch-install option if only to run the install process in the batch mode
- pts-core: Updates to PTS user module architecture
- pts-core: Support for having SH modules in the module local directory
- pts-core: Standardize PTS test comparison hashes
- pts-core: If a test needs RequiresRoot access and is running in batch mode and is not root, do not try to run the test
- pts-core: When generating new graphs for a result file, delete any old files first
- pts-core: Add ExportResultsTo as a user-config.xml option for hooking in external scripts for statistics support
- pts-core: Support setting the result proportion for a test profile from writing it to ~/pts-results-proportion during the testing process
- phodevi: Improve hardware detection for OpenSolaris SPARC systems using DDU information
- phodevi: Improve motherboard detection for Sun SPARC systems
- phodevi: Start new reporting infrastructure
- tandem_Xml: Add addXmlObjectFromReader() function for improved performance
- tandem_Xml: Add saveXMLFile() function
- pts: Fixes for compress-lzma and compress-pbzip2 test profiles with name change of compressfile
- pts: Fix for warsow test profile results not showing up on Linux
- pts: Support using aptitude for external dependencies when available on Ubuntu/Debian
- pts: Update minion test profile against Minion 0.9
- pts: Add OpenMPI to external dependencies as openmpi-development
- pts: Add ATLAS math to external dependencies as atlas-development
- pts: Add hpcc test profile for HPC Challenge

Phoronix Test Suite 2.2 Alpha 2
September 15, 2009

- pts-core: Add display mode support to download-test-files option
- pts-core: Handle pts_storage_object on failure correctly to avoid fatal error during testing process
- pts-core: Export $DEBUG_REAL_HOME environmental variable to all test scripts
- pts-core: Fix reporting of test notes at end of testing process
- pts-core: Release the run_lock when a test is running and returns due to an error
- pts-core: With the batch display mode, report the "estimated time remaining" at the start of each run
- pts-core: Add change-results-display-order option for changing the ordering of the results within the results viewer / graphs
- pts-core: Add rename-identifier-in-result-file option for changing the name of a test identifier within a results file
- pts-core: With GTK2 GUI don't get to the point of emiting a fatal error over GtkWindow
- pts-core: No longer support having test result XML files outside of the test-results/ directory where an absolute path was supplied to PTS
- pts-core: Add support for performing argument / start-up checks for a PTS option prior to ever calling the run function
- pts-core: When a test installation fails, save the install.log to install-failed.log
- pts-core: Add option in batch-setup for still prompting for test option selection when running in batch mode
- phodevi: Don't repeat the system vendor in motherboard string if it's also present in the product string
- phodevi: Don't report hard drive bytes on Mac OS X 10.6 when reading the capacity of a disk
- pts: Update Fedora external dependencies XML for blas-development and popt
- pts: Update OpenSolaris external dependencies XML
- pts: Update warsow test profile against Warsow 0.5
- pts: Update gnupg test profile against GnuPG 2.0.11
- pts: Update build-imagemagick test profile against ImageMagick 6.5.5
- pts: Use OpenMP support in stream test profile
- pts: Build libpng within graphics-magick test profile so it will run on Mac OS X 10.6
- pts: Add aio-stress test profile

Phoronix Test Suite 2.2 Alpha 1
August 27, 2009

- pts-core: Begin writing the results XML during the test run process earlier
- pts-core: When a test is running, save the active pts_test_run_manager to a pts_storage_object on the disk
- pts-core: Backup the XML results when a test is running to active.xml in the result's folder
- pts-core: Add recover-run option for recovering the test results / finish running the tests when a crash had occured
- pts-core: Fix bug with the result identifier being empty when using list-saved-results / list-results
- pts-core: Setup a pt2so object for storing some pts-core data using a pts_storage_object, move some of these items out of user-config.xml
- pts-core: Move out the modules that are loaded by a default to a separate file
- pts-core: Add support for writing to different display modes / abstract out the output process from the testing process
- pts-core: Drop the email_results module since using Phoromatic is a better solution
- pts-core: Add a display mode for batch runs or when DefaultDisplayMode in the user-config.xml is set to "BATCH"
- pts-core: Add support to allow anonymous usage reporting / statistics information
- pts-core: Add "free" virtual suite that contains all tests that are of a FREE type for its license
- pts-core: Multiple download cache directories can be specified in the user-config.xml and PTS_DOWNLOAD_CACHE environmental variable when delimited by colon
- pts-core: Move relevant functions that touch hardware settings / properties out of pts-core and now using the phodevi::set_property() framework
- pts-core: Fix check buttons for boolean values in the GTK2 GUI preferences window
- pts-core: Allow the display mode to be set using PTS_DISPLAY_MODE environmental variable
- pts-core: Add display mode support when a test is being installed
- pts-core: Add ArchitectureSpecific tag support to distro-xml files for specifying packages to install on only select architectures for the OS
- pts-core: Rework test timers to avoid false results on crashes
- pts-core: Improve estimated run time reporting when running tests
- pts-core: Handle the results identifer and saved results file name within the pts_test_run_manager
- pts-core: Add experimental support for weighted suites / results
- pts-core: Add "cache share" support for test profiles and the AllowCacheShare XML tag to expose this functionality
- pts-core: Add "Test All Options" to test menu when prompting for test option selection
- pts-core: Don't uselessly prompt a user for test options when trying to run an unsupported test
- pts-core: Allow the user to select multiple test options when running a test by delimiting each option with a comma
- pts-core: When using analyze-batch, support using bar graphs on comparisons where relevant, instead of just line graphs
- pts-core: Don't repeat reporting the system hardware/software for the same results identifier in a results file
- pts-core: Smarter detection when dealing with user trying to input a results identifier that is already found in the results file
- pts-core: Launch web browser (instead of DashCode) on Mac OS X for analyze-batch and merge-results command
- pts-core: Add remove-from-result-file option for removing a set of test results from a saved file
- pts-core: Don't report the elapsed time for a test run when it's using a cache share or no valid results were gathered
- pts-core: Stop forcing the time-zone to UTC
- phodevi: Be more non-RandR friendly in phodevi_gpu::gpu_available_modes()
- phodevi: Add infrastructure so that properties / settings can be set using phodevi::set_property()
- phodevi: Add CPU usage monitoring support for Mac OS X
- phodevi: Improve memory capacity detection on newer Apple systems
- phodevi: Don't repeat the memory reporting in GPU string on Mac OS X
- phodevi: Add sensor for reading system's uptime through system object
- tandem_XmlWriter: Don't add the "Generated" timestamp each time that getXML() is called, since it's now being hit multiple times with each test run
- pts_Graph: Properly order keys in the header when rendering
- pts: Add glibc-development as an external dependency for providing GNU C library development files
- pts: Add python external dependency for Python language
- pts: Add stream test profile for Stream memory benchmark
- pts: Add tachyon test profile for Tachyon Parallel Ray-Tracing benchmark
- pts: Add python-environment base test profile for placing external dependency on python and reporting Python version to test notes
- pts: Add pybench test profile for Python's PyBench
- pts: Add tscp test profile for the chess benchmark
- pts: Add chess test suite for test profiles that use chess AI
- pts: Add john-the-ripper test profile for John The Ripper password cracker benchmark
- pts: Add geekbench test profile for Primate Labs Geekbench benchmark
- pts: Add npb test profile for NASA NAS Parallel Benchmark
- pts: Drop test length option from ramspeed test profile, default to 10
- pts: Add Mac OS X support for openarena test profile
- pts: Add Mac OS X support to warsow test profile
- pts: In nexuiz test profile change from demos/bench1 to demos/demo2 for now
- pts: In video-cpu-usage test profile add support to use CoreVideo vo adapter on Mac OS X
- pts: Update graphics-magick test profile against GraphicsMagick 1.3.6