Phoronix Test Suite

An Automated, Open-Source Testing Framework

Latest Release: Phoronix Test Suite 5.0.1

Home Downloads Features Documentation Contact OpenBenchmarking.org Forum Support

Phoronix Test Suite 2.2 Changes

Phoronix Test Suite 2.2.0 "Bardu"
November 16, 2009

- pts-core: Various code path optimizations and cleaning
- pts-core: Add dump-system-sensors option to display a readout of all supported Phodevi sensors and their values and print unsupported sensors
- pts-core: Fix TEST_RESULTS_NAME environmental variable not working
- pts-core: Fix network-setup network testing check actually use the proxy, hook in the HTTP proxy override properly
- pts-core: Clean-up download cache directories code
- pts-core: Add "installed-tests" virtual suite of tests that are currently installed
- pts-core: On GTK2 GUI Don't show buttons/header for reference system comparisons for results when there are actually no results to compare
- pts-core: Small OpenSolaris 2010.02 support updates
- pts-core: Various small bug fixes
- phodevi: Support for the GPU fan-speed sensor with ATI graphics driver
- phodevi: Add support for reading the stock GPU engine/memory clock when using the open-source Radeon driver
- phodevi: Fixed detection of the xf86-video-radeonhd display driver
- phodevi: Fix throwing SUSE Linux 11.2 into an infinite loop when running "X -version" on this OS, "Xorg -version" works fine instead
- pts: Update darwin test suite
- pts: Support using zypper to install external dependencies on SuSE/OpenSuSE since it can ignore missing / removed packages without failing
- documentation: Update documentation for Phoronix Test Suite 2.2

Phoronix Test Suite 2.2 Beta 3
November 9, 2009

- pts-core: Fix API breakage for installing dependencies from install-dependencies option call
- pts-core: Report the disk scheduler to the test notes area when a disk test is run
- pts-core: Add SelectSuitesOrTests tag to user-config.xml for selecting between tests / suites showing by default in GUI
- pts-core: Add initial analyze-result-file command for showing basic analytical / stats for a given result file
- pts-core: Support showing information from info command when supplying a virtual suite
- pts-core: Don't contain tests that are of license RETAIL or RESTRICTED in the "all" virtual suite
- pts-core: Fix where recover-run would not recognize the result file if there was not at least one successful one previously
- pts-core: When using recover-run, put back in batch mode state if the test was originally run in batch mode
- pts-core: Validate test installations earlier in the test run process
- pts-core: When clicking "View Results" from GTK2 GUI, generate graphs if needed
- pts-core: Fix regression that broke reference comparisons and removing identifiers from result files
- phodevi: Add support for GPU fan speed monitoring (currently only implemented for NVIDIA's binary display driver)
- phodevi: Add property to disk object for reading the scheduler in use
- phodevi: DMI parsing support on Linux through /sys/class/dmi/id/ interface, for now implement motherboard way
- phodevi: Detect and support reading the X.Org DDX driver module version for Cirrus and other obscure drivers/hardware
- pts: Update for Fedora external dependencies XML
- pts: Start adding new reference comparison systems with runs against Phoronix Test Suite 2.2

Phoronix Test Suite 2.2 Beta 2
November 3, 2009

- pts-core: With toggle_screensaver module, disable the display(s) from sleeping when on AC power in GNOME during testing
- pts-core: Clean-up more code used during the test execution process
- pts-core: Rename SaveSystemDetails user-config.xml option to SaveSystemLogs
- pts-core: Add option to archive a test's installation log to a sub-directory of the results file
- pts-core: Check for user-config-template.xml template in pts-core/static/
- pts-core: Add PTS Desktop Live user-config.xml template
- pts-core: Move txt file lists from pts-core/static/ to pts-core/static/lists/
- pts-core: Allow test options to be specified externally via PRESET_OPTIONS environmental variable (see commit for example)
- pts-core: Fix bug where from GTK2 GUI selecting to not save test results would prompt you from CLI to save results or not
- pts-core: Vastly re-work the pts-includes-run_options code
- pts-core: Move images to pts-core/static/images/
- pts-core: Create modules tag section in user-config.xml, move default modules to load into the LoadModules tag instead of text file
- pts-core: Start adding user-interface options to the user-config.xml
- pts-core: Updates for library / Phoromatic server usage
- pts-core: Don't ask user whether to try again on a failed test download when in automated/batch mode
- pts-core: When installing external dependencies fails, continue installing just the tests where the dependencies are satisfied]
- pts-core: Add __event_user_error hook to PTS module API
- phoromatic: Prevent overloading the Phoromatic server if the client module keeps trying to communicate too many times per minute
- phoromatic: Support for reporting system/PTS warnings to Phoromatic server
- phoromatic: Support for setting the default system title on the Phoromatic server as the test system's hostname
- phoromatic: Report new system software / hardware for test system to Phoromatic server when appropriate
- phoromatic: Add support for uploading Phoromatic-sanctioned test results to Phoronix Global, if Phoromatic server tells it to
- phoromatic: Add support for Phoromatic server to tell client whether or not to archive the test results locally upon completion
- phodevi: Improve phodevi smart caching, bring actual caching into pts-core and place the cache_store in the core.pt2so
- phodevi: Fix possible space when getting resolutions from xrandr
- phodevi: Don't report GPU memory clock in GPU string when GPU memory clock is 0MHz (NVIDIA reports this for at least some IGPs)
- pts: Add workstation test suite
- pts: Add disk-ssd test suite
- pts: Revamp server test suite
- documentation: Add a basic reviewer's guide

Phoronix Test Suite 2.2 Beta 1
October 27, 2009

- pts-core: Fix for running result files that have forced AA/AF levels and then fail to merge
- pts-core: On missing timezone setting, set to UTC if the PHP version is 5.3.0 or newer, emits a warning otherwise
- pts-core: Rename change-results-display-order option command to reorder-result-file
- pts-core: Add edit-result-file meta option
- pts-core: Add MinimumLength tag to XML test profile specification
- pts-core: Add MaximumLength tag to XML test profile specification
- pts-core: Few fixes/changes to test execution code
- pts-core: When dealing with the stats support for validating test results, don't increase the run count prematurely
- pts-core: Fend off divide by zero errors in the standard deviation code
- pts-core: Improve performance of PTS assignment functions
- pts-core: Don't remove the PTS run_lock when running a secondary instance of PTS
- pts-core: Add debug-run option to provide more run-time information to assist test profile writers in debugging / verifying profiles
- pts-core: Drop central pts-core debugging code since it was very rarely used and not too helpful
- pts-core: Add in additional LC_NUMERIC override to eliminate non-US decimal problems with test results
- pts-core: Various test run code clean-ups
- pts-core: With make-download-cache option only check for downloads on installed tests
- pts-core: Support "all" as an argument with remove-installed-test option
- pts-core: Improve object security
- pts-core: Improve desktop responsiveness of GTK2 GUI
- pts-core: Add automated Internet connectivity check on start
- pts-core: More efficient paths for determining reference system comparisons
- pts-core: Add __pre_test_download, __interim_test_download, and __post_test_download hooks to the PTS module API
- pts-core: Start work on GTK2 GUI interface for during the test installation and run processes
- pts-core: Rewrite web-browser launching code
- pts-core: Rework list-tests option code
- pts-core: Build select caches on start-up when running GTK2 GUI
- pts-core: New bash completion support
- pts-core: Support using the update_checker module from the GTK2 GUI for notifying of new PTS releases
- pts-core: Add initial bisect module to allow users to hook PTS into any Git repository for bisecting with external script
- pts-core: Use pts_test_profile within pts-includes-run
- pts-core: Check for allow_url_fopen php.ini setting if it's disabled rather than letting it produce a warning
- pts-core: Rename the pts_standard_display_mode to pts_basic_display_mode
- pts-core: When running in batch mode or with debug-run, at end of testing report any tests/options that failed to properly run
- pts-core: When tests are not installed when called in run-test option, prompt the user to automatically install the tests from there
- pts-core: At end of the install process, list all tests that failed to properly install
- pts-core: Various bug fixes
- phodevi: Fend off possible infinite loop on OpenBSD when phodevi_parser::read_sysctl() fails
- phodevi: Fix for reporting fglrx version string with newer versions of ATI Catalyst Linux driver
- phodevi: Improve motherboard detection on BSD operating systems
- phodevi: Improve CPU temperature detection on BSD operating systems
- tandem_Xml: XML parsing fix when reading
- pts: Fix for prey test profile
- pts: Update openal external dependency in fedora-packages.xml
- pts: Update compress-7zip test profile against p7zip 9.04 version
- pts: Update sqlite test profile against SQLite 3.6.19
- pts: Update wine-unstable base test profile against Wine 1.1.32
- pts: Update graphics-magick test profile against GraphicsMagick 1.3.7
- pts: Use fedora-packages.xml external dependencies on CentOS instead of centos-packages.xml that is the same but less maintained
- pts: Upstream request regarding cmake change for minion test profile

Phoronix Test Suite 2.2 Alpha 4
October 12, 2009

- pts-core: Fix regression that caused running/installing suites to fail from Alpha 3
- pts-core: Add support for stopping/skipping the current test by touching ~/.phoronix-test-suite/skip-test during the process
- pts-core: Remember radio and check button selects in menu on GTK2 GUI for the duration of the program
- pts-core: Optimize a few common code paths
- pts-core: Improve detection of Catalyst driver with newer releases
- pts-core: Don't go awry when computing standard deviation on an array that may also contain errors from a test run due to parsing failure
- pts-core: Fix fatal error message with GtkWindow missing
- pts-core: Fend off possible warnings on pts_test_option
- pts-core: Rename pts_batch_display_mode to pts_concise_display_mode
- pts-core: Fix in subtitle reporting on graphs for tests with no options under certain conditions where it reported ":"
- pts-core: When using rename-identifier-in-result-file option, rename the benchmark-logs and system-logs folders too for that identifier
- pts-core: When using remove-from-result-file, remove the benchmark-logs and system-logs folders for that identifier if present
- pts-core: Move system parsing functions out of pts-core and into phodevi_parser object in Phodevi
- pts-core: Support exporting some functions from pts-core to the parse-results process during test runs
- phodevi: Don't try using aticonfig when reading the AMDPCSDB on an unsupported graphics adapter, use the internal parser instead
- phodevi: Support reading OpenGL vendor string
- tandem_Xml: Optimize the XML string parser
- pts: Updates to hpcc test profile
- pts: Add initial opencl-ati test profile
- pts: Add python-boost-development, boost-thread-development, and python-numpy external dependencies
- pts: Add initial pyopencl test profile
- pts: Update Nexuiz test profile against Nexuiz 2.5.2
- pts: Update netbook test suite
- pts: Change over relevant test profiles to using pts_report_numeric_result() rather than print statements

Phoronix Test Suite 2.2 Alpha 3
September 28, 2009

- pts-core: Add support for HTTP communication through a proxy server
- pts-core: Add network-setup option for configuring network settings
- pts-core: Add NoNetworkCommunication boolean option to user-config.xml for disabling network access
- pts-core: Set timeout on HTTP connections to eight seconds
- pts-core: Fix for results not showing up if renaming the first result identifier using rename-identifier-in-result-file option
- pts-core: Fixes for FreeBSD 8.0 support
- pts-core: Fix for reporting "Already Installed" tests with install option
- pts-core: Better handle toggle_screensaver module when xdg-screensaver is missing
- pts-core: Greater fallback disk support, should help with some BSD operating systems
- pts-core: Add support for pts-core to dynamically increase a test's run count if the standard deviation exceeds a predefined threshold
- pts-core: Add DynamicRunCount and LimitDynamicToTestLength for controlling the statistics / standard deviation settings
- pts-core: Add StandardDeviationThreshold to user-config.xml for setting the standard deviation threshold percentage
- pts-core: Better support floating values within the preferences window of the GTK2 GUI
- pts-core: Add batch-install option if only to run the install process in the batch mode
- pts-core: Updates to PTS user module architecture
- pts-core: Support for having SH modules in the module local directory
- pts-core: Standardize PTS test comparison hashes
- pts-core: If a test needs RequiresRoot access and is running in batch mode and is not root, do not try to run the test
- pts-core: When generating new graphs for a result file, delete any old files first
- pts-core: Add ExportResultsTo as a user-config.xml option for hooking in external scripts for statistics support
- pts-core: Support setting the result proportion for a test profile from writing it to ~/pts-results-proportion during the testing process
- phodevi: Improve hardware detection for OpenSolaris SPARC systems using DDU information
- phodevi: Improve motherboard detection for Sun SPARC systems
- phodevi: Start new reporting infrastructure
- tandem_Xml: Add addXmlObjectFromReader() function for improved performance
- tandem_Xml: Add saveXMLFile() function
- pts: Fixes for compress-lzma and compress-pbzip2 test profiles with name change of compressfile
- pts: Fix for warsow test profile results not showing up on Linux
- pts: Support using aptitude for external dependencies when available on Ubuntu/Debian
- pts: Update minion test profile against Minion 0.9
- pts: Add OpenMPI to external dependencies as openmpi-development
- pts: Add ATLAS math to external dependencies as atlas-development
- pts: Add hpcc test profile for HPC Challenge

Phoronix Test Suite 2.2 Alpha 2
September 15, 2009

- pts-core: Add display mode support to download-test-files option
- pts-core: Handle pts_storage_object on failure correctly to avoid fatal error during testing process
- pts-core: Export $DEBUG_REAL_HOME environmental variable to all test scripts
- pts-core: Fix reporting of test notes at end of testing process
- pts-core: Release the run_lock when a test is running and returns due to an error
- pts-core: With the batch display mode, report the "estimated time remaining" at the start of each run
- pts-core: Add change-results-display-order option for changing the ordering of the results within the results viewer / graphs
- pts-core: Add rename-identifier-in-result-file option for changing the name of a test identifier within a results file
- pts-core: With GTK2 GUI don't get to the point of emiting a fatal error over GtkWindow
- pts-core: No longer support having test result XML files outside of the test-results/ directory where an absolute path was supplied to PTS
- pts-core: Add support for performing argument / start-up checks for a PTS option prior to ever calling the run function
- pts-core: When a test installation fails, save the install.log to install-failed.log
- pts-core: Add option in batch-setup for still prompting for test option selection when running in batch mode
- phodevi: Don't repeat the system vendor in motherboard string if it's also present in the product string
- phodevi: Don't report hard drive bytes on Mac OS X 10.6 when reading the capacity of a disk
- pts: Update Fedora external dependencies XML for blas-development and popt
- pts: Update OpenSolaris external dependencies XML
- pts: Update warsow test profile against Warsow 0.5
- pts: Update gnupg test profile against GnuPG 2.0.11
- pts: Update build-imagemagick test profile against ImageMagick 6.5.5
- pts: Use OpenMP support in stream test profile
- pts: Build libpng within graphics-magick test profile so it will run on Mac OS X 10.6
- pts: Add aio-stress test profile

Phoronix Test Suite 2.2 Alpha 1
August 27, 2009

- pts-core: Begin writing the results XML during the test run process earlier
- pts-core: When a test is running, save the active pts_test_run_manager to a pts_storage_object on the disk
- pts-core: Backup the XML results when a test is running to active.xml in the result's folder
- pts-core: Add recover-run option for recovering the test results / finish running the tests when a crash had occured
- pts-core: Fix bug with the result identifier being empty when using list-saved-results / list-results
- pts-core: Setup a pt2so object for storing some pts-core data using a pts_storage_object, move some of these items out of user-config.xml
- pts-core: Move out the modules that are loaded by a default to a separate file
- pts-core: Add support for writing to different display modes / abstract out the output process from the testing process
- pts-core: Drop the email_results module since using Phoromatic is a better solution
- pts-core: Add a display mode for batch runs or when DefaultDisplayMode in the user-config.xml is set to "BATCH"
- pts-core: Add support to allow anonymous usage reporting / statistics information
- pts-core: Add "free" virtual suite that contains all tests that are of a FREE type for its license
- pts-core: Multiple download cache directories can be specified in the user-config.xml and PTS_DOWNLOAD_CACHE environmental variable when delimited by colon
- pts-core: Move relevant functions that touch hardware settings / properties out of pts-core and now using the phodevi::set_property() framework
- pts-core: Fix check buttons for boolean values in the GTK2 GUI preferences window
- pts-core: Allow the display mode to be set using PTS_DISPLAY_MODE environmental variable
- pts-core: Add display mode support when a test is being installed
- pts-core: Add ArchitectureSpecific tag support to distro-xml files for specifying packages to install on only select architectures for the OS
- pts-core: Rework test timers to avoid false results on crashes
- pts-core: Improve estimated run time reporting when running tests
- pts-core: Handle the results identifer and saved results file name within the pts_test_run_manager
- pts-core: Add experimental support for weighted suites / results
- pts-core: Add "cache share" support for test profiles and the AllowCacheShare XML tag to expose this functionality
- pts-core: Add "Test All Options" to test menu when prompting for test option selection
- pts-core: Don't uselessly prompt a user for test options when trying to run an unsupported test
- pts-core: Allow the user to select multiple test options when running a test by delimiting each option with a comma
- pts-core: When using analyze-batch, support using bar graphs on comparisons where relevant, instead of just line graphs
- pts-core: Don't repeat reporting the system hardware/software for the same results identifier in a results file
- pts-core: Smarter detection when dealing with user trying to input a results identifier that is already found in the results file
- pts-core: Launch web browser (instead of DashCode) on Mac OS X for analyze-batch and merge-results command
- pts-core: Add remove-from-result-file option for removing a set of test results from a saved file
- pts-core: Don't report the elapsed time for a test run when it's using a cache share or no valid results were gathered
- pts-core: Stop forcing the time-zone to UTC
- phodevi: Be more non-RandR friendly in phodevi_gpu::gpu_available_modes()
- phodevi: Add infrastructure so that properties / settings can be set using phodevi::set_property()
- phodevi: Add CPU usage monitoring support for Mac OS X
- phodevi: Improve memory capacity detection on newer Apple systems
- phodevi: Don't repeat the memory reporting in GPU string on Mac OS X
- phodevi: Add sensor for reading system's uptime through system object
- tandem_XmlWriter: Don't add the "Generated" timestamp each time that getXML() is called, since it's now being hit multiple times with each test run
- pts_Graph: Properly order keys in the header when rendering
- pts: Add glibc-development as an external dependency for providing GNU C library development files
- pts: Add python external dependency for Python language
- pts: Add stream test profile for Stream memory benchmark
- pts: Add tachyon test profile for Tachyon Parallel Ray-Tracing benchmark
- pts: Add python-environment base test profile for placing external dependency on python and reporting Python version to test notes
- pts: Add pybench test profile for Python's PyBench
- pts: Add tscp test profile for the chess benchmark
- pts: Add chess test suite for test profiles that use chess AI
- pts: Add john-the-ripper test profile for John The Ripper password cracker benchmark
- pts: Add geekbench test profile for Primate Labs Geekbench benchmark
- pts: Add npb test profile for NASA NAS Parallel Benchmark
- pts: Drop test length option from ramspeed test profile, default to 10
- pts: Add Mac OS X support for openarena test profile
- pts: Add Mac OS X support to warsow test profile
- pts: In nexuiz test profile change from demos/bench1 to demos/demo2 for now
- pts: In video-cpu-usage test profile add support to use CoreVideo vo adapter on Mac OS X
- pts: Update graphics-magick test profile against GraphicsMagick 1.3.6