Phoronix Test Suite

An Automated, Open-Source Testing Framework

Latest Release: Phoronix Test Suite 5.0.1

Home Downloads Features Documentation Contact OpenBenchmarking.org Forum Support

Phoronix Test Suite 1.4 Changes

Phoronix Test Suite 1.4.2
November 13, 2008

- pts-core: Fix bug in xorg_ddx_driver_info() that could have caused some driver versions to be misreported
- pts-core: Fix for possible regression (depending upon DRI driver) in IS_*_GRAPHICS setup
- pts-core: Temperature reporting fix in graphics_processor_temperature() that made it look like sometimes there was no sensor
- pts-core: Add "00" to HAL $remove_words
- pts-core: Add "none" to HAL $remove_words
- pts: Fix openal-development external dependency for ubuntu-packages.xml due to package name change in Ubuntu 8.10
- pts: Set file size to 256MB with compress-lzma test profile

Phoronix Test Suite 1.4.1
November 8, 2008

- pts-core: Add support for reading X.Org module / driver versions using read_xorg_module_version()
- pts-core: Where applicable, report X.Org DDX driver and version to software information string
- pts-core: Fix RAM detection on Dell servers where "<OUT OF SPEC>" was string and then failed the XML to render (reported by pjwelsh)
- pts-core: Add "not specified" to HAL $remove_words
- pts-core: Revert trackerd from $check_process until better method is determined
- pts-core: Fix read warnings in SELinux detection
- pts-core: Improve *BSD operating system functions
- pts-core: Add support for OpenBSD 4.4
- pts: Add gcc-g++ to external dependencies support for Cent OS and Fedora (reported by pjwelsh)
- pts: Report 2D EXA/XAA acceleration mode in the test notes when running gtkperf test

Phoronix Test Suite 1.4.0
November 3, 2008

- pts-core: Improve RAM model/speed/type identification
- pts-core: Fix showing Phoronix Global URLs in browser (reported by Marox)
- pts-core: Add Tracker (trackerd) to $check_processes
- pts-core: Add support for checking if SELinux is enabled/running
- pts-core: Improve reporting of system RAM information when not all DMI data is present
- pts-core: Add -L argument to curl to follow download links
- pts-core: Add support in read_osx_system_profiler() to read multiple objects
- pts-core: Add support for reading detailed RAM information on Mac OS X
- pts-core: Add support for reading detailed RAM information on OpenSolaris using DMI
- pts-core: Fix bug that caused installation of multiple Cascading Test Profiles to only go one level deep
- pts-core: Rework code to pts_hw_string() and pts_sw_string()
- pts-core: Add timed kill infrastructure to kill a process from a test script by calling $TIMED_KILL <process_name> <time_in_seconds>
- pts: Switch tandem-xml test profile to using $PHP_BIN instead of calling php (thus allows it to run where PHP isn't part of their path, such as Solaris)
- pts: Switch pts-graph test profile to using $PHP_BIN instead of calling php (thus allows it to run where PHP isn't part of their path, such as Solaris)
- pts: Add additional run-time tests for x11perf test profile
- pts: Update insertion count to 12,500 for sqlite test profile
- pts: Update unigine-tropics test profile against Tropics 1.1 from Unigine Corp
- pts: Update unigine-sanctuary test profile against Sanctuary 2.1 from Unigine Corp
- pts: Drop test profile named wine for running WINE git code since it looks like it's never used anymore and poorly named
- pts: Add wine-unstable base test profile that provides WINE 1.1.7
- pts: Add wine-humus-tests base test profile that extends wine-unstable and provides functionality for Humus.name tests
- pts: Add wine-vf2 test for a WINE-driven OpenGL Volumetric Fogging test
- pts: Add wine-cloth test that uses WINE
- pts: Add wine-domino test that uses WINE
- pts: Add wine-fire2 test that uses WINE
- pts: Add wine-hdr test that uses WINE
- pts: Add wine-metaballs test that uses WINE
- pts: Add wine-water test that uses WINE
- pts: Add wine-graphics test suite that contains all WINE graphics tests
- pts: Drop system-memory test suite since it's basically the same as the memory test suite
- pts_Graph: Support resizing the font size in write_text_center() if it's too large to fit within the bounding box
- pts: Add Qt4 development files to external dependencies through qt4-development generic name
- pts: Ensure all *-devel generic names are now named *-development
- pts: Add qgears2 test profile for a Qt graphics test that can test XRender and OpenGL
- pts: Add xrender test suite for tests that use the X11 RENDER extension
- update_checker: Fix version recognition / comparison
- documentation: Bring all documentation up-to-date with Orkdal

Phoronix Test Suite 1.4.0 Beta 2
October 27, 2008

- pts-core: Only call a PTS SH module if the command being called is in pts_module_processes() but no pts_module_events() support
- pts-core: Add result-info option for showing basic information about a saved result file
- pts-core: Improve motherboard detection for vendors that report a version string of "1234567890" (such as Super Micro)
- pts-core: Move timer-start.sh and timer-stop.sh from pts-core/scripts/ to pts-core/test-libraries/
- pts-core: Rename pts-core/pts-results-viewer/ to pts-core/results-viewer/
- pts-core: Move some abstracted shell commands to new pts-functions_shell.php file
- pts-core: Add pts_move_file() function for moving a file that abstracts the shell_exec command
- pts-core: Add pts_extract_file() for extracting a tar file that abstracts the shell_exec command
- pts-core: Add pts_run_shell_script() for extracting a sh script that abstracts the shell_exec command
- pts-core: Add base/ support to locally stored test profiles and resources
- pts-core: Create TEST_ENV_DIR and SAVE_RESULTS_DIR directories on start-up
- pts-core: Copy the Xorg.0.log when saving system details
- pts-core: Output dmesg when saving system details
- pts: Update build-php php-5.2.5 download links
- pts: Add new mirrors for lzma-4.32.6.tar.gz in compress-lzma test profile since current ones have been removed

Phoronix Test Suite 1.4.0 Beta 1
October 23, 2008

- pts-core: Improve detection of graphics cards using common open-source drivers
- pts-core: Fix regression in pts_trim_double() that would return an array instead of float-formatted string
- pts-core: Add support for detecting php binary at /usr/php/bin/php
- pts-core: Improve OpenSolaris hardware detection with motherboard and processor
- pts-core: Make micro-timer framework use $PHP_BIN instead of php so that it can run on OpenSolaris
- pts-core: Fix bug in pts_contained_tests() that caused Phoronix Global IDs to not be detected
- pts-core: Track motherboard, processor, OS, and compiler version when a test is installed, and if any one ends up getting changed, force re-install of that test
- pts-core: Unify pts_test_generate_install_xml() and pts_test_refresh_install_xml() into pts_test_refresh_install_xml()
- pts-core: Don't report "Test Configuration" header for tests where there are no configuration options
- pts-core: Add and implement pts_test_result() class
- pts-core: Delete pts-results-description, pts-test-version, and pts-results-scale after reading them
- pts-core: Add pts_add_test_note() function for reporting items to list in the test notes of saved results
- pts-core: Allow test profiles to append a test note to a results file by writing the information to pts-test-note in the test installation directory
- pts-core: Change directory to the test installation directory before running post.sh / post.php
- pts-core: Unify running pre/post/parse-results script running with pts_call_test_script()
- pts-core: Support pts_call_test_script() using Cascading Test Profiles so that extended pre/post/parse-results scripts can be run
- pts-core: Move ExternalDependencies tag location from TestInformation to Test Profile block (This breaks compatibility with earlier test profiles!)
- pts-core: Start reordering and optimizing pts-functions-run.php code
- pts-core: Allow select objects to be exposed to PTS modules when calling them, by passing them as the second argument in pts_module_call()
- pts-core: When removing a saved result, remove the SVG files too (just not PNG)
- pts-core: Support for reading HDD model on Mac OS X (currently SATA only and limited to a single drive)
- pts-core: Rework downloads.xml code to use pts_objects_test_downloads() with pts_test_file_download() class
- pts-core: Clean-up test install code and move External Dependencies code to pts-functions-install_dependencies.php
- pts-core: Pass select variables to PTS modules and describe the functionality in pts_module_interface
- pts-core: Support for running a test from a PTS graph file that's rendered as an SVG and then the user will be prompted with available choices
- pts-core: Add support for reading DMI information from dmidecode where available through the read_dmidecode() function
- pts-core: Add Linux support for reading DIMM speed, capacity, and bank information through DMI
- pts-core: Add __event_global_upload to PTS modules support for calling this function and passing the Phoronix Global URL when results have been uploaded
- pts: Update espeak test profile with eSpeak engine 1.39
- pts: Change run-count for gtkperf test profile from 2000 to 5000
- pts: Drop etc/ folder full of XML template files
- pts: Fix compress-* test profiles with the proper 2GB file size
- pts: OpenSolaris External Dependencies XML was named opensolaris-package.xml when it should be named opensolaris-packages.xml, now fixed
- pts: Update all test profiles as to whether they support Solaris/OpenSolaris
- pts: Add nevada test suite for key OpenSolaris tests
- pts: Add java-environment base test that provides the java external dependency and reports Java version to test notes
- pts: Update the location of the ExternalDependencies tag in all relevant test profiles
- pts: Set TimesToRun to 5 in timed-audio-encode base test
- pts: Add bork test profile for Java Bork File Encrypter test
- pts: Set TimesToRun to 3 for timed-build instead of 1
- pts_Graph: Write Y-axis title horizontally along the top of the graph instead of vertically
- pts_Graph: Round all dimensions and coordinates when using the SVG renderer

Phoronix Test Suite 1.4.0 Alpha 3
October 14, 2008

- pts-core: Fix regression that broke compatibility to run phoronix-test-suite benchmark <Phoronix Global ID>
- pts-core: Embed hidden information in the SVG graph files that identify the test, GlobalID, and user, to provide new end-user options in the future
- pts-core: Don't complain about test mismatch in merge function if using a Global ID
- pts-core: Prepatory work for SVG-driven test execution / identification
- pts-core: Add support for reading hard disk model number and true disk capacity
- pts-core: Fix CPU frequency detection for Intel Xeon LV processors and potentially other CPU families
- pts-core: If PTS micro-timer is less than three seconds, return zero as the test likely failed
- pts-core: Add RunMode tag to suite XML and if it's set to "PCQS", enable IS_PCQS_MODE
- pts-core: Add RequiresCoreVersion tag if the test profile is only compatible with select versions of the Phoronix Test Suite
- pts-core: Keep a running average how long it takes to run a given test. This is stored in the AverageRunTime tag within the pts-install.xml
- pts-core: Show average run-time with info <test> command
- pts-core: Don't pass $test_results as an argument to parse-results.sh or parse-results.php with tests using $LOG_FILE
- pts-core: Change format of list-test-usage
- pts-core: When running list-suites, don't show suites that aren't supported at all by the installed software/hardware and indicate only partially supported suites
- pts-core: Add average run-time to list-test-usage
- pts-core: Restructure the end-user run-time variables code and add support for $HOSTNAME
- pts-core: Dump end-user run-time variables with diagnostics option
- pts: Move more test profiles to using $LOG_FILE for result information
- pts: Update openssl test profile to OpenSSL version 0.9.8i
- pts: Update encode-mp3 test profile to LAME version 3.98.2
- pts: Update sqlite test profile to SQLite version 3.6.3
- pts: Update ramspeed test profile to RAMspeed version 2.5.2
- pts: With gnupg test profile use a 2GB file to encrypt instead of 1GB
- pts: With compress-gzip test profile use a 2GB file to compress instead of 1GB
- pts: With compress-lzma test profile use a 2GB file to compress instead of 1GB
- pts: With compress-pbzip2 test profile use a 2GB file to compress instead of 1GB
- pts: Many test profile updates
- pts: Mandatory major version bump for most test profiles
- pts_Graph: Set viewbox in SVG renderer
- documentation: Update profile writing how-to with latest pts-core features
- system_monitor: Add support for using the SVG renderer in pts_Graph

Phoronix Test Suite 1.4.0 Alpha 2
October 6, 2008

- pts-core: Improve AMD motherboard chipset detection
- pts-core: Add detection support for ATI/AMD Southbridge parts
- pts-core: Strip "Host Bridge" in pts_clean_information_string()
- pts-core: Strip "DRAM Controller" in pts_clean_information_string()
- pts-core: When rebuilding the modules-config.xml table, check if the module exists for each identifier otherwise remove it from the list
- pts-core: Add support for installing the Phoronix Certification & Qualification Suite with user agreement support
- pts-core: Add IS_FIRST_RUN_TODAY that is set to TRUE if this is the first time PTS has been run that day
- pts-core: Add pts_available_tests_array(), pts_installed_tests_array(), and pts_available_suites_array()
- pts-core: Use pts_test_tandem_XmlReader() in list-tests option
- pts-core: Fix regression that caused info <test> with a valid test identifier not to be recognized
- pts-core: Add PostRunMessage tag for showing the user a message after the test has run
- pts-core: Add PostInstallMessage tag for showing the user a message after the test has run
- pts-core: Add PreRunMessage tag to the suite specification for showing the user a message before testing begins (similar to PreRunMessage in tests)
- pts-core: Add PostRunMessage tag to the suite specification for showing the user a message before testing begins (similar to PostRunMessage in tests)
- pts-core: Add support for OS-specific test profiles through <OS_PREFIX>_<test> name but is accessible by just running <test>
- pts-core: Don't call wget directly to download files but use pts_download()
- pts-core: Set user-agent in pts_download() to PhoronixTestSuite/<PTS Codename>
- pts-core: Add support for using curl to download files instead of just wget
- pts-core: Clean-up SH module calls by using the new pts_sh_module_call()
- pts-core: Add pts_remove() as a recursive file/directory removing function
- pts-core: Fix fatal error with list-modules when a module unloads itself and is then re-included
- pts-core: Clean-up PHP module calls by using the new pts_php_module_call()
- pts-core: Add pts_module_call() as a meta function for pts_sh_module_call() and pts_php_module_call()
- pts-core: Move Phoronix Global tag creation code into pts_promt_user_tags()
- pts-core: Move test note creation code for saved results into pts_generate_test_notes()
- pts-core: Update upload-results to use new functions
- pts-core: Various minor improvements in pts-run-cmd.php
- pts-core: Move test option handling to using pts_test_options() and new pts_test_option() objects
- pts-core: Fix regression from Alpha 1 that caused no test information to be reported with list-test-usage option
- pts-core: Add ArgumentPrefix tag to test profile support as a replacement to using the ArgumentName tag
- pts-core: Add ArgumentPostfix tag that is a string to be post-fixed to the end of the argument, similar to ArgumentName/ArgumentPrefix
- pts-core: Add $LOG_FILE environmental variable during testing that the test profiles can write their log file / results to
- pts-core: Add option to user-config.xml in SaveBenchmarkLogs with setting to TRUE will backup any tests utilizing the log file ($LOG_FILE)
- pts-core: In user-config.xml, change LogVerboseSystemDetails tag to SaveSystemDetails
- pts-core: If SAVE_BENCHMARK_LOGS environmental variable is set, is the same as setting SaveBenchmarkLogs to TRUE for that run
- pts-core: If SAVE_SYSTEM_DETAILS environmental variable is set, is the same as setting SaveSystemDetails to TRUE for that run
- pts-core: Start on IS_PCQS_MODE define
- pts-core: Cleanup pts-functions_system.php and move most of it to pts-functions_system_hardware.php and pts-functions_system_software.php
- pts-core: Drop $OS from pts_env_variables() as instead test profile maintainers should use $OPERATING_SYSTEM
- pts-core: Switch back to using read_amd_pcsdb() with read_amd_pcsdb_direct_parser() fallback instead of just read_amd_pcsdb_direct_parser() in hopes AMD doesn't break aticonfig again
- pts-core: Add support for using hddtemp to read hard disk drive temperature on Linux
- pts-core: graphics_gpu_usage() returns -1 on failure
- pts-core: Add support for reading software and hardware information on Mac OS X
- pts-core: Use open command to launch web browser on Mac OS X
- pts-core: Add UnsupportedPlatforms tag to test profiles, which is evaluated the opposite of SupportedPlatforms
- pts-core: Add PlatformSpecific tag to download file XML for OS-specific files
- pts-core: Add ArchitectureSpecific tag to download file XML for hardware architecture-specific files
- pts-core: When RandR detection fails in xrandr_available_modes(), apply some logic to sort the stock modes that should work
- pts-core: Add in a micro-timer framework that is platform universal
- pts-core: Implement support for parse-results.sh
- pts-core: Default to 3 runs for each test
- pts-core: Reformat list-tests for better alignment and not showing redundant information
- pts-core: Add remove-installed-test to remove a test installation
- pts-core: Add a index.html forwarder to each result file and have the launch browser script open that, since on some systems .xml opens up in an editor instead of browser by default
- pts-core: When installing a suite and a test fails to download a file due to an MD5 mismatch, continue on with next test instead of exiting
- pts-core: Avoid over-using pts_exit() and replace with improved logic
- pts-core: Add support for backing up downloads to the cache from base tests being extended
- pts: Move pcqs tests out of tree
- pts: Switch all relevant test profiles using ArgumentName to using ArgumentPrefix
- pts: Move the byte test profile to using the new $LOG_FILE capability for an example and for testing
- pts: Move over all relevant tests to using the new universal micro-timer functionality
- pts: Add darwin test suite for the main Mac OS X tests
- pts: Add mac-packages.xml that contains the location of the OpenAL framework since it ships by default with OS X so suppress the warning message
- pts_Graph: Provide experimental SVG rendering support for graphing objects, more tweaking, testing, and improvements still needed
- update_checker: Move code to check for PTS updates out of pts-core and into its own module
- system_monitor: Add support for monitoring hard drive temperature using hdd.temp through system_hdd_temperature() within pts-core
- install-sh: Remove the Phoronix Test Suite shared and documentation files prior to copying over new files, in case some files were changed removed in new release

Phoronix Test Suite 1.4.0 Alpha 1
September 18, 2008

- pts-core: Change test/suite type strings to define statements
- pts-core: Add test type caching support
- pts-core: Create the unified PTS test/suite type handler
- pts-core: Clean-up some test installation functions
- pts-core: Provide new architecture for test/suite checking through the type handler
- pts-core: Drop PTS_USER_DIR usage
- pts-core: Add support for having tests and suites stored locally within ~/.phoronix-test-suite/ as an option for users installing new tests/suites or third-party files
- pts-core: Add support so that a test profile can extend another test, for Cascading Test Profile support. This is similar to inheritance in programming languages or Cascading Style Sheets.
- pts-core: Add in support for Self-Contained Test Profiles (SCTP) that initiate the Phoronix Test Suite and runs a stand-alone test
- pts-core: A test profile isn't real: likely incomplete or just a base test within a Self-Contained Test Profile extension
- pts-core: Report identifiers of test profiles being extended to the test profiles being run
- pts-core: Drop PTS shared download support for test profiles, instead they should be using the Cascading Test Profiles functionality
- pts-core: Don't allow the user to save a test result file if it's the same name as a real test profile or test suite
- pts-core: Add Mac OS X operating system detection support (using the IS_MACOSX define)
- pts-core: Improve VirtualBox virtualization detection for guest operating systems running
- pts-core: Add $COMPILER and $VIDEO_DRIVER user variables when saving the results and supplying the test identifier
- pts-core: Let the size of files being downloaded be specified individually within the test's downloads.xml profile instead of a generic one through the test profile XML file
- pts-core: When downloading files for tests, don't first download the file to the PTS temporary directory, but just rename it initially within the test's environment directory
- pts-core: When running a test (or suite that contains a test) on an unsupported architecture or platform, don't abruptly exit
- pts-core: Add module_setup() to pts_module_interface for returning an array of objects that can be configured by the end-user
- pts-core: Add support for persistent configuration options that end-users can easily adjust for PTS modules by running "phoronix-test-suite module-setup MODULE-NAME" that are stored to ~/.phoronix-test-suite/modules-config.xml
- pts-core: Add pts_module::set_option() function to allow a module itself to set a persistent configuration option
- pts-core: Add PTS_QUIT module return type for having the Phoronix Test Suite stop immediately and prematurely (this should only be used in the event of reaching a fatal bug or other problem)
- pts-core: Have the Phoronix Test Suite check once a day to see if the version in use is the latest version, otherwise inform the user of an available update
- pts-core: Add fall-back support for reading the PCI subsystem vendor if the motherboard vendor can't be determined using the other HAL information
- pts-core: Add a PreInstallMessage tag for test profiles that will be shown to the user prior to installing the test
- pts-core: Don't have PreRunMessage or PreInstallMessage hang when in batch mode
- pts-core: pts_find_file() -> pts_find_result_file()
- pts-core: Add initial support for detecting the motherboard Southbridge where applicable
- pts-core: Omit OEM in read_hal()
- pts-core: With list-tests, don't list tests that aren't compliant with the running system's architecture and platform
- pts-core: Add support so test options can be set as an environment variable by using <TEST NAME UPPER_CASE>_<NUM OPTION>=<INDEX OF REQUEST>
- pts-core: Switch to using php_uname() in pts-init.php
- pts-core: Add support for reading all ATI forced anti-aliasing types: Box, Narrow-tent, Wide-tent, and Edge-detect
- pts: Switch all audio encoding tests to extend a timed-audio-encode base test instead of all tests having some of the same code copied throughout
- pts: Switch all build tests to extend a timed-build base test instead of all tests having some of the same code copied throughout
- pts: Switch all video tests to extend a video-sample base test instead of all tests having some of the same code copied throughout
- pts: Change all current and relevant test profiles shipping the Phoronix Test Suite to using the FileSize tags within their downloads.xml files
- pts: Improve the reported environment size for many of the test profiles
- pts: Update compress-7zip test profile to use p7zip 4.58
- pts: Add World of Padman test profile (named padman)
- pts: Add padman to ioquake3-games test suite
- pts: Update Mandriva Linux packages XML file (Thanks To Stéphane Téletchéa)
- tandem_Xml: Add a pts_test_tandem_XmlReader object that extends tandem_XmlReader but takes advantage of the handleXmlZeroTagFallback
- tandem_Xml: Add fallback function handler support when a tag isn't found
- graphics_event_checker: Add graphics_event_checker module that makes sure the graphics system isn't messed up or having unintended options enabled while tests are running
- graphics_event_checker: Merge the gpu_error_counter module functionality into the graphics_event_checker module
- timed_screenshot: A module that takes a screenshot while tests are running at a pre-defined interval
- documentation: Add basic documentation concerning the Self-Contained Test Profiles (SCTP) XML tags that are added