MS-DOS 6.0 DoubleSpace Test - Final Summary.
 
Delivered To:
Microsoft Corporation
By:
Chris Farmer
Senior Project Leader
Prepared June 3, 1993
XXCAL Credits
Chris Farmer, Senior Project Leader/Automated Test Manager

Ted Novak, Senior Project Leader/Network Test Manager
Spiro Kourtessis, Network Test Engineer
Colby Burkett, Network Test Engineer
Joe Benedello,  Network Test Engineer

Veronica Romero, Automated Test Engineer
Mark Tillinghast, Automated Test Engineer

Kris Cieply, Applications Test Engineer
Andre Andreos, Applications Test Engineer

Jeff Duncan, Aquisitions Manager
Dean Conant, Aquisitions Support
Billy Angle, Aquisitions Support
Executive Summary
This document is the Final Summary Report on the XXCAL's test of MS-DOS v6.0 with specific focus on the DoubleSpace compressed volume utilities.  Testing was done to evaluate the compatibility and functional integrity of DoubleSpace from XXCAL's expert independent standpoint.  The project was  executed under the terms of the  Microsoft Corporation DOS v6.0 File System Test Proposal Rev. 2, Proposal #9305003.
Methodology -
45 personal computer systems (various models and configurations) were configured with MS-DOS 6.0. Two distinct compressed volume file (CVF) setups were used.  One group of systems utilized a 'default' configuration with a CVF  designated as drive C: and an uncompressed 'Host' volume designated as drive H:.  Another group of systems utilized an 'alternate' configuration with a CVF  designated as drive N: and an uncompressed 'Host' volume designated as drive C:.  In both cases, DOS and Windows applications software were installed on the compressed volume.  Windows permanent swap files were created on the uncompressed 'host' volumes on 386/486 based systems.  All test platforms were networked to a dedicated file server running Novell Netware v3.11 to provide test log storage and print job services.
XXCAL created a custom automated test script to drive continuous application execution and file operations on each test platform.  Once launched, the script ran for at least 72 hours on each of 45 test systems.  The test scripts were designed to monitor and log disk file integrity using output from a combination of DOS CHKDSK, DBLSPACE /CHKDSK, and CRC-32 file integrity check utilities.
Test Results -
45 test systems completed the continuous 72 hour test.  No file corruption occurred on these test systems at any point during the test period.
Conclusions -
No problems were encountered when upgrading the test systems from MS-DOS 5.0 to 6.0 or during subsequent execution of DoubleSpace on any of the test platforms included in the test bed.  Variants of XXCAL's automated test script ran continuously for over 72 hours on each of 45 test platforms in the test bed.  This test scenario represented over 3240 hours of testing and, during that time, no problems were encountered with test platform file subsystems on either the CVF  drive or the uncompressed 'Host' drives.
Introduction
This document is the Final Summary Report on the XXCAL's test of MS-DOS v6.0 with specific focus on the DoubleSpace compressed volume utilities.  This test was executed  at XXCAL's testing facilities in Los Angeles, California during May 1993.  Testing was done to evaluate the compatibility and functional integrity of DoubleSpace compressed file volumes (CVFs) from XXCAL's expert independent standpoint.
XXCAL's mandate was to design and execute a test suite to evaluate the long term reliability of the compressed volume file system as implemented under MS-DOS v6.0 via MS-DOS supplied DoubleSpace utilities and device drivers.  Testing was performed in a test bed environment utilizing multiple vendor's computer systems.  The project was divided into two phases that included:
Phase 1 - Development of an appropriate product evaluation test suite
Phase 2 - MS-DOS v6.0 file system test suite execution
The project was  executed under the terms of the  Microsoft Corporation DOS v6.0 File System Test Proposal Rev. 2, Proposal #9305003.  Test results are the primary focus of this final report.  In addition, product specifications, test methodologies and other project information are included to provide a frame of reference for interpreting the test results.
Test Methodology
System Configurations
Testing was performed at the Client's request across multiple vendor's personal computer systems including 80286, 80386, and 80486 CPUs.  The Client specified this range of test systems in order to verify the functional integrity and performance of the DoubleSpace disk compression in a variety of test hardware envionments.
Test Platform Specifications
The test environment consisted of forty-five (45) networked computer platforms operating with MS-DOS 6.0 along with the DoubleSpace compressed volume device driver/utilities.  Initially, all computer systems were configured under MS-DOS v5.0 (and subsequently upgraded to MS-DOS v6.0).  Test systems were primarily rental systems acquired by XXCAL during the project period.  Systems utilized represented commonly available vendor models and CPU types.  The composition of systems in the test bed included:
  
Qty	Configuration	Model

7	80286/10 - 2MB RAM - 40MB HD	AST Premium 286
9	80286/10 - 4MB RAM - 40MB HD	AST Premium 286
10	80386/33 - 4MB RAM - 110MB HD	NEC Powermate 386/33i
10	80486/33 - 8MB RAM - 200MB HD	IBM PS/ValuePoint 6385 M50
9	80386SX/20 - 4MB RAM - 60MB HD	Compaq Deskpro 386s/20
Network Environment
A LAN environment was necessary to support the test suite methodology developed to test MS-DOS 6.0 and the DoubleSpace compressed volume utility.  During execution of XXCAL's automated test suite, all test platforms were connected to the LAN with the server providing storage for each test system's testlog data and print jobs.  In addition, "master" profiles of test platform hard drives (including test scripts and applications) were stored on the server.  These profiles were utilized during preliminary system setup to "clone" platform configurations and insure identical software configurations on similarly configured (RAM, hard drive volumes, model) test systems.  In addition to storage functions, loading network drivers on each test system/workstation provided additional stress on system memory during test execution.
File Server Configuration:
XXCAL installed NetWare 386 v3.11 (100 user) on a COMPAQ Deskpro 486/66 M Series system per Novell documentation.
LAN Adapters:
XXCAL configured the test server to support both ethernet and token ring topologies.  Server and workstation LAN adapters selected from available in-house products (using the most up-to-date NetWare driver support) included:
- Ethernet adapters: Novell NE2000, SMC 8000, 3Com 3C501, 3C503, 3C507
- Token-Ring adapters: IBM TokenRing 16/4
Workstation Configurations:
Various test systems utilized differing memory configurations during the test cycle.  The majority of test platforms contained 4MB of RAM, with some platforms containing either 2MB or 8MB of RAM.  DOS 6.0 device driver software was loaded both high and low to simulate 'real world' configurations.  For the most part, the 'default' DOS v6.0 installation changes were maintained, simulating 'typical' end-user configurations.  Two DBLSPACE compressed volume configurations, distributed across test systems, were utilized during the project.
The exact configuration of each platform under test (i.e. memory size, hard drive configurations, BIOS revision, LAN adapter configurations, etc.) was recorded in XXCAL Test Reports generated for each test system -- see Appendix B - Test Reports.
Test Suite Development
Software Applications and Tools
XXCAL's challenge on this project was to simulate a "real world" test of MS-DOS 6.0 and DoubleSpace while maintaining an automated test approach to maximize testing time.  Test automation (using a custom designed test suite) was required to:
 1.	Enable continuous testing across multiple test systems for at least 72 hours.
 2.	Establish a consistent and repeatable test suite that could be modified/tuned to the capabilities of each test platform.
  
To meet these requirements, XXCAL employed various popular DOS and Windows applications to create test data files.  Applications included:
  
Title	Vendor

Lotus 123 for Windows v1.1	Lotus Development Corporation
Freelance Graphics for Windows v2.01	Lotus Development Corporation
Word for Windows v2.0	Microsoft Corporation
Lotus 123 for DOS v2.4	Lotus Development Corporation
WordPerfect for DOS v5.1	WordPerfect Corporation
dBASE IV v1.5	Borland International
In addition to applications software, XXCAL employed various utility programs and automated test tools to verify file integrity, manipulate application data files, maintain testlog files, and control application software execution.  These utilities and test tools included:
  
Title	Vendor

CRCMAN.EXE	Greenleaf Software
MOVREN.EXE	XXCAL Testing Laboratories
LOG.EXE	XXCAL Testing Laboratories
MS-TEST v1.0	Microsoft Corporation
XXBENCH for DOS v1.1	XXCAL Testing Laboratories
While the use of applications to create data files is a straight forward process, testing the Windows applications (as well as WordPerfect 5.1) required the use of automated test tools.  Additional utilities were also employed to provide file integrity checks and file manipulation functionality.  An explanation of each program utilized in the automated test suite follows.
CRCMAN.EXE -
Initially, this program was used to build a list of CRC-32 values for all of the executable files in a selection of directories on the test platforms.  Before launching the automated test on each platform, a data file of CRC-32 values was generated for critical executable files residing in each application directory on the test system.  Once the automated test was underway, the CRCMAN.EXE program was used during each test cycle to verify the CRC-32 values of the 'live' files against those values stored in the 'pre-test' data file, giving assurance of the integrity of the files as the test progressed.  If a CRC check failed at anytime during the test, the automated script was designed to abort the test and report the error.
MOVREN.EXE -
This program is used to move and rename 'input'  files to uniquely named files.  During each test  suite cycle, the DOS and Windows applications created data files which were left on the hard drive after the application completed execution.  Later in the test cycle, the MOVREN.EXE utility was used to move the files to a '\cleanup' directory and give them unique file names.  This enabled the applications to create new files using standardized naming conventions during each test suite cycle.
LOG.EXE -
Throughout each test suite cycle, test status information and disk/file integrity check data was sent to a testlog on the file server.  In order to maintain a sequence of testlogs and preserve results for each test suite cycle, the logs had to be renamed in sequential order.  The LOG.EXE program was executed at the end of each test cycle to rename the 'testlog' file to the next numerical extension .xxx (1-999).  In addition, the program deleted all files in the \cleanup directory based upon a test suite cycle iteration value.
MS-TEST v1.0 -
XXCAL elected to use Microsoft's MS-TEST automated Windows software test tool to control the Window applications specified for this test suite.  MS-TEST  code was created to open the applications, perform data file creation operations, data file deletions, and additional disk integrity checking during the Windows application test portion of the automated test suite.
XXBENCH for DOS v1.1 -
XXCAL's XXBENCH for DOS v1.1 was utilized to control the execution of WordPerfect 5.1 during the DOS application test portion of the automated test suite.  In contrast to Lotus 123v2.4 and dBASE IV v1.5,  there was no way to get WordPerfect v5.1 to automatically execute internal macros after an initial load.  Therefore,  to continue automated test execution without interruption, XXBENCH was used to load WordPerfect and launch a series of WordPerfect macros to create the necessary data files.
Test Platform DoubleSpace Configurations
Two types of compressed volume file (CVF) configurations were used in testing.  All platforms were intially prepared with MS-DOS v5.0 (including the installation and basic testing of NetWare shells).  XXCAL's intention was to utilize approximately equal numbers (distributed across platform models) of the following configurations:
Default configuration -
The 'default' configuration was what a typical end-user would probably utilize when upgrading to MS-DOS 6.0, with drive C: as the CVF and drive H: as the 'host for C' volume.  Before the MS-DOS 6.0 upgrade, pre-installed applications and test files, including Windows v3.1 and representative Windows applications, DOS applications, and other files were copied from LAN storage or another 'master' system onto the test platform's C: drive.  A basic test was performed to insure the integrity of the copied test environment.  As the last preparation step immediately after the DOS upgrade, the 'default' configuration platforms (with CVF configured as C:) had existing data and applications compressed and stored on the C: volume using DoubleSpace in the 'Express' installation mode.  Additional Windows swap file configuration procedures were executed to adjust the size of the permanent swap file (standardized to 4MB on each 386/486 test platform) on the uncompressed H: volume.
Alternate configuration -
The 'alternate' configuration was created to replicate a previous third party test, with drive N: as the  CVF  and drive C: as the 'host for N' volume.  'Alternate' configuration platforms were intially setup with only MS-DOS 5.0 and network driver files and subsequently upgraded to MS-DOS 6.0.  After completing MS-DOS 6.0 file installation, the remaining empty space (with the exception of 9MB to accommodate a Windows permanent swap file of 4MB) was compressed as drive N: using DoubleSpace.  Pre-installed applications and test files, including Windows v3.1 and representative Windows applications, DOS applications, and other files were copied from LAN storage or another 'master' system onto the test platform's N: drive.  A basic test was performed to insure the integrity of the copied test environment.  Additional Windows swap file configuration procedures were executed to setup a permanent swap file (standardized to 4MB on each 386/486 test platform) on the uncompressed C: volume.
MS-Windows 3.1 Configuration, Memory Management, and SmartDrive
Microsoft Windows v3.1 was installed using the default options.  All 80386 and 80486 based systems were running Windows 3.1 in Enhanced Mode and using a permanent swap file of 4MB located on the uncompressed file volume.  On the AST Premium 286 test systems, no swap file option was available since Windows v3.1 was running in Standard Mode (as expected on 80286 based platforms).  In addition, the MS-DOS 6.0 memory management features such as loading various device drivers and utilities in high memory were not available on the 80286 based systems, but were used on all 80386 and 80486 based systems.
Regarding Microsoft's SmartDrive disk cache software, all systems were running SMARTDRV.EXE in the default mode (with delayed writes on) as configured by the MS-DOS 6.0 installation program.  However, on the IBM PS/ValuePoint 486/33 systems, SmartDrive was also installed as a device driver in the config.sys file and included the double buffer switch.  The following line was present in the config.sys file on the IBMs:
Device=c:\dos\smartdrv.exe /double_buffer
The /doube_buffer switch in SMARTDrive provides support for bus mastering controllers through a Microsoft "standard" called Virtual DMA Services.  These services provide  a memory buffer that has physical and virtual memory addresses that are the same.  This allows bus mastering controllers to get correct addresses and avoid problems with reading data from the wrong location in memory or writing data to the wrong place in memory.  This type of problem can occur during any disk I/O operation when running either MS-DOS or Windows applications.
Automated Test Script Summary
The primary target for all test script operations was the compressed volume file (CVF) on each test system.  While all test systems were networked, application execution, data file creation/deletion, and additional file operations were done 'locally' on the CVF drive.
The following automated test script outline is presented to provide some insight into the variety of operations performed during each automated test suite cycle.  This hierarchical list of procedures gives an overview of data/disk integrity checks, application software execution, and file manipulation operations executed during  numerous iterations of the script during each test platform's 72 hour test period.
 1.	Perform intial chkdsk on compressed volume and create testlog (Drive T: on the file server)
 2.	Execute Windows 3.1 and Applications Test Suite
	2.1 Run CRCMAN.EXE on Windows and Lotus files
2.2 Run Windows, MS-Test, Lotus 123 applications
2.3 Call diskchk2 batch file to check for file irregularities
2.4 Run CRCMAN.EXE on Windows and Freelance files
2.5 Run Windows, MS-Test, Freelance applications
2.6 Call diskchk2 batch file to check for file irregularities
2.7 Run CRCMAN.EXE on Windows and Word for Windows files
2.8 Run Windows, MS-Test, Word for Windows applications
2.9 Call diskchk2 batch file to check for file irregularities
 3.	Execute DOS file operations commands
	3.1 MD, COPY, DEL, REN, MOVE, XCOPY operations
3.2 Call diskchk2 batch file to check for file irregularities
 4.	Execute DOS Applications Test Suite
	4.1 Run CRCMAN.EXE on Lotus 123 r2.4 executable file (123.exe)
4.2 Run 123r2.4 application
4.3 Call diskchk2 batch file to check for file irregularities
4.4 Run CRCMAN.EXE on WordPerfect v5.1 executable file (wp.exe)
4.5 Run WordPerfect v5.1 test
4.6 Call diskchk2 batch file to check for file irregularities
4.7 Run CRCMAN.EXE on dBASE IV v1.5 executable file (dbase.exe)
4.8 Run dBASE IV v1.5 application
4.9 Call diskchk2 batch file to check for file irregularities
 5.	Execute additional routines to cleanup after DOS file operations (Step 3)
 6.	Execute DOS file operations on application files
	6.1 Application .EXE file copy, rename and delete operations
6.2 Temporary directory deletions
6.3 Call diskchk2 batch file to check for file irregularities   
 7.	Execute application data file cleanup routines using MOVREN.EXE utility
	7.1 Move leftover DOS application data files to \cleanup directory
7.2 Move leftover WINDOWS application data files to \cleanup directory
7.3 Call diskchk2 batch file to check for file irregularities   
 8.	Execute additional routines to cleanup after DOS file operations (Step 3)
 9.	Execute logfile and \cleanup directory maintenance routines
	9.1 Rename current testlog file with numeric extension
9.2 Delete contents \cleanup directory every "n" iterations
10.	Loop back to beginning of test script (Step 1)
  
  
Note   The test scripts were designed to abort at any sign of disk file system corruption as reported by either DOS CHKDSK , DOS DBLSPACE /CHKDSK, or the CRCMAN.EXE utility.  Thus if, a test platform continued to run the script up to any point in time, the conclusion was that no file system problems were encountered as of that point.
  
Test Execution
Automated test execution on each test platform was done in the following steps:
 1.	Setup the test system in 'default' or 'alternate' configurations.
 2.	Install automated test scripts.
 3.	Launch automated test.
 4.	After 72 hours (or more) of test execution,  abort the test script and archive testlogs (results).
 5.	Generate configuration report using Microsoft MSD.EXE and write test report in Q&A 4.0 database.
  
References
Documents referenced by XXCAL throughout the project included:
  
Document	Author	Date

Microsoft Corporation DOS v6.0 File System Test Proposal Rev. 2, Proposal #9305003	Troy Sukert, XXCAL	5/6/93
MS-DOS 6.0 Manual	Microsoft Corporation	n/a
Test Results
Test Report Summary List
Testlogs were generated during each test run and archived at the conclusion of the project.  However, this data is too voluminous (over 200MB of ASCII text files) to present in this final report.  Therefore, the logs were analyzed and a series of summary reports created in Q&A 4.0 to document and summarize test results on an individual platform basis.  These test reports were classified as PASS.  The test systems in this list are sorted by model, then memory configuration, then CVF partitioning strategy.
  
Platform Mfg/Model	Plat#	EXTENDed Memory		Hard Drive Partitions	Class

AST Premium 286	38	1408K EXT, 320K XMS		Compress=C: &Host=H:	PASS
AST Premium 286	33	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	40	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	32	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	31	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	4	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	6	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	7	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	8	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	9	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	10	1408K EXT, 320K XMS		Compress=N: &Host=C:	PASS
AST Premium 286	5	3456K EXT, 2368 XMS		Compress=N: &Host=C:	PASS
AST Premium 286	36	3456K EXT, 2368K XMS		Compress=C: &Host=H:	PASS
AST Premium 286	3	3456K EXT, 2368K XMS		Compress=C: &Host=H:	PASS
AST Premium 286	2	3456K EXT, 2368K XMS		Compress=C: &Host=H:	PASS
AST Premium 286	1	3456K EXT, 2368K XMS		Compress=C: &Host=H:	PASS
Compaq Deskpro 386s/20	12	1024K EXT, 1932K XMS		Compress=C: &Host=H:	PASS
Compaq Deskpro 386s/20	11	1024K EXT, 1932K XMS		Compress=C: &Host=H:	PASS
Compaq Deskpro 386s/20	13	1024K EXT, 1932K XMS		Compress=C: &Host=H:	PASS
Compaq Deskpro 386s/20	19	1024K EXT, 1932K XMS		Compress=C: &Host=H:	PASS
Compaq Deskpro 386s/20	15	1024K EXT, 1932K XMS		Compress=N: &Host=C:	PASS
Compaq Deskpro 386s/20	14	1024K EXT, 1932K XMS		Compress=N: &Host=C:	PASS
Compaq Deskpro 386s/20	17	1024K EXT, 1932K XMS		Compress=N: &Host=C:	PASS
Compaq Deskpro 386s/20	18	1024K EXT, 1932K XMS		Compress=N: &Host=C:	PASS
Compaq Deskpro 386s/20	16	1024K EXT, 1932K XMS		Compress=N: &Host=C:	PASS
IBM PS/ValuePoint M50	47	7168K EXT, 4870K XMS		Compress=C: &Host=H:	PASS
IBM PS/ValuePoint M50	44	7168K EXT, 4870K XMS		Compress=C: &Host=H:	PASS
IBM PS/ValuePoint M50	46	7168K EXT, 4870K XMS		Compress=C: &Host=H:	PASS
IBM PS/ValuePoint M50	45	7168K EXT, 4870K XMS		Compress=C: &Host=H:	PASS
IBM PS/ValuePoint M50	48	7168K EXT, 4870K XMS		Compress=C: &Host=H:	PASS
IBM PS/ValuePoint M50	50	7168K EXT, 4870K XMS		Compress=N: &Host=C:	PASS
IBM PS/ValuePoint M50	42	7168K EXT, 4870K XMS		Compress=N: &Host=C:	PASS
IBM PS/ValuePoint M50	49	7168K EXT, 4870K XMS		Compress=N: &Host=C:	PASS
IBM PS/ValuePoint M50	41	7168K EXT, 4870K XMS		Compress=N: &Host=C:	PASS
IBM PS/ValuePoint M50	43	7168K EXT, 4870K XMS		Compress=N: &Host=C:	PASS
NEC Powermate 386/33i	26	3072K EXT, 1676K XMS		Compress=C: &Host=H:	PASS
NEC Powermate 386/33i	27	3072K EXT, 1676K XMS		Compress=C: &Host=H:	PASS
NEC Powermate 386/33i	25	3072K EXT, 1676K XMS		Compress=C: &Host=H:	PASS
NEC Powermate 386/33i	23	3072K EXT, 1676K XMS		Compress=C: &Host=H:	PASS
NEC Powermate 386/33i	24	3072K EXT, 1676K XMS		Compress=C: &Host=H:	PASS
NEC Powermate 386/33i	29	3072K EXT, 1676K XMS		Compress=N: &Host=C:	PASS
NEC Powermate 386/33i	30	3072K EXT, 1676K XMS		Compress=N: &Host=C:	PASS
NEC Powermate 386/33i	21	3072K EXT, 1676K XMS		Compress=N: &Host=C:	PASS
NEC Powermate 386/33i	28	3072K EXT, 1676K XMS		Compress=N: &Host=C:	PASS
NEC Powermate 386/33i	22	3072K EXT, 1676K XMS		Compress=N: &Host=C:	PASS
Conclusions
A flood of computer industry press coverage has accompanied the release of Microsoft's DOS 6.0.  The reviews have been generally positive.  However, much attention has been focussed on the MS-DOS 6.0 disk compression  features offered by the DoubleSpace utility program.  Some personal computer industry publications are reporting end-user complaints that DoubleSpace has numerous installation problems, damages or destroys data or application files stored on the system hard drive, and even renders hard disks inaccessable.  XXCAL's MS-DOS 6.0 DoubleSpace Test was designed to exercise the DoubleSpace software under controlled conditions to verify the product's integrity.
The latest version of DOS (6.0) bundles a variety of utility programs together to provide the end-user with additional control and options for configuring and maintaining their personal computer system.  This increase in capabilities has also increased the overall complexity of the operating system's features.  XXCAL's installation strategy was to follow the product documentation faithfully during MS-DOS 6.0 installation and while subsequently configuring the test system hard drives using the DoubleSpace disk compression utility.  No problems were encountered when upgrading the test systems from MS-DOS 5.0 to 6.0 or during subsequent execution of DoubleSpace on any of the test platforms included in the test bed.
Variants of XXCAL's automated test script ran continuously for over 72 hours on each of 45 test platforms in the test bed.  This test scenario represented over 3240 hours of testing and, during that time, no problems were encountered with test platform file subsystems on either the CVF drive or the uncompressed 'Host' drives.

 