SAFELY GETTING MAXIMUM BENEFIT FROM CACHE AND COMPRESSION PROGRAMS There has been a lot of confusion on whether it's safe to use disk cache and compression programs such as DOS 6's DoubleSpace (now called DriveSpace), Stacker, and cache programs such as SmartDrive. (Most of the controversy centers on the compression programs.) You've probably heard reports of people losing their entire disk due to cache or compression problems. These experiences have quite justifiably alarmed many people into not using these type of programs. Many people regard any increased risk of harm to their data as unacceptable and I agree. Yet, it would be a shame to lose out on the speed and extra capacity that these programs can provide if there's a way to be sure there won't be any problems. I'll show you what the real problem is and how to avoid the risk of damage to your disk from these programs. The DOS 6 DoubleSpace Saga Microsoft DOS 6's DoubleSpace is a disk compression program that allows you to increase the amount of data you can store on your disk. It creates what appears to be another disk on your PC. All files that go to this disk are automatically compressed. Recently it became clear that many people were experiencing damaged files on compressed disks. InfoWorld had an alarming experience (Reported in the May 3rd and May 10th, 1993 issues). Thirty of their fifty-three test machines experienced problems during the installation and testing of DoubleSpace. Some problems appeared immediately upon installation and others showed up only after several weeks of use. Some PCs experienced only individual corrupted files, but many had damage to the entire compressed disk. I've gotten similar reports here at Stiller Research. In some cases, files would change without warning and then the files would suddenly be back to normal. Some of these changes would only be obvious with a data integrity checker. In other cases, the damage is more obvious with "Sector not found," "data error reading drive," "cross-linked files," "lost cluster" or "Damaged CVF" messages on the compressed volume. In the case of the damaged CVF (compressed volume file) this resulted in complete loss of the compressed volume. Is DoubleSpace (or DriveSpace) a Problem? No, it turns out that DoubleSpace doesn't seem to be the problem. We've had similar reports with compression products other than DoubleSpace. We've come up with some techniques that I'll share with you today that allow safe use of these compression and cache programs. InfoWorld cleared up all but six of their failing PCs by eliminating problems caused by other software. The SmartDrive (Cache) Saga SmartDrive is a disk cache program. Such programs speed up disk access by anticipating disk reads and having the information already read into memory. When a program asks to read information that the cache already has in memory, the disk access appears to happen very quickly. It's also possible to cache disk writes as well as reads. SmartDrive (SMARTDRV.EXE), the cache program bundled with DOS 6 or Windows 3.1, comes with disk write caching enabled. This feature, known as "Lazy Write," provides faster response by delaying writes to the disk. This means that when a program asks to write a record to disk, it will think the write has completed very quickly while in reality the record hasn't been written to disk yet. Instead, SmartDrive has simply stored the record in memory and will write it later. This speeds up most programs since they no longer need to wait for each write to disk. Lazy Write will cause a problem if your PC should lose power or if a buggy program should hang the PC before SmartDrive physically writes the data to disk. This could result in damage to one or many of your files. When information is damaged that keeps track of other files (e.g., the FAT or directory entry), this could spell disaster since one or more directories or even the entire disk could become corrupted. Compressed volumes such as DoubleSpace volumes are particularly vulnerable to this type of damage since a small error will cause greater damage on such a volume. Flushing the SmartDrive Cache It's critical that the cache program write all information to disk from memory before you stop your PC. This is called "flushing the cache." When you boot your PC using Control/Alternate/Delete, SmartDrive will automatically flush the cache. To flush the cache when you're planning to turn the PC off, enter the command, "SMARTDRV /C" (without the quotes) and hit ENTER. It's vital that everyone religiously follow this procedure if you allow Lazy Write caching. If you happen to turn the power off before SmartDrive has done its delayed write to disk, you will experience corruption to one or more of your files. Disabling Lazy Write Because of the risk of damaged files, many people have chosen to disable Lazy Write (write caching). If you're using SmartDrive, you can disable Lazy Write by including the disk letters on the Smart Drive command line. If you look in your AUTOEXEC.BAT file you will probably see a line that looks something like: LH C:\DOS\SMARTDRV.EXE or it may look like SMARTDRV 2000 To disable write caching just use a text editor (e.g, "EDIT" which comes with MS/DOS 5 or 6) to add the disk letters to command line. If you want to disable Lazy Write for disks C and D, your lines would now look like: LH C:\DOS\SMARTDRV.EXE C D or SMARTDRV C D 2000 Is this worth doing? Yes, at least initially. If you know you are unlikely to lose power or experience unexpected program hangs (where you can no longer boot with CTRL/ALT/DEL), then the risk is probably worth the benefit. Just Say No? Many people have responded to reports of problems by just not using their disk cache or compression programs. Is this really the best plan? No, I think it would be unfortunate to lose out on all the benefits that these programs can provide. A better solution is to take steps to verify that these programs are working correctly with no undetected damage. These potential problems are not restricted to just disk compression and cache programs but can affect all memory resident programs that access the disk. The success of DOS 6 has simply made DoubleSpace and SmartDrive the best known trouble makers. So, What's the Real Problem? The core problem seems to be that when multiple programs execute simultaneously there is always a risk of timing problems and unexpected interactions. Each PC represents a different environment for these programs. Even variations in disk speed can cause timing to vary between what appear to be identical PCs. This means that a software combination that works well on one PC could cause problems on another apparently identical PC. With resident programs that access disk such as spoolers, disk compression programs, or caches, these types of conflicts can be particularly troublesome. This a particularly tricky issue since these problems may take weeks before they become obvious. Is There a Solution? Yes! I've worked with numerous people who have experienced damaged files due to these types of software conflicts and in each case we've been able to find a way to get a stable and reliable configuration. Step-By-Step Guidelines for Installing Compression and Cache After working through detected corruption on DoubleSpace compressed disks with several of our customers and using DoubleSpace on drives here at Stiller Research, I've found some guidelines to assure that you can use compression and cache safely on your PC. Note, that I also have reports of customers using DoubleSpace and SmartDrive with absolutely no problems (for months) and NOT following my suggestions. I'll be talking about specific steps for safely installing and using DoubleSpace and SmartDrive, but my suggestions apply equally well with other similar programs. These tips will improve performance of your compressed disk and avoid known pitfalls associated with installing disk compression and cache. Before installing compression (DoubleSpace), follow these steps: 1. Eliminate all resident programs (TSRs) and unnecessary device drivers other than memory managers. If you are already using DOS 6, you can clean boot by hitting F5 at boot time. Otherwise, copy your CONFIG.SYS and AUTOEXEC.BAT files to a diskette and then use a text editor to eliminate any unneeded lines. Boot your PC in this minimal configuration. 2. If you have data integrity software, (such as Integrity Master) do a quick check of your disk so that you have up-to-date information describing all your files. If you don't use integrity software, then you'll have to use your backups to check your files. 3. Make sure you have a current backup for your disk. (I suggest you always have at least two backups for your critical data.) This is vital so that you're protected in case something goes wrong during conversion to a compressed volume. 4. Defragment the disk so that you have as much contiguous unallocated disk space as possible. DOS 6 "DEFRAG" will do this. When you run DEFRAG you want to make sure you have as much available memory as possible but I'd suggest you not do anything new to free up memory. Keep things simple. 5. If you're using data integrity software, run a check of all your files to make sure that the defrag didn't damage anything on your disk. You're now ready to install your disk compression software. If you're using DoubleSpace, you can play it safe by asking it to convert the unused space on your volume to a compressed volume instead of converting everything to a compressed volume. You can then copy your files a few at a time from the uncompressed to the compressed volume. This allows you to catch any problems very quickly since you can check each set of files after you copy them to make sure everything is going well. If you need extra space, DoubleSpace will allow you to resize the compressed volume to make it larger. Don't worry -- this is all easy; you just execute "DBLSPACE" and then follow the menus. This technique also makes it easy to keep the programs that are most response time sensitive on the uncompressed disk. (Double space compressed volumes are slower than your uncompressed disk) I suggest you leave your DOS files and most frequently used programs on the uncompressed volume to improve performance. After creating your compressed volume: 1. Run CHKDSK to make sure DoubleSpace thinks everything is OK. 2. Verify that nothing was damaged in the process of conversion. Do a complete integrity check of both your uncompressed and compressed volumes or use your backup program in verify mode to compare all files making sure that nothing was damaged in the conversion. 3. Add your normal memory resident programs (TSRs) and device drivers back into your CONFIG.SYS and AUTOEXEC.BAT files. If you're planning to run DOS 6 MemMaker, run it now. If you're planning to use a cache (SmartDrive), add it at this point but don't use Lazy Write (see my earlier comments). Boot this configuration. 4. Now, let's test your compressed volume to make sure everything is really going to continue working reliably. Copy and delete a large number of files on your compressed volume. Execute some programs that access the disk and then move and copy more files. 5. Now run another check of your disk (steps two and three above). If there are some problems, try eliminating TSRs or device drivers (DEVICE= statements in your CONFIG.SYS file) to see which is causing the problem. Eliminate the cache (SMARTDRV.EXE) first; I've found it to be the most likely suspect. 6. If you want to use Lazy Write (i.e., if you're brave), try enabling it now. With Lazy Write enabled, copy some files and run some programs that access the compressed disk. Boot your PC a few times while using these programs. Afterwards, run a full check (steps two and three again). Make sure that you try out all your most common programs. If you run into any problems at this point, I'd strongly advise you turn off Lazy Write. If everything looks good so far, the odds are very good that you've got a reliable configuration. There's still a small chance that your configuration might trigger a subtle timing related problem. Therefore, it's best to do additional checks (steps two and three) of your disk. Two weeks is usually sufficient to shakeout any configuration problems but even an uncompressed volume could experience undetected data corruption at any time. There's no way to guarantee that resident programs are ever 100 percent safe but these steps provide maximum assurance that you have a stable and reliable configuration. Considering the disasters that some people have experienced, it's absolutely vital that you follow these steps to make sure your disk is safe. I think you'll agree that it's not safe to install this type of software and merely hope that everything works well -- hoping is not enough! I believe I've given you a way to enjoy the speed and extra disk capacity that these programs offer without taking a chance on losing all your data. Copyright © 1995 Stiller Research. Document Last Modified 5/22/95.