Exchange Database online vs offline defrag

One of the more common questions I hear from my clients is "Would I ever need to run an offline defrag on my Exchange Databases?" The answers do vary, but in the end, they all boil down to "yes, occasionally." This invariably leads to questions about why offline defrag is necessary when Exchange 2000/2003/2007 does online defragmentation on a regular basis.

Defragmentation in Exchange Server comes in two different forms, online and offline. While both take steps to make Exchange run more smoothly, they are designed to do different things. They are also designed to run at different times. Online defragmentation is performed during maintenance (usually nightly); while offline defragmentation is a manual process.

Online defragmentation is the process of compacting the database, so that white space is pushed to the tail-end of the EDB files. White space is - simply put - empty pages within a database. In the case of Exchange, white space is created when items are permanently deleted during deleted item retention cleanup, when pages are removed from the database due to normal operations, or any other time something within an EDB gets removed. Since databases like Exchange don't really delete anything normally, what happens here is that the areas of the database/disk where objects are deleted are simply marked as empty. These areas, however, still exist within the database file (and on disk), and can cause performance slowdowns as read and write operations have to happen over a larger area of physical disk, and the database system has to keep track of all the little white space pockets.

During a nightly (by default) maintenance run, in addition to cleaning up deleted items and performing other tasks, the Exchange system will identify all white space and move it to the end of the database. Compacting like this allows the database to work much more efficiently, and allows the maximum amount of white space to be re-used for new data before the system must take up more of the empty disk space.

While this process does allow Exchange to more efficiently use white space, there are times when just moving stuff to the back of the database isn't quite enough. While offline defragmentation shouldn't be done routinely, it is a great tool to use when you:

1 - Need to remove a large amount of unused space from a database. This could be because a large amount of mail was removed all at once from a system - as would occur if an archiving solution were implemented. Online defragmentation would remove the data, but leave a large amount of white space that may be taking up too much room on disk. Note that this only applies if a *large* amount of white space exists. Unless there was a huge amount of data removed, you would end up with the database growing to the same exact size as before the offline defrag, probably within a few weeks or less.

2 - Certain types of errors appear in the Exchange system. See this link for specifics. Offline defragmentation can correct for some common errors, fixing a database in the process.

3 - A "Mail Storm" impacts your server. Mail Storms are large amounts of mail (several times your normal volume) suddenly moving across a server. Since this can create a large amount of white space and badly fragment a database, running an offline defrag can correct for any negative effects of the Storm. This is another instance where we're talking in terms very large numbers, as smaller storms (such as a sudden increase in normal spam) would not fragment the database enough to allow you to gain benefit from an offline process.

One thing to note, you will need a place to hold the temporary files created during the defragmentation in order for the process to work. Generally, Microsoft recommends about 110% of the size of the database you'll be defragmenting, which is often quite a large chunk of space. This can be on another local disk, or on the same volume as the database itself, space permitting. While it is technically possible to use a remote or network drive for temporary space, it isn't recommended, since it slows down the process and will cause it to fault if the network link hiccups. 

Offline defragmentation is not required to be run on any form of schedule, as a matter of fact Microsoft doesn't recommend running it at all unless there's a concrete reason to do so - such as those listed above and here. However, if those conditions are true, then an offline defrag can help make Exchange run better, use less space, and stay healthy for the long haul.