Deleted Item Retention in Exchange Maintenance

Sorry for the delay in this posting - end-of-quarter stuff tends to push blogs off the table, but I'm back!

As promised in my last full posting, I'd like to talk about deleted item retention as it relates to how Exchange Online Maintenance and your users in general.  As most Exchange Admins and Engineers know, if you delete an item in Outlook or a mailbox in Exchange, things don't instantly disappear.  More on Mailbox retention next time, but for now, let's look at what happens when you delete a mail item.

If a user Soft-Deletes (normal delete operations) an item from their mailbox, it is moved to the Deleted Items folder in their local client.  If you Shift-Delete in Outlook or otherwise delete with a non-Outlook client, this recovery method is not available for those items.  It can be enabled for so-called Hard Delete operations in Outlook (See this MSFT KB article) but not for 3rd party email systems like POP3 mail.  So this information only applies to those using Outlook (sorry about that).

When an email is deleted from Outlook, Exchange doesn't remove the item itself from the server-side data store.  Instead, it removes the item from Outlook, but simply marks the item as ready for deletion in its own database.  This is why deleting a large amount of email won't cause an immediate reduction in your database size - the data didn't go anywhere.  It will not delete email from the mail store itself until the Deleted Item Retention (or Recovery) (DIR) time has passed.  By default, this means that all deleted mail items will be held for 30 days before being permanently removed.

You can indeed change the time that Exchange holds onto these emails, both on a server-wide-level or for individual users.  These tools are available as part of the standard Exchange tool-set, and as usual, details can be found over at TechNet.  What you set your DIR to is really up to the organization in question, as many industries have regulatory requirements that may help you set policies, and your internal rules might give you guidance as well.  Though the default of 30 days is fine for most, a little poking around will help you find out for sure what your numbers should be.

So, what does this have to do with Online Maintenance (OM)?  Well, since Exchange just marks items for deletion in (by default) 30 days, when does it actually delete the emails?  The answer is, on the first OM after the DIR time for that item has expired.  During OM, the Exchange system performs several processes.  One of these is to comb the databases for any items that have their DIR flags set, and ensure that any which have aged past their DIR date since the last maintenance was run.  OM then deletes these items permanently and creates white-space in the database (see the last posting about what happens with white-space).  If OM is not allowed to go through all its stages, or otherwise can't do them within the time you've allotted, then these items will not get deleted, taking up more and more space over time.

This is yet another reason why OM is vital to any healthy exchange server operation, as never deleting email is a bad idea all around.  Note that DIR is *not* designed to remove the need for proper backup and DR strategies, as the loss of the server would result in the loss of the DIR information too - see previous posts on that topic.

Now, keep in mind that OM doesn't remove the white space that's created, it just moves it to the back of the database, but since Exchange can re-use this white space before creating more space in blank disk areas, not allowing OM to finish means your databases will grow at a much faster rate.  So while your databases will not shrink as a result of OM like they would with *offline* maintenance, your databases should grow much more slowly if you're doing OM than if you were not allowing this process to complete.

On the other side of the equation, your end users can recover any item that is in the DIR cache and has not yet been removed from the server.  Handy if something is accidentally or maliciously deleted.  In most cases, the users just needs to click on the Deleted Items folder in Outlook and then go to Tools and select Recover Deleted Items.  You can see a more detailed explanation on TechNet.  This means that you do not have to manually restore the item for them, making your life a lot easier, as long as they realize they need it back within 30 days by default.

Next time - with any luck next week - Deleted Mailbox Retention - going beyond getting back one item.

Meet me at TechEd 2009!

For those going to TechEd in LA this year, I'll be there with the Double-Take crew!

Here's my TechEdConnect QuickConnect card:

Hope to see you there!

As promised, some more info on Online Maintenance Part 1

Quite some time ago I promised to go into some more detail on what happens during the Online Maintenance period of Exchange Server.  This is an often-confusing topic as it happens in the background and - except for the whopping load it puts on the server when it's running - there is not a lot of indication that it happened at all.  Today, let's talk about some of the basics, such as when it happens and why you should allow it to happen regularly.

Online maintenance (OM) is an automated set of procedures that Exchange 2000-2007 servers perform nightly by default.  Though often referred to as one big process, OM is in fact a catch-all term that describes several different procedures, one or all of which may happen during the OM window each night.  These processes include, but aren't limited to, Deleted Item Retention; Mailbox Cleanup, online defragmentation and Public Folder Cleanup.

Left to its own devices, Exchange will perform OM automatically between Midnight and 4am each day.  This is mostly due to the fact that OM will use a significant amount of processor power and disk I/O while it runs through its various processes, so it isn't a great idea to let OM run during the normal business day.  If, however, your peak times are between Midnight and 4am, you can definitely change when OM runs through the GUI for Exchange by right-clicking the Store in question and going to the Properties dialog box.  That will let you go to the Database tab, where you can adjust the Maintenance Interval to match your less-than-normal server load times.

Even if the default schedule is good for your organization, you may still want to edit the times for individual Information Stores if you have more than a few of them.  This is a good idea since you'll be able to stagger the OM runs, allowing the server to spread the overhead of performing OM on multiple stores by not forcing the server to run them all at once.  Running OM on all of the Stores at once is possible, but will put a heavy burden on the server if you have many Stores, and may cause some maintenance tasks to fail.  They'll be retried in the next window, so you aren't out of luck, but if the load can be spread out, that's always a good idea with Exchange as a general rule of thumb.  Check the Application Logs in Windows Event Viewer in the AM, and you can see the results of the OM processes as log events.  That will give you an idea of some things are not finishing.

As to why you should do OM often and completely, that's a bit easier to explain.  Exchange is a database, not a collection of mail files.  Like just about every other database, when items are written out to disk, they are done in the fastest way possible, instead of trying to group everything into (human) logical areas of control.  This translates into faster write times, but can lead to a highly fragmented database structure, which can cause slowdowns in general and errors down the line.  Also, since we're not talking about discrete files for each email, when something is deleted, what really happens is that Exchange marks the areas of its database where that information existed as available to be reused.  This is commonly known as creating white space in the database, and can add to the overall level of fragmentation as you will have pockets of white space all over the place.

OM - during one of its processes - combs through the database to find areas where defragmentation can occur.  This allows Exchange to move all the white space to the back of the database, where it can more effectively be re-used for new information coming in.  While this isn't strictly required, it does help speed up the overall Exchange system and make it more efficient, slowing the growth rate of your databases in the process.  Note that your databases won't shrink during this process, and we'll talk about that more in a future column, but they will grow slower, as Exchange will be able to use available space more effectively before it starts allocating new space.

In addition to fragmentation, Exchange doesn't really delete an email or mailbox when you hit delete in Outlook or the Exchange management GUI's. Most Exchange Engineers will know that items are marked for future deletion, and that doesn't happen until OM is run.  We'll delve into Deleted Item Retention and what OM does to address it in later columns, but the take-away here is that unless you allow OM to run regularly, you will have to manually run processes to remove old, deleted mail items and mailboxes.  For those with strict compliance and regulatory concerns about deleted email, this becomes a vital consideration.

To wrap up Part 1, you should allow OM to run nightly if you can, spreading out the times over as long a period of time as you can, with different Stores having OM run at different times if possible.  This will allow for the most effective running of the OM processes, but constrict them to happening when fewer users are trying to send/receive mail.  You should also run OM at least monthly if not more often, because otherwise your databases will become less efficient and items that are deleted will never be removed.

Next week...How Deleted Item Retention works in Part 2!