If I cannot bring you comfort, then at least I bring you hope

As we get ready to celebrate the beginning of a new year (in many countries), now is a good time to plan for your Exchange Servers and what you'll be doing in the year ahead.  While this is not the best time to plan out new infrastructure or massive overhauls immediately, you can start planning what you'll do later in the year on that scale.  You can also plan for what you'll do right off the bat in January for things like maintenance and updates/upgrades.

I plan to dedicate entire columns to these topics in the next few weeks, but here's the overview to get you thinking (once the champagne wears off...):

Offline Maintenance is a good thing to do regularly for Microsoft Exchange.  About once or twice a year you will probably want to take the time and walk through the process.  Microsoft says that this is not a solid requirement, but it can help identify long-trending problems deep within a database, so doing it once or twice per year is time-consuming, but worth it.  Plan now for downtime, so that you don't have to worry about when you'll have the ability to do it later. 

Online Maintenance is automatic, but you may want to look at your Exchange Server profile and see when the best time to have it done really is.  By default, the online maintenance system will run every night, but if you're also performing backups overnight, the two systems can overlap, causing the maintenance to be halted and never finish.  Take a look at your event logs to see if both systems are operating normally, or if the online maintenance run is terminating early.  If it is, you may want to change the times it runs, or even only run it on certain days.  Unlike with Exchange 2000, later versions of Exchange can hold up admirably well with even weekly maintenance runs, but be aware of what won't get done until maintenance by checking out Microsoft's Knowledge Base and/or waiting until my upcoming column on the topic.

Test, test, test, test, test......then test again! Now is the perfect time to set out your goals for testing of recovery/availability solutions (see disclaimer at the bottom of the page).  Backup systems, failover systems and other recovery solutions MUST be tested, otherwise how do you know they're doing what they say they're doing.  Best bets: do a test restore of Exchange data (at all the levels available from your tool like database, mailbox, mail item) to a different directory on the Exchange server and mount the DB in a Recovery Storage Group to ensure it is working properly.  This should most likely be done once per month, but at least once per quarter.  It doesn't require downtime, and doesn't require end-users to do anything.  Perform a failover test (if you use a failover tool) or a total restore test at *least* once per year.  2x per year or once per quarter is better, but of course since this type of testing will impact end-users, scheduling can be an issue.

Housekeeping is never a bad thing to look at after the clock strikes midnight on January 1.  Are you reaching your database limits in Exchange Standard edition?  If so it's time to either start ratcheting down mailbox limits or else starting to plan your upgrade strategy.  If you're already on Enterprise edition, are you pushing the limits of the free disk space you have on your drives?  Plan now to move the databases to another partition with more space.  Have your Stores grown to alarming proportions?  Might be time to start planning out new Stores and even Storage Groups to distribute the load.

Finally, give a look through the event logs and such to see if there are repetitive errors or other trends that can indicate problems brewing.  One more common example of this is recurring event ID's 1018, 1019, and 1022.  These indicate that there could be problems brewing in the database itself, though it is very possible that the error events are the only indication you see until the DB crashes.  If you're looking for a tool to help wade through the log information, check out Splunk, an IT search solution set that can make your life a lot easier.  [Due Disclosure: Splunk is currently working on partnerships with Double-Take Software.]

Have a healthy, happy and safe New Year's celebration.  Here's hoping 2009 exceeds everyone's wildest dreams in terms of greatness, and that Exchange never ceases to help us all live in these very interesting times.

Until next year.....

Mike Talon, Chief Columnist, BeingExchanged.com

PS: The title of this posting is taken from the song "Closing of the Year" from the movie soundtrack to "Toys."  The lyrics of the first verse are incredibly relevant to our combined experience this year, and I offer them here:

If I cannot bring you comfort,
then at least I bring you hope
For nothing is more precious than the time we have, and so
We all must learn from small misfortunes
Count the blessings that are real
Let the bells ring out for Christmas
At the closing of the year

- The Musical Cast of Toys Featuring Wendy and Lisa

Monitoring the holidays

As many of us plan to take some time off for various and assorted holidays (Christmas myself, though I do celebrate Festivus as well), our Exchange servers will hopefully run along on their own, brimming with Holiday Cheer!

However, due to the nature of the business, it might not be a bad idea to keep an eye on the servers without actually being in the office.  Not only does this mean you'll be able to look like a rock start to your boss, but it also means you can enjoy the time off without constantly worrying about your systems blowing up.

Monitoring tools can take many forms.  They can range from relatively simple ping-test solutions that just make sure the server looks alive, to more complex solutions that monitor application or even individual service health over time.  Which one you need really depends on the complexity of your environment and what you feel you need to keep an eye on.

Looks-alive solutions will regularly poll a server via some form of heartbeat system - generally an ICMP Ping.  If a server fails to respond to a pre-set number of pings, then the server is declared offline and alerts are sent out or other pre-configured actions are taken.  These systems are great for smaller shops that just want to be sure the servers are responding, or as part of larger monitoring solutions for enterprise systems. They are generally either free or very low cost, and often found as part of different solutions for Disaster Recovery (see disclaimer below), backup tools, etc.

More extensive monitoring packages can come in many flavors, but the two most common are service/application monitoring tools and log monitoring tools.

Service/application monitors are very common for most mid-sized and larger organizations to have running in their networks.  This can be something like Microsoft Operations Manager (now called Systems Center Operations Manager) - mostly for windows systems - to the more universal tools like Tivoli, NetIQ and SiteScope.  These tools generally require agents to run on servers (though not always) and allow you to not only test for simple connectivity, but also see if Exchange itself is responding to common queries, that its component services are properly up and running, etc.

Most of these types of solutions also monitor logs of various types.  That gives them the ability not only to see current problems, but to track errors and other hiccups over time to alert you if there might be a bigger issue brewing.  This can be things like looking for Exchange page errors - and indicator that a disk could be failing on you.

The drawback is that these solutions are typically not free, and in many cases will cost quite a lot.  They're worth every penny in nearly every situation, but it is a budget item you'll need to plan for and another expense you'll have to pay out.

Monitoring of your servers can let you know if things are either going wrong, or about to go wrong, within your Exchange environment.  They allow you to keep an eye on your servers even if you're not physically watching your servers.

Happy Holidays everyone!  And here's hoping you don't find yourself beating the living daylights out of your servers with The Festivus Pole!

Happy Holidays Everyone!

To all my great readers:

I have friends who celebrate just about every form of holiday around this time of year, and even a few who don't officially celebrate anything at all.  To all of them, and all of you, please have a happy and safe Holiday Season, and thanks for helping me make the first year of BeingExchanged.com a success.

I've got a holiday card to share with you!

I will almost definitely have one more posting before Christmas, but in case you don't get to check back before then, see you next year.  Here's hoping 2009 is better than anyone dreamed possible!

Virtualization update - it's kinda supported - sorta

Exchange Server running on virtual machines has always been a somewhat uncomfortable topic for many Exchange Engineers.  Microsoft's support policies for VMware virtualized servers has always been confusing - to say the least - but with the advent of Hyper-V, their stance has softened significantly.  These days, the official line is that you can run Exchange Server 2007 SP1 on Hyper-V or any other Server Virtualization Validation Program (SVVP) -cleared virtualization product. In short, MSFT will now offer best-effort support for Exchange, even if it's running in a VM, with a few key exceptions.

First, the VM platform has to be part of the SVVP approved list.  There's only a few right now, but they're the major players. Hyper-V, ESX and Xen are all on the list, and for more info you can visit the SVVP site and click under "Products" to see who's been certified.

Also, you cannot be running the Unified Messaging role.  MSFT will still only support this role on physical servers, most likely due to the close interaction of the UM role with PBX and other phone systems.

Finally, you still need to be aware of this article from the MSFT knowledge base.  Even with the new best-effort support policies, MSFT may request that you try to re-create the problem either on physical servers or Hyper-V VM's, which could be a cumbersome process.

So, while MSFT is making great strides in extending support to virtualized Exchange servers, it may still be some time before complete support is available.  This isn't to say that you can't virtualize Exchange successfully, it's just that you definitely should look before you leap on this particular topic. Be sure to line up support resources from VMware, Citrix, or whomever produces your VM platform ahead of time. This way, if something goes awry, you know who to call for support - and in what order to make the calls.

BES you is, or BES you ain't my baby...

Managing, protecting and just plain nurturing your Exchange Servers is a pretty common theme in this blog, but many times we can ignore eco-system servers that are just as critical.  Blackberry Enterprise Server (BES) is one such example, though many corporations use GoodLink and other tools as well.

Eco-System servers are systems that are not Microsoft-created add-ons for Exchange Server, but that rely on (or are even installed on) Microsoft Exchange boxes.  They can range from something simple like an anti-spam service or appliance, to something as complex as a Customer Relationship Management (CRM) solution that uses Exchange as a data warehouse.   BES is extremely popular among mid-sized and larger organizations to provide remote access to email where Windows Mobile or other Exchange Activesync devices may not be the best choice. While the Blackberry platform from Research in Motion is a great way to deliver mail to mobile workforces, it will almost definitely add at least one more server to your environment that you have to keep an eye on.

BES runs on a Windows platform in most cases, and generally will be installed on a different machine than the Exchange Server itself.  (I'm keeping this discussion specifically to Windows and Exchange, please visit Research in Motion, LTD for other options).  If you choose, you can use BES as a hosted service, and therefore not have any servers on your site, but most mid-sized and larger organizations bring the BES system in-house.

There are two common configurations for BES.  The simplest is a single-server that hosts both the BES application and a small MSDE or Microsoft SQL database.  MSDE is a great engine, but is limited in terms of how large the database can grow.  Once you go beyond that limit, a full license of SQL is required.  Single server configurations offer simplicity, only one box to manage and worry about backing up, replicating, etc.  Their drawback is scalability. Once you go past a couple of hundred users (easy to do in larger firms), the overhead of both the BES application and the SQL systems on the same box will slow the server to a crawl when it is under any form of stress.

Multi-server configurations take over where single-server configurations get bogged down.  You can host the BES applications on one server, and the database on another.  This allows each application to use the power of the entire server it is located on.  In addition, you can have multiple BES application servers sharing the SQL back-end server, which gives you higher levels of scalability without having to use multiple SQL licenses, offering a cost savings that adds up fast.

In all of these situations, there are a couple of considerations that you need to be aware of when setting up redundant and/or Disaster Recovery for BES systems.  The first is that the application servers are high-I/O load systems, but don't store much data locally.  Nearly all BES data is stored either on the Exchange Server or in the SQL database.  While that means that there is less stuff to protect on a BES application server, it doesn't mean there is nothing.  The Server Routing Protocol (SRP) information and a critical set of application settings are still housed on the application server, so protection is mandatory.

In short, you have to protect Exchange, SQL and the BES application server in your DR plan, otherwise your mobile users will quickly find themselves with no mobile e-mail.

SQL and Exchange are relatively well documented in terms of protection.  But the application server poses a very specific issue.  If the SRP is installed on two machines (as in a traditional failover solution), and these two machines accidentally transmit data to the RIM network at the same time, you will be forced to manually correct the issue by calling RIM tech support and getting the SRP unlocked.  So, your basic options for protection of this type of server are to either use a backup/restore method; use a full-server failover system (like Livewire or FSFO - see disclaimer below); or be prepared to build a new application server in the event of an emergency.  The theory of having a standby BES server with the SRP loaded but kept offline does work (it's called Knife-Edge Cutover), but can be dangerous in situations where many administrators work on the servers and don't necessarily talk to each other before doing things.

Any of these theories can be used, but you will need to have the method you will use ready to go well before the actual emergency.  Plan well, and your Exchange, SQL and BES services will perform as expected, but fail to plan and the mobile devices you rely on to get emergency info out to your employees will be turned into expensive paperweights.

VMworld Ask the Expert Session for Double-Take

Hi folks!  We're working with the VMworld.com site to help create discussion topics and answer questions about using Double-Take technologies with VMware solution sets.  So if you have some topics to talk about, or some questions you'd like to discuss with DT Employees and civilians alike, head on over.

You can reach the forum via this link.

2008, Exchange 2007, SP1, head going to explode....

For those looking to install Exchange 2007 on Server 2008, you have do doubt run into the fact that only the SP1 version of Exchange 2007 is supported on that platform.  This leads to a conundrum for those of us raised on the previous versions of Microsoft Exchange, namely how do you install the Release to Market (RTM) version of 2007 onto server 2008 in order to do the SP upgrade?

The short answer is, you don't.

The long answer is, you can go ahead and toss out your RTM Exchange 2007 disks with the trash, you don't need them anymore at all.

Microsoft has started a new trend with Exchange Server 2007, namely slipstreamed Service Packs that no longer require at least one previous version be installed first.  When you download the Service Pack Binaries from here, you will notice that the overall package size is gigantic compared to SP's from other platforms. That's because the original binaries - updated where necessary for SP1 - are also in that package.

In short, you can have a perfectly new Server 2003 or Server 2008 box, download only the SP1 version of Exchange 2007, and do the entire install from the SP1 version of the bits, no need to install the RTM first.  This is how you get around the fact that only SP1 is supported on Server 2008, just install only the SP1 bits, and you get the whole package at once, already updated.

You still have to have a valid activation key if you are installing from scratch, even if you use the SP1 bits, so be ready to type it in when requested or you're stuck with a 180 day trial timer.

The good news is that it looks like Microsoft will be pre-slipstreaming other applications SP's as well.  This will make life a great deal easier when you need to install an application from scratch, especially for those - like me - who can never seem to get the slipstreaming process to work correctly no matter what tools they try!