Confusion over “2003 AD” and “2008 AD” with Exchange Installs.

One of my colleagues ran into this one last week, so I thought I’d share the issue here, and detail the fix.  When you go to install various versions of Exchange Server (2003 and up) and/or try to activate certain Exchange functionality, you may find that your Active Directory (AD) domains are not at the correct functional level, even though they appear to be.  Here’s a case in point:

I checked, and my domain is at a 2003 domain level, but Exchange Server 2010 says that it isn’t…

At first glance, this is an infuriating problem, because my colleague definitely confirmed that the Domain Functional Level was “2003” and the installer just wouldn’t work. The issue is a matter of one word versus another – specifically “Mixed” versus “Native.”

Mixed mode domains are designed to allow earlier versions of Windows AD Servers to operate properly in an environment that is being upgraded to a new AD platform.  Today, it is not uncommon to see 2003, 2008 and 2008 R2 Domain Controllers (DC’s) in a single environment, and the Mixed mode of each of those AD versions can allow for interoperability.  Exchange Server 2010, however, cannot operate if there are components of AD/Domain Services from Server 2000 or NT.  As such, it requires that only Server 2003 or higher AD controllers exist.  This means that you must be in 2003 Native mode (or higher) to install Exchange Server 2010.

Native mode ensures that no DC’s of any version prior to the functional level can communicate with AD in this domain.  This allows software packages like Exchange Server to not have to worry about passing commands that might be incompatible with earlier versions of AD.  Additionally, some features of Exchange Server 2010 go one step further and require either 2008 Native mode or 2008 R2 Native mode to function properly, so you may be on 2003 Native and still get errors if you are attempting to enable those services.

If you get the error during the pre-flight check on the installer (or later when adding features), make sure that your AD systems are in the appropriate Native mode, instead of Mixed, and you should be just fine. Note: you will often see these modes referred to as “Legacy” and “2008 Only” or similar language.  Legacy, as the name implies, allows for earlier AD controllers to communicate.  2008 Only or 2008 R2 Only limit the DC’s to those levels and higher.

Sometimes, when it comes to Exchange Server, one word can really make all the difference.

Don’t uninstall to fix Exchange!

Exchange Server does, occasionally, act oddly.  Alright, it can do that a lot, but hopefully you won’t run into the phenomenon.  If you should see strange behavior from your Exchange Server, the first thing to remember is that it is not a truly stand-alone application, and can’t be treated as such.  When problems happen with many software packages, the default fix-it is to uninstall the applications, reboot and reinstall.  While that may work with many apps on the Windows Server platform, doing that for Exchange Server will lead to rather disastrous results.

When Exchange Server installs, one of the first things it does is start building structures in Active Directory.  If you have no Exchange Servers at all, this includes extending the Schema and creating an Organization for Exchange and the related security constructs in both the Forest Root domain and the domain you’re installing into.  The thing many folks don’t realize until it is too late, is that much of this mucking about in Active Directory is undone when you uninstall an Exchange Server, which could be a problem if you plan on just fixing something and moving on.

When you uninstall Exchange Server, the uninstallation protocol will remove all objects in AD that were assigned to that server, which could lead to unexpected end results for users, Public Folders and other objects that don’t exist elsewhere in your organization.  This means that an attempt to uninstall to fix a somewhat minor issue could result in an absolute disaster for your Exchange environment overall that cannot be fixed by simply failing over.  The Organization and Forest Root updates will remain, but most of what you did in terms of configuration for the specific Exchange Server will be wiped out.

So, what to do in order to avoid this?  First, hit http://technet.microsoft.com and http://support.doubletake.com to see if there are articles on how to fix the problem you’re seeing.  These documents generally detail much easier fixes for common Exchange problems, and not-so-common ones as well.  Correcting the issues with Exchange still in place is the best route to take when it’s available.

If you do have to reinstall Exchange, your best bet is to shut down the malfunctioning server, build a new one with the same name, and use the disasterrecovery or recoverCMS switches with the Exchange Server installer tools.  This will transfer the settings over to the new machine, instead of removing everything permanently from AD.  Nearly all configuration for Exchange is stored in AD, so using these switches during the installation will allow you to get back up and running quickly, ready to restore the mailbox and folder data from an independent backup.  More information on these switches, the exact syntax for each version of Exchange Server and guidelines for use can be found in various places. http://support.microsoft.com, http://www.msexchange.org and http://technet.microsoft.com are three great resources.

As with most other topics, Exchange acts differently from most other applications when it comes to reinstallation. Always check the support sites before you take any drastic measures, and remember that Exchange is a totally AD-integrated application, so you’re never working on “just the Exchange Server itself.”

Friday Poll – DNS types.

Why use AD DNS?

My recent article on the need for (and use of) PTR Records in DNS have sparked quite a few questions on using DNS with Exchange Server in general.  The biggest one I get is “Do I need to use Active Directory DNS in order for Exchange Server to work?”  The answer to that one is a bit complicated, but in its simplest form, it boils down to, “No, but you really, truly should.”

Exchange Server 2000 and up required some form of DNS in order to function correctly.  This is mainly because the Windows Internet Naming System (WINS) was “depreciated” starting with that version of Exchange.  What that means is that MSFT officially asked the community to stop using it whenever possible, because it could be removed completely soon.  As it turns out, WINS was phased out in Exchange 2007, though it may still be required for certain Outlook functions.  That’s a topic for a whole different series of blog posts though.

As for DNS integration, it’s quite possible to install Exchange 2000-2007 without having Active Directory DNS configured in your domain, though it isn’t a best practice.  As long as your DNS system can handle Server Name Records (SRV type records), you can successfully use a 3rd-party DNS for your Exchange environment.  There are, however; some good reasons to go with the native Windows Active Directory Integrated DNS solutions:

1 – Exchange can natively talk to Active Directory DNS, and therefore can do some interesting tricks with that DNS platform that it can’t do with 3rd-party DNS.  Things like AutoDiscovery when you move a user to different mailbox servers, or after a recovery operation with Database Portability just don’t work the same way if you’re not using Active Directory DNS.

2 – Many 3rd-Party tools leverage AD DNS to figure out where Exchange resources are.  Note, I’m far from unbiased on this topic, so please see the disclaimer at the end of the blog.  Since many Windows-based tools will natively use AD DNS API calls (like DNSCMD and the newer variants in PowerShell), you may need to make manual updates to your 3rd-Party DNS, or may have to give up functionality.

3 – Many other non-mailbox objects are stored in AD DNS, and must be mapped manually in other DNS systems in order for Exchange to work properly.  You will have to track your Global Catalog servers, Domain Controllers and other resources in order for Exchange to function.

So, as you can see, there are some very good reasons to use Active Directory DNS if you plan on using Exchange Server.  While you may have external DNS records hosted with an ISP or other provider; internally you will be better off with the native DNS solutions in Windows unless you are ready and willing to fine tune your DNS systems and stay on top of it. 

If you are in doubt, you can use the Exchange Best Practices Analyzer to test your environment before you begin to install Exchange.  This tool will test for many things that Exchange needs, including properly configured AD or 3rd-Party DNS systems.

Exchange 2007 Installation – continued (finally)

Some avid readers brought to my attention that I had promised an article on the minimum required for installation of Exchange 2007 in a default server config last week, and didn’t deliver.  Sorry about that, and here we go!

Once you’ve stepped through the pre-requisites for Exchange 2007 (see two articles back), you’ll be ready to run the installer proper for the Exchange system.  Today, we’ll focus on the setup for a single Exchange 2007 server holding all required roles.

Before you can install Exchange 2007, we’ll have to get AD ready to roll.  Previous versions of the Exchange installer had the /forestprep and /domainprep switches that could be run by Domain Admins who didn’t have Exchange permissions and didn’t want to give those domain privileges to the Exchange Admin. Exchange 2007 doesn’t have those switches, but instead segments out the different tasks into a set of 5 switches, each doing a specific prep job.  You have two choices here.  First you can get Enterprise Admin rights and just run the setup wizard.  Second, you can have someone with Enterprise Admin rights at the root domain and Domain Admin rights for each sub-domain to run the individual commands.  They can be found, and are explained, in this TechNet article. 

Once the domain and forest have been prepped, you can run the Exchange installer on the server where you want Exchange installed directly.  While there are various command-line and silent install methods, I’m going to focus on the wizard-based installation.

After you step through the welcome screens, you’ll be asked a few critical questions. From here on out, we’re going under the assumption that you’re running as an Enterprise Admin and that you’re doing everything (domain prep and all) at once.

You’ll need to define the Organization Name.  This is the Exchange Org, and not the company name or AD domain name.  Though the three names (Domain, Company and Exchange Org) may be related, the Domain Name and Exchange Org Name can’t be identical.  Choose something that makes sense, and doesn’t use any special characters – stick to numbers, letters and underscores.

You’ll also have to allow or deny permission for Microsoft to be informed about errors that Exchange sees, and you’ll tell Exchange if there are users on legacy Outlook clients (Outlook XP and 2003) and if there are 3rd-party MAPI clients (like Entourage) in the client-base.  Be careful here, if there is any chance that you’ll have non-Outlook 2007/2010 clients, use the legacy setting.  This creates a Public Folder hierarchy to handle administrative Public Folder tasks like the Offline Address Book distribution; even if you do not use Public Folders for anything else or you are setting up different Public-Folder Only servers.  Without the administrative folders, Outlook before 2007 will not be able to function correctly.

You will also need to choose what Roles to install.  The default is to create Mailbox (MBX), Hub/Transport (HT) and Client Access Services (CAS) roles, which are the three mandatory roles you must have in place for Exchange 2007 to run.  While you do not have to have these all on one server, each site has to have at least one of each of these roles running somewhere.  The default selection in the wizard will put these three roles onto the server you’re installing to, which is fine for smaller organizations.  If you have heavy MAPI users or a lot of users, you will want to install the MBX role only and put CAS and HT on one or two independent servers.  If that’s the case, select to install MBX role only here, and run the installer on the machine(s) that will host the HT and CAS roles and re-run the installer there, choosing the appropriate options as you go to install just the roles you need on each box.

The remainder of the installation is pretty much automated.  You’ll be able to watch the progress of each installation task, and as Exchange moves forward you will see a status report (with green check, yellow bang or red stop sign) as each is completed.  Hopefully, you’ll only see the green checks across the board.

While it is sometimes not required to reboot after the install, it’s not a bad idea to reboot anyway.  The reboot should be quick, and will ensure that resources used by the installers are freed up, and that all the services start properly.  Neither of those things is bad, and no one is yet using the server, so reboot, please.

For a default installation, that’s about it!  Next week, we’ll start talking about what the individual roles do, so you can decide if you want them on independent servers (or at all, for the non-required roles).  I’ll also endeavor to actually write up what I promise next week  And finally, for my non-Exchange-2007 users, I promise to do some articles on Exchange 2003 again in the very near future!

Outlook, can you hear me? Can you feel me near you?

Might be showing my age and/or taste in music with that particular title (and if you’re totally confused by it, check out This YouTube video), but I think that it’s a great way to describe an annoyance that can happen if you’re using versions of Outlook before Outlook 2007.  Since a large portion of the users of Office are on the 2003 version (and many even earlier than that), resolution to a new server in the event of a disaster recovery event is a subject that is just as confusing as the famous rock opera I’m making use of in my title today.

When Outlook 2007 was introduced to the world with Exchange 2007, a lot was made (and rightfully so) of the new AutoDiscover features that this platform brought into the Enterprise Email marketplace.  The long and short of the AutoDiscover solution set is this:

When an Outlook 2007 client cannot find its home server – either because it is a brand new install of Outlook or because the home server has moved or been replaced – the Exchange 2007 AutoDiscover system can help Outlook 2007 find its home.  If the Outlook client can see an Exchange Server (or be directed to one by Active Directory), the Server can tell Outlook where the mailbox information for the user’s profile exists, and direct Outlook to connect to the appropriate CAS or Mailbox systems and get connected.  All the user/Admin has to do is tell Outlook the user’s email address and password, and AD with Exchange 2007 will handle it from there.  So if you’re installing Outlook for the first time, you don’t have to manually configure the Profile anymore – a great boon to Admins everywhere.

This system also kicks in if you perform Database Portability during a disaster, and have replicated the database with SCR; or have used a 3rd party disaster recovery/availability solution (see disclaimer below for all my bias information on that one =).  Once the Exchange system is responding again, AD can ferry the Outlook 2007 client to the new home for that mailbox, requiring only that the end-user close and reopen Outlook to complete the process.

However, what many folks do not realize right off the bat is that this solution set is ONLY available if you have both Outlook 2007 and Exchange 2007 as your messaging platform.  All users who need to take advantage of AutoDiscover must be using that combination of tools, and no other.  As you might expect, POP3 and IMAP systems do not AutoDiscover, but the majority of my clients were unaware that Outlook 2003 and earlier also cannot take advantage of this system, even if you have upgraded to Exchange 2007 as the messaging platform of choice.  It’s also worth noting that AutoDiscover doesn’t officially work in Exchange 2003 – no matter what Outlook version you are on.  Before I get blasted by mail on this one, I know some folks have sometimes seen it to work on Outlook 2003 with Exchange 2007, but it bombs more than it works, and officially it’s not supported.  For proof, I direct you to this article by the MS Exchange Team.

Since the code to perform AutoDiscover wasn’t in Outlook 2003, users on that client software will not be able to dynamically re-link to the new Exchange server unless the original mailbox server is still responding.  If it is, then Outlook can find the new server via the original server and re-home itself.  If not, Outlook must be manually re-directed to the new server.

Of course, there are ways around this.  You could update DNS to re-direct anyone calling for “Server 1” to the IP Address of “Server 2” – effectively re-routing all client software including POP and IMAP.  Outlook 2003 will still need to be re-profiled unless you take over the Service Principle Name (SPN) of “Server 1” on “Server 2,” but it will be a smoother transition.  Using a 3rd party tool (see disclaimer below) you may have the option of automated DNS and SPN updates, which will allow even legacy Outlook clients to jump to the new server with no more intervention than is required on Outlook 2007 with Exchange 2007 – even if you’re whole system in on the 2003 versions of those software platforms or earlier.

So, you are not without lots of options if you have any legacy servers and/or clients – or non MAPI clients – in your environment. You just need to be aware that the Exchange 2007/Outlook 2007 solutions for AutoDiscover services are not backward compatible, and plan accordingly.  Right now it looks like Exchange 2010 will have AutoDiscover that is backward compatible to Outlook 2007 only, so this soon-to-be-released platform is not going to solve this particular problem unless you’re planning on upgrading everything else in the environment to at least Outlook 2007 first.

Since I like to let folks continue to discover on their own, here’s a link to the White Paper from MSFT on AutoDiscover.

Finally, Update Roll-Up 8 for Exchange 2007 is out there, which can make life easier if you’re doing a fresh install of Exchange 2007 and want to get up to date with patches and fixes post SP1 quickly.  You can get it at this link.

Mailbox Separation Anxiety

As we have been discussing over the last few weeks, Exchange 2000-2007 runs through quite a few operations during the Online Maintenance (OM) process.  We've talked about how databases get cleaned up, and how messages get deleted (temporarily and then permanently).  This week, let's look at Deleted Mailbox Retention (DMR) and what it can do for you.

DMR is the process that Exchange uses to remove mailboxes that have aged past their expiry date in the Private Database.  These expired mailboxes can be created by different operations within Exchange, either directly by the administrator of Exchange, or by operations in Active Directory that inadvertently cause a mailbox disconnect.  Disconnected mailboxes are simply mailboxes that not longer have an associated user or object in Active Directory. The simplest way to do this is to delete a user account in AD, which will disconnect the mailbox assigned to that user as part of the process.  However, this isn't the only operation that can cause mailboxes to become marked for deletion.  There is an odd side-effect to moving mailboxes that can happen if you jump between AD sites and don't finish fast enough (You can read about it here), and of course; if you get hit with AD corruption it could disconnect mailboxes as well.

Once a mailbox is disconnected, the first OM run that happens after the disconnect will mark the mailbox for expiry in a user-defined number of days. The default is 30 days, so if you run maintenance daily, the mailbox will stay in the database, but disconnected, for 30 days after it is disconnected from the mail-enabled object (like a user) in AD.  It also means you have to take when you do OM into account for your deleted item retention numbers.  For example, if you only do OM once per week, then you will need to add up to 7 days to your overall retention time for planning purposes.  Since the 30 day clock doesn't start ticking until the mailbox is marked during OM, you will extend the amount of time data remains in the database.  This is critical for regulatory compliance, as holding data for an extra week could put you over the line in terms of meeting your regulatory guidelines.

The other part of this OM process happens when Exchange scavenges the database for expired mailboxes.  If it finds one, and the mailbox hasn't been re-assigned to another AD object, Exchange will delete the data from the database permanently and create white-space in the database that gets cleaned up by OM to be reused by Exchange.

So, here again, it becomes vital that OM is permitted to run on a regular basis.  If not, deleted mailboxes will not be removed from the database unless you manually launch the cleanup wizards.  This is more than just a regulatory problem, as without removing the old information and freeing up white space, your databases will grow much more rapidly than if you are regularly cleaning up your mail stores.

Next week... Public Folder operations during Online Maintenance, so stay tuned!

Exchange on DC's

Smaller shops have always been faced with a fundamental quandary when the time has come for them to move off Small Business Server and into the world of the independent version of Microsoft Exchange.  One of the biggest considerations is where to put the Domain Controllers for Active Directory.  The first thought most of my clients have is to put it on the same server as the Exchange system, but outside of SBS that's a very bad idea for a handful of important reasons.

For example, if you're ready to make the move from a single server to a cluster, you cannot install Exchange 2003 on a cluster node that is also a DC.  That is just simply not supported by Microsoft, and therefore isn't an issue up for debate.

It's much more likely that you'll be moving to a single-server, however, as if you're just moving off of SBS then the likelihood you need a cluster is reduced quite a bit.  In those cases, while you could install Exchange on the DC, it's still not a good idea.  The Exchange Best Practices Analyzer will even flag this condition as going against best practices, which should be a giant red flag for anyone installing an Exchange Server.

Here's a few reasons it isn't a good idea:

1 - Since Exchange runs many services under Local System accounts with elevated privileges, a successful attack via the Exchange system could easily expose your AD controller to outside influences.  With the potential to gain access to all domain resources if the AD controller is compromised, reducing attack surface is vital, and installing Exchange (or any app, this is not an Exchange-specific issue) will *increase* attack surface.  If you get hit on an Exchange Server that is not a DC, the attacker has access to only that server, not the entire Domain.

2 - 3rd Party integration can suffer.  Since many 3rd Party tools may expect to use Local Accounts and Groups (which wouldn't exist on a DC) or may otherwise need to control aspects of the Exchange System that may not be available or accessible on a DC, integration with Anti-Virus, Backup and Recovery solutions can suffer if the server hosts both roles.

3 - Performance impacts will be clearly visible.  Since the Exchange Server constantly uses the DC and vice-versa, placing both on one physical server can effectively double the amount of work that server must do.  Some visible effects of this are slower overall Exchange performance; longer response times for Domain commands (such as logging in, changing passwords, etc.; and much longer reboot times.

Putting the Active Directory and Exchange Server systems on one physical machine may seem like a way to save money, but in reality it will cost more in downtime and headaches than it will save on server hardware.

 

[Note: Small Business Server contains a highly modified version of Microsoft Exchange Server that is specifically designed to run on an SBS server.  It does not permit all 3rd Party applications, limits functionality and is also limited to only being able to run on SBS.  Therefore, running the SBS version of Exchange on an SBS DC is both a supported installation and a best practice. This article specifically applied to non-SBS versions of Exchange Server.]