Why you need centralized event log management

Article Author: Peter Gubarevich (MVP – Enterprise Security), Certified Ethical Hacker

Everything was just fine until yesterday.

Usually, I’m managing local group membership using Active Directory Group Policies, either Restricted Groups node, or GP Preferences node. It’s quite questionable which node to choose, but anyway, Group Policy makes it easy to perform updates and enforce membership throughout a domain, and exactly fits my requirements. However, there’s another side of the coin: unwanted changes are applied as easy and quickly, and there’s no simple way of reverting changes back.

This time, I have been assigned a task to configure all Windows Servers in our domain to allow Remote Desktop sessions for IT Security staff members, which are not Domain Admins. Easy as a cake, isn’t it? Okay, I’ve launched Group Policy Management console, opened ”MyCompany Servers Group Membership Baseline” Group Policy object and updated the GP Preferences part of it to include “IT Security staff” group into Remote Desktop Users local group.

Being linked to “Servers” AD Organizational Unit, this GPO was applied to all 80 servers in a production environment, excluding Domain Controllers. In at most 90 minutes, all servers have read the policy and updated their local groups. Then, I’ve figured out that something went very wrong: first-level support reported that accountants are unable to launch Axapta, multiple users cannot log on to Terminal Server farm, developers are getting ”Access Denied” message trying to establish RDP sessions to their SQL servers… and everything of this was Remote Desktop. I’ve opened GPO again and noticed my mistake:

Well, now I got the root cause of the failure: this “Remove all members” checkbox was accidentally copied from another GP Preference object. Trying to get everything back to the original state, I’ve figured out the following: simply clearing checkbox does not return previous Remote Desktop Users members. Even removing entire policy won’t help, because there is no any group membership history saved in Registry, Windows files or any other location.

OMG, what have I done.

The only thing that saved me was Auditing, which I have previously configured to log successful group membership changes on all domain members. The logic of returning previous state was as following:

• Launch Computer Management console and connect to a particular server’s Event Log
• Filter Security Log by Date/Time and User Account Management Success, Event #4733
• Write down all members removed from Remote Desktop Users local group
• Switch to Local Users and Groups, then add lost members back to the required group
• Proceed to the next server

Even looking that simple, this procedure may take up to 10 minutes per server, and an Event Log parsing requires the most time. Multiplied by 80 servers, total recovery may take up to two days, which is quite a big downtime, isn’t it?

Now try to figure out the value of centralized event log management system.