It’s not an exciting or glamorous subject but it’s an absolutely critical concept for properly managing your applications and infrastructure. CMDB, CMS, SIS, EIEIO (joking) or anything else you want to call it these days is a concept that has been poorly implemented from the very beginning. The concept itself is sound; have a single source of truth that describes your application and infrastructure environments to enable IT operations efficiency (at least that’s the core concept in my mind).
CMDB’s are Awesome, but Not Really
Back in the my days working for global financial services institutions I relied heavily on the CMDB as a starting point for many different activities. I say it was only a starting point because invariably the information within the CMDB was wrong. It was either originally input wrong, not updated regularly enough, or updated with incorrect information. No matter what the real cause, my single source of truth became a partial source of truth that always required extensive verification. Not very efficient!
This is how a traditional CMDB represents component relationships.
Getting back to my reliance upon the CMDB… Here are the types of activities that required me to query the CMDB:
- Change controls – Anytime I needed to change anything on a production server I needed to understand what applications had components existing on that server.
- Application upgrades – I needed to know all of the application components at that exact moment to make sure they all got the update.
- Application migrations – There were times when we simply needed to move the application from one data center to another. This required a complete understanding of all application components, flows, and dependencies.
- Performance troubleshooting – When I was asked to get involved with a performance problem, one of the first things I wanted to understand was all of the components that made up the application and any external dependencies.
There are many more uses for the data in the CMDB but those were my top use cases. As I said before, invariably the CMDB was wrong. There were usually components missing from the CMDB, and components in the CMDB that were no longer part of the application, and incorrect dependencies, and, and , and…
Salvation by Auto Discovery and Dependency Mapping, but Not Really
So what’s a good IT department supposed to do about this problem? Buy a discovery and dependency mapping tool of course. And that’s exactly what we did. We explored the market and brought in the best (relative) tool for the job. It was one of those agentless tools that makes deployment way faster and easier in a large enterprise like mine was. The problem, as I would later realize, is that agentless discovery tools only see what’s going on when they login and scan the host. Under normal conditions you can scan your environment maybe once or twice a day without completely overwhelming the tool. What that means is that all of those transient (short lived) services calls into or out of each application are misses by the discovery tool unless they happen to be running at nearly the exact time of the scan.
Discovery and dependency mapping tools did a better job at maintaining and visualizing relationships but still fell short.
To add further insult to injury, most organizations don’t want a bunch of scanning activity going on during heavy business hours so the scans are typically relegated to the middle of the night when there is little or no load on the applications that are being scanned. This amplifies the transient communication dependency mapping problem. Now the vendors who sell these solutions will claim that there are ways to deal with this issue if you just use their network sniffer in conjunction with their agentless appliance. I won’t comment much on this but I will say that this creates another slew of deployment problems from a political and technical perspective and the thought of every trying it again makes me wince in pain. (Where did I put my therapists phone number again?)
The Application Knows, Really It Does
What better source of understanding application components and dependencies is there than the application itself? Let’s explore this for a moment. If you can live inside of the application and see all of the socket connections opening and closing then you absolutely know what else the application is communicating with. Imagine if there was a system that could automatically see all of these connection that open and close at any given time and draw a picture of the application and all of it’s dependencies at that exact moment in time or any point in the past. And imagine if this system had a published API that allowed your other systems to query it for this information. Regardless of transient or persistent connection types, you would have the ability to know all of the components of your application and all of its external dependencies. This is exactly what AppDynamics does out of the box.
Discovery of application components and dependencies from within the application, in real time, provides the most accurate information possible.
I believe that the CMDB of old should be an ecosystem of information points that provide the truth at the moment it is requested. Forrester calls this a SIS (service information system) in their research paper titled “Reinvent The Obsolete But Necessary CMDB”. Click here to read it if you’re interested. The SIS isn’t some vendor tool, instead it’s an architectural construct that should be different for each company based upon their tools and requirements. From my perspective it is incredibly difficult and inefficient to manage a datacenter or group of applications without implementing this type of concept.
If you’ve already got AppDynamics deployed, consider using it as a significant source of truth about your applications. If you’re stuck with an outdated CMDB, consider shifting your architecture and check out how AppDynamics can help with a free trial.
