Security, General IT Architecture and other matters explored by Paul Slater, a Brit far away from home
My first column on Enterprise Architecture is now published at Server Management magazine. Right now, they are very short (around 500 words) and only every other month, but the magazine is hoping to scale back up when times improve.
There are a few strange edits, but overall I’m pretty happy with it. Hopefully I’ll be writing it for as long as my previous security column (8 years).
As many of you know, for the last few years, I’ve been writing a regular column for Server Management magazine on security. Unfortunately due to the downturn the magazine has been scaled back a great deal and has gone online only, but the good news is that I will still have an opportunity to write regularly on topics of significant interest to me - Enterprise Architecture and Strategic Planning.
Once the column is published, I’ll be sure to post a link to it from here.
I just re-read a nice summary of what constitutes an Agile Architect. This article has been around for years (I think I first read a version when I was working on a project back in 2005) but if anything I think it holds more relevance today. I’m looking forward to pursuading the rest of my team that it makes a lot of sense to embrace these principles.
This post talks about an intersting Google labs feature. It’s a fairly useful way of capturing a snapshot of useful information that can be consumed offline. I’ll check it out over the next few days.
The more observent among you will notice that this blog carries advertisements. I don’t make much money out of them. Actually I’ve yet to make any. Google first pay you when you reach $100, and after around a year of carrying advertisements, I am at a grand total of almost $11.
I’ve never really worried about it. Blogging here is not my day job after all. My main point in writing here is to organize my thoughts, and have a home for my pre-edited magazine articles. But recently I started to wonder just why more people don’t read this blog. Sure, it get’s a little technical sometimes, but so do many others. And it may not be the most perfectly written material out there, but its better than most. I have a pretty good idea how to write - I’ve had several hundred magazine articles published, and over a dozen books. Compare that to the average missive you’ll find on LiveJournal, and I’m practically Shakespeare. Yet still almost nobody comes.
So I set to figure out why, and in the process I’ve discovered something that is illuminating to me and perhaps to the empty forest that is my audience. I started by thinking about the sites that I consume most regularly at the moment, either directly or through RSS feeds. In no particular order, these would be Google, BBC News, Technorati, Gartner, eWeek, Twitter, LinkedIn and Facebook. Now my list is probably different to yours, but all of these sites are themselves very popular. I have the freedom to visit any site on the web, yet in practice I follow the crowd.
But that is only part of the picture. Of all the sites I mentioned, I use two mainly as portals (Google and Technorati). And through them I visit a large number of sites not even mentioned here. But I may only visit them once, and possibly only for a few seconds. This isn’t terribly unusual behaviour of course. When I visit a small number of sites a large number of times, and a large number of sites a small number of times, I’m just providing my own personal example of a a power-law distribution.
You can see power-law distributions all over the Internet. From the perspective of web site owners, there are a small number of hugely successful websites and a huge number of very unsuccessful websites (if looked at in terms of number of hits). You won’t be in the first category as a part-time individual, and you are more than likely to be in the second category.
But there are of course websites in the happy middle. Those with a few hundred or perhaps a few thousand hits a day consistently over time. Looking at it purely mathematically there are a couple of ways to get to that status. Either get a lot of people to visit your site infrequently, or a smaller number of people very frequently. In reality, a large number of infrequent visits doesn’t seem like much of a recipe for success. It suggests that your website provides little value to its visitors, or they would be returning. So returning visitors are the key. To have a successful website, all you have to do is attract them. And to do that, you have to be consistently interesting to a large enough group of people.
This, I believe is ultimately the problem. Unless I become famous, only a small number of family and friends will be interested in what I have to say just because of who I am. Most of those people are more interested in hearing what I’ll tell them on Facebook or Twitter - that I’m buying a new car, headed to the pub, or visiting a mutual friend. By contrast my prolonged semi-professional musings are typically on topics that dont interest my family and most of my friends in the slightest. A significantly larger number of people may be interested in a particular blog post, but unless I remain tightly focused on that specific area over time, I’ll lose most of them too. Which leaves this blog where it is destined to remain, largely unread.
Do I care about this? In one sense yes. Many of the greatest minds in history were magnificent generalists. Da Vinci, Newton and Darwin are just three that spring to mind. That sort of generalization is impossible today, but individuals expressing themselves across a wide range of subjects on the Web will typically be lost in the noise, with nobody listening to their ideas. Specialization and only specialization brings an audience. And with it, the audience itself becomes more specialized - watching only the channels that interest them on TV, visiting only the websites that conform to their view of the world. On the web, you can find anything, but in reality you only find what you are looking for. It’s sad, but its probably inevitable.
Great post on this, with specific company examples, and suggested action items
Good post on consuming design patterns from Dwayne Taylor. I have co-authored several patterns with Dwayne, and he is a very smart guy.
This column marks the end of an era of sorts for me at Server Management magazine. After more than five years of security columns for the magazine, its time to pass the baton over to one of my colleagues. Nigel Stanley will be taking over the role of Security Editor, and I know he will bring a fresh perspective to what continues to be one of the most important areas of IT. As for me, I’ll be fulfulling a new role here, that of Enterprise Strategy Editor. My regular column will cover both Enterprise Architecture and Strategic Planning issues, and I’m very much looking forward to it.
My first security column was written in April 2002, just a few months after the 9/11 attacks. Up until then, many companies had just been thinking about IT security in terms of targeted hacker attacks, viruses and worms. An IT security department (assuming that such a thing even existed) had to mitigate those specific threats and respond to an attack if it occured. If it got that right, its job was pretty much complete. Probably the fundamental change that happened immediately after 9/11 was companies thinking about IT security in terms of business continuity. Companies began to seriously ask themselves the question “what will happen if I lose a building, or there is a regional disaster?” Depending on the size of the organizations, this may or may not be dealt with directly by an IT security department, and in some organizations IT security is only peripherally involved, but probably the most important thing that emerged from this shift in mindset was that IT security departments began to expand their horizons and thinking somewhat.
Now, at the end of 2008, I’m seeing really two types of IT Security Departments - those that have grown into mature adult centers of excellence, and those that are still wrestling with teenage angst.
“Teenage” security departments generally have a number of common themes. They are very reactive in nature, and while most will rely on security bulletins to keep abreast of the latest threats, some may only become aware of them when there is a major incident. IT security in this type of organization is really about keeping the lights on, and the team is doing the best that it can to respond to issues with the staffing they have. Frequently managers in these types of departments complain of being understaffed. Another problem that these security departments have is being rewarded for their own success. If things “go quiet” they may see their budget be trimmed, as the business believes that threats have been mitigated. Over time, budgets swing like a pendulum as security becomes more or less important for the business, or as particular projects are funded. The business also has difficulty in understanding exactly what IT security does, and sees it as a cost center that doesn’t provide any real value in its own right. In some cases, the perception of IT security is negative across the whole company. Project managers see IT security audits as something that adds cost and provides little value, and end users see IT security measures as ways of making their lives more complex, to the detriment of their productivity.
By contrast “adult” security departments have got their act together. They are proactive and strategic in nature - typically spending as much time thinking about tomorrow’s battles as fighting today’s and yesterday’s. Their staffing is stable and sufficient to meet the challenges they face (even though in reality it may be no larger than the teenage security department described above. Their funding is also stable, and they are seen as a partner to the business in addressing the challenges that it faces. The work of the IT security group is well communicated throughout the organization, and all users clearly see the value that IT security is providing. In many cases, users are proactive about contributing to the overall security of the IT environment.
Obviously if your IT security department is a teenager, it needs to grow up. Fast. But how do you get there?
In every case where I’ve seen the transition work well, there has been one common theme. Senior IT management has made it a priority to get IT Security intimately involved in the business. This is important on both sides of the relationship. If the business doesn’t understand IT security, its likely to cut its budget and not help IT security meet its goals. But at least as critical - if IT Security doesn’t understand the business it cannot hope to be truly successful. The role of the IT security department should never be to have the maximum security possible for the budget available. Instead it should be to provide the appropriate levels of security for the needs of the organization. It is not intrinsically right or wrong to implement a particular security mechanism. The decision should be based on a thorough understanding of the needs of the business. I’m still seeing that some in IT fundamentally don’t get this. For example I’ve been asked questions like “should I implement a network intrusion detection system?” without any other supporting information. The implication here is that there is a technically right or wrong answer, and that is the only thing that matters. Once you start thinking about the needs of the business, it becomes clear how impossible it is to answer that question.
When IT Security Departments are closely involved with the business, great things can start to happen. IT Security plays a critical role in educating the business in the risks associated with funding or not funding particular initiatives, and the well educated business leaders can than make a decision based on the risk they are willing to absorb. This is a much more neutral position than is present in many companies, but it recognizes that these decisions are not black and white technology issues. The analogy I typically use is that of a buglar alarm. The burglar alarm companies tell my how their product will protect my house, but its my business decision as to whether I want to spend the money in buying the alarm.
A good understanding of the business also helps the IT security department understand how to take a position on new technology that enters the business. As an example, imagine a new service is being implemented that monitors the files users are accessing on their local drives and file servers, and generates consolidated reports for business managers. Two SaaS vendors are being evaluated. In each case the file type is sent across a secure channel, along with the user that accesses it. This would allow you to see for example that Joe Bloggs is accessing lots of MPEG files. But one of the SaaS vendors offers additional functionality - it allows you to drill down further and see the specific file that the user is accessing. Is this additional functionality a good thing or a bad thing? Well, it depends. On the one hand, the company might want to know if particular sensitive files are being viewed by its users. On the other hand the company may be more worried about legal exposure - that in a lawsuit a rival company may request a litigation hold on the SaaS data. Only a well educated IT security department that is a partner with the business will understand which of those two things is more important.
Once the IT security department and the business are working more closely together, it will be easier to make the business understand the value in IT security, and in in making IT security more proactive. Ultimately this involves making sure that at least some in your IT security organization are involved in thinking strategically about problems, and in preparing the organization for change. Change of course can take many forms. There may be a change in the way threats emerge, or in vulnerabilities that are exposed. There may also be change in the way security measures are deployed, or in security measures that are available. But at least as important are changes in the overall IT environment - changes that mean different things must be protected, and existing defenses are no longer adequate. I’ve been referring to some of these, including smarter devices on the internal network, virtualization and cloud computing in recent columns. The last of these trends, the shift of IT resources to the cloud, is the one area that I think will fundamentally transform the way that we approach IT security. When services are provided out of the cloud, you will have less control over them and their security. Instead your focus will need to be on understanding what is being provided by the cloud, and ensuring that it meets with the security requirements of your organization.
Of course in planning for the future of your IT security department, you will make some mistakes. I’ve certainly made quite a few in writing this column. I remember writing with some confidence a few years ago about the death of the password, but it is still still stubbornly alive (even though I still firmly believe it should not be). I was even fairly bullish about Microsoft Palladium taking off - some of you may not even remember that one. Ultimately some of the work there ended up giving us Bitlocker, so perhaps I was not that far off the mark. But one prediction that I’m confident about is that IT security will continue to be a very important area, and jobs in IT security will continue to be interesting, varying and challenging. If you have been a regular reader of this column, thank you, and I hope you continue to enjoy reading it under new ownership.
“This is your last chance. After this, there is no turning back. You take the blue pill - the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill - you stay in Wonderland and I show you how deep the rabbit-hole goes.” From The Matrix - 1999
Many of you will remember “blue pill” malware from when it first came to prominence in 2006, and more recently – when the NBP (new blue pill) also got some significant attention. Every so often I’m truly surprised by the attention certain potential security attacks get, and how unrelated that attention is to the true nature of a threat. But in the case of “Blue Pill” malware – it is hardly surprising. The malware references one of the more popular film choices of computer geeks and sci-fi addicts, and the consequences of such an attack are potentially devastating.
Over the last few months I’ve been increasingly asked if the blue pill threat is real and if so what the implications are. So this month, I want to address the blue pill, and look at what the broader implications are of this type of threat.
Blue Pill malware was developed by Joanna Rutkowska in 2006, and at the time several newspaper articles predicted imminent doom and gloom for the computer industry. The malware was originally an attack against Windows Vista, but what interested many was less the target, and more the nature of the malware itself. “Blue Pill” malware takes the form of a small hypervisor, similar in nature to VMWare ESX or Microsoft Hyper-v. Like those hypervisors, the malware takes advantage of the virtualization capabilities built into the x86 chipset (in this case specifically the AMD chip). But unlike a conventional hypervisor – this malware “inserts” itself below the existing operating system unknown to any person or any detection software. From that point on the operating system is unwittingly running virtualized, and the malware can perform various nefarious activities theoretically without any fear of detection.
Various claims have been made about such a blue pill – including that it would be completely impossible to detect. Later that claim has been revised somewhat to suggest that it would be extremely difficult to detect. Recently, I’ve spent some time looking at the various publications, blog posts and articles surrounding blue pill, including one of the more recent presentations that Joanna Rutkowska presented on the NBP (New Blue Pill) and its consequences at http://tinyurl.com/4mkhzx). On reflection my view is that losing sleep over blue pill malware is pretty much a waste of time. There are a number of reasons for this – some of them are related to the Blue Pill malware itself, and others relate to some general concepts related to this type of threat.
One of the main worries about Blue Pill is the claim that it is very difficult to detect. Whether that is true or not really depends specifically what it is you are trying to detect. If you are attempting to answer the question “Is blue pill malware running on my system?” then I would agree that detection is difficult. If on the other hand you are trying to detect a hypervisor running on a computer – that isn’t difficult at all. (You basically use an external time source). But the real question you need to be able to answer to address the problem is “is an unauthorized hypervisor running on my system”. And that question is actually fairly easy too. “Normal” hypervisors should make no attempt to conceal themselves, so if you know you are running a hypervisor, and you know you shouldn’t be, or you cannot identify it, then you have an unauthorized hypervisor on your hands, and it’s time to deal with it.
The New Blue Pill supports “nesting”. In other words you install a hypervisor on top of NBP (and that hypervisor can even be NBP). Now this does affect the ability to determine from your guest operating system that something unusual is going on. But no matter how much nesting you do, at some point you are at the lowest “legitimate” hypervisor, and from there it would be possible to check for unauthorized hypervisors.
The reality is that blue pill type malware has never been used for any successful attack that we know of, and my opinion is that many of the more extravagant claims about Blue Pill are just plain unfounded. But just for a moment – let’s assume that they are not. What if this malware truly was undetectable, or even if it was extremely difficult to detect that you had undesirable malware on a system? (For example, if it would require an initial understanding that the malware existed, and then a dedicated, engineered solution, based on the specific characteristics of that malware).
What then?
Well, I don’t think anyone would doubt that the possibility is alarming. In the Blue Pill case Rutkowska would argue that the malware takes advantage of a particular characteristic of the AMD chipset (and similar characteristics of the Intel chipset), namely hardware virtualization technology. So a possible solution would be for AMD and Intel to remove this technology and resort to a more traditional chipset. But is this really practical? Sure, it would deal with this specific problem, but if NBP represents a new type of threat, surely there must be other examples of “undetectable” malware, ones that do not require hardware virtualization. And those other examples may not be advertised at a Black Hat conference.
So the irony is that it probably doesn’t really change anything. We can be alarmed, but if undetectable malware does exist, there is essentially nothing you can do about it, and so nothing to do to prepare for it. Even extremely difficult to detect to detect malware, such as the type I already described, can only really be dealt with at the time it is discovered.
What I am certainly prepared to concede is that this whole discussion should make us think further about the nature of malware detection, and whether current approaches to tackling the problem of malware are sufficient. I think it is very likely that the next few years will bring significant additional changes in the way malware is handled in the future, both in terms of the technology used to detect and combat it, and where that technology is implemented. This change will likely be related to an increasing trend to embed virtualization in the hardware, and the likelihood that the vast majority of both server and desktop workloads will run virtualized.
Many will disagree with my analysis, and one of the most interesting parts of the whole Blue Pill controversy is that if you read the blogs, you will see that much of that disagreement is based on something other than fact. I believe it stems from two main characteristics of blue pill malware. 1) Blue Pill is difficult to understand and 2) the consequences of a successful Blue Pill attack are truly scary. In cases where a lack of understanding and a fear combine, then logical argument and scientific debate frequently take a back seat to belief and faith. As an analogy, think of the number of people who do or do not “believe” in Global Climate Change. Or Evolution.
So, are there concrete lessons that can be learned here? I believe so. Security Planners need to constantly remind themselves that humans are naturally biased towards devoting resources towards preventing low risk threats with a potentially high impact, particularly if they have been exposed to the potential consequences. Think how much we spend preventing another 9/11 type attack versus lowering the number of preventable deaths from cancer. Using a statistically based risk analysis procedure should definitely help combat this problem.
Secondly, even if (or perhaps especially if) a threat is very difficult to understand – it essential to get a good high level and accurate assessment of the risk it poses and precisely what is at risk. Many of the questions I have received about Blue Pill have come from the assumption that somehow this malware reduces the security of virtualization. The nature of the malware itself (a hypervisor) has led people to believe that hypervisors themselves are inherently insecure. But just because the malware itself is uses a form of virtualization, that doesn’t say anything explicit about the security of virtualization itself. Indeed, you could argue (and interestingly) Rutkowska herself does argue) that modern hypervisors such as VMWare ESX and Hyper-v actually reduce the risk of this kind of attack. Certainly, not virtualizing in your environment does nothing to mitigate any risk.
Finally, even if the risk is there, it is only useful to plan for events that you can in some way affect or respond to. The sun could fail to come up tomorrow, but nothing I do today will change that outcome. And if that happens, I won’t be around much longer anyway. So it’s probably a waste of my time worrying about it.
I’m becoming an increasing fan of Google Trends. Enter any search term, and provided it is popular enough, you will see a graph showing the relative popularity of a search term over time, and the significant events in that period – rather like the graphs you see for stock prices.
I can imagine a lot of marketing people spend a significant amount of time on this site, but it can be pretty interesting for the rest of us. Recently I did a search for the term “Cloud Computing”. If you do a search on the site, you’ll notice that the x axis (the Search Volume Index) on this graph is not an absolute number, and I’ll leave the specifics of how it is calculated to the Google Trends website, but basically the index is an indication of how popular the search term is at a particular time, relative to the average over the full time period measured. A Search Volume Index of 2 means that double the average number of searches for that term are being performed.
This Google Trends graph indicates something that will probably not be much of a surprise to many of you – Cloud Computing is a very hot topic right now. (To put it into perspective – search on the term “Barack Obama” and you will see a peak Search Volume Index of around 10). So, this month I’m going to spend some time looking at security considerations with regard to Cloud Computing.
Probably the first question I get asked any time I talk about Cloud Computing is “what exactly is it?” We are notoriously bad at defining things in the Computer Industry, in part because journalists and analysts and companies continuously redefine terms and introduce new ones that mean almost the same thing. So one could argue that Cloud Computing is just the new snazzy term for SaaS, and SaaS was just the new snazzy term for ASPs. One could argue that, but I think its missing the point somewhat. Cloud Computing certainly can encompass SaaS (and more traditional ASPs), but it’s really an umbrella term that covers those and more. The key differentiator of Cloud Computing is its flexibility – you can easily scale up and scale down your use of Cloud Computing offerings according to your requirements. In fact, for many IT Leaders I’ve heard from, the most attractive promise of Cloud Computing is a move towards utility computing – where computing resources are provided in much the same way that gas or electricity is provided today – metered according to use. (Incidentally, the Search Volume Index of “utility computing” on Google Trends peaks at around 100 - now that is a hot term.
Regardless of whether we call the term Cloud Computing in a few years time, it is clear that there is a medium term trend towards increased outsourcing, and what is being outsourced is changing. Previously companies would have retained ownership of their services, but outsourced some or most of the staff needed to support that service. Now they are going a step further and outsourcing the entire service, and may use different outsourcing providers for each service (rather than a single staffing provider). The bottom line is that the entire model of how IT services are being provided to customers is changing, and increasingly the IT department is acting as a broker between a number of external companies and their own internal customers.
There is nothing about Cloud Computing that makes it inherently less secure. In fact I’ve seen some analysts argue that a Cloud Computing model has the potential to be more secure than more traditional IT models, but there is no doubt that it represents a fundamental shift in the way IT is provided, and with it, a different set of risks that IT security professionals must deal with.
Probably the greatest concern for security professionals is the cloud itself. In other words, what is inside the cloud? When a service is outsourced to the cloud, you lose direct control of it, and with it, the ability to directly ensure that the service is secure. Your information is frequently residing in a shared data center, and may even be alongside that of your direct competitors. Taking on trust that the cloud computing environment is secure is not a risk that many security teams are prepared to take, particularly with smaller cloud computing providers. You need to focus on understanding the security that is provided, and determining if it is sufficient to meet the needs of your organization. As a starting point, I’m increasingly seeing companies ask providers to complete a security questionnaire, so that they can at least get an understanding on the security practices used by the other company. The questionnaire would deal with key questions such as how they authenticate your users, and prevent unauthorized users from gaining access, how they separate your data from other organizations that share the infrastructure, how they meet government regulatory requirements, how they ensure business continuity in the event of a disaster, and how they support any legal need you may have to investigate your own data (for example to defend a lawsuit).
This type of questionnaire allows you to get the appropriate initial information to see what issues need to be followed up on, and some of the information contained within it may form the basis of the contract you sign with the service provider. Increasingly I’m seeing that cloud computing providers are asking third party security firms to provide an independent analysis of their security practices. I think this is a very good move, as they can safely provide those firms with more detail than they can your company (just think how much detail you want them to provide your competitors), and a dedicated security firm is likely to have much more specialized expertise in assessing their security practices than you do.
Assuming you are happy with the security practices of the Cloud computing provider itself, there is still the question of the network connectivity to the provider. Of course, typically the connection is over the Internet, and uses SSL to secure the data, and in most cases this is sufficient, but it is not always the only option. For more highly secure data, you may be able to set up a dedicated VPN type connection between your data center and the provider. There may also be an option where the Cloud Computing provider, typically through an arrangement with a third party provider, is able to provide you with a local point of presence that you can connect more directly to. This may be to improve performance, security, or both.
Another key thing to consider, particularly in larger companies, is the possibility of IT Security procedures in your organization being skipped entirely. Unfortunately, there is often a perception in some companies, that if a service is entirely outsourced, then IT does not need to be involved at all in its implementation. I’ve seen a number of instances recently where an individual business group in an organization will sign a contract with a SaaS provider or an ASP directly, and only later does IT become aware of it. At that point, all the best laid plans you have provided will come to naught.
Security is only part of the reason that IT must be involved in the decision to outsource an IT service. Performance is another. For example, will your network meet the requirements (in terms of speed and latency) for the application in question, and will the new service you are outsourcing mean that the network has to be upgraded to meet all its other requirements. A third reason is cost. If you already provide a similar service, or could expand the service to meet the requirements of the business, outsourcing the service may not make financial sense.
When you think about it, it’s not that surprising that the IT department frequently gets skipped. From a business unit perspective, if you have made the decision to go with an external provider, the only thing that IT will do is introduce unnecessary time delays and costs. So a key part of ensuring good security with Cloud Computing is educating the rest of the company on why IT needs to be involved even in services that are outsourced, and providing sufficient governance controls, backed by senior management, in order to ensure compliance.
The likelihood is that in the immediate future, Cloud computing environments are likely to become more complex rather than less so. For example, providers are likely to themselves partner up or outsource. For example, an e-mail service provider may outsource the network to another company, and the data center itself to a third. So understanding what is going on inside the cloud may become ever trickier. I think the ultimate solution will be some sort of standardized security certification for Cloud Computing providers. If this is sufficiently rigorous, then much of the worry for security professionals will go away.
In the meantime, it’s important to remember that while there may be problems today in fully assuring yourself of the security of a Cloud Computing provider, there is nothing intrinsic about a Cloud Computing that makes it a less secure option. If you focus on understanding and managing the security measures that are implemented by each provider, you to take advantage of some of the benefits that a cloud computing solution can bring, and prepare yourself for the seemingly inevitable flood of cloud computing offerings that will be offered in the near future.
