Virtual-machine-based rootkits could be used to compromise virtual operating systems

Researchers at Microsoft and the University of Michigan (UOM) have created virtual-machine-based rootkits (VMBRs) to demonstrate how the security of virtual operating systems could be compromised.

This news may alarm companies using virtualisation to consolidate services onto commodity hardware with higher CPU utilisation rates; or firms using virtualised desktop operating systems to tackle security problems.

The research staff assumed "the perspective of the attacker who is trying to run malicious software (malware) and avoid detection", according to their paper entitled SubVirt: Implementing malware with virtual machines, which has been conditionally accepted for the 2006 IEEE Symposium on Security and Privacy, which will be held in May.

Brian Gammage of analyst company Gartner issued a warning at Intel's Digital Office initiative in October that virtualisation could create new security weaknesses. A VMBR would operate below the virtual operating system, effectively controlling it.

In their paper, the researchers give details of the implementation of two proof-of-concept VMBRs, one aimed at a Linux/ VMWare system, the other at a Windows XP/VirtualPC system. To complement these VMBRs the researchers developed malicious systems including a keystroke sniffer, a phishing web server, and a data probe for finding sensitive data. They also created a countermeasure to foil the "redpill" method for detecting virtual machines.

To detect VMBRs, the researchers suggested the best way is to take control at a lower level than the VMBR. This would mean detection through a low-level security chipset – a method already proposed by processor vendors Intel and AMD – or booting from "sandboxed" media such as CD-ROMs or USB keys.