Integrity: Letting Your Enemies Enhance Your System

Protecting secrets within a computer system is hard when faced with malicious software. Protecting the integrity of critical systems and data is arguably very much harder. To maintain confidentiality, the system can be physically isolated with only authorized people permitted physically access to it, as is done in “system high” or “dedicated mode” processing.   On such a physically isolated system, software of dubious origins can be installed and executed without fear that the system will pass secrets to unauthorized users because their are none.  But, if the integrity of data is of concern, then one must think about who wrote the software you are running, and who provided all of the data that enters your system.   The player can ensure that only authorized people physically access high integrity systems, but, if so designed, the application and operating system software of unknown origin can modify data independent of the desires of the authorized users. Who wrote the software for the laptop? Who wrote the software in critical military systems? Techniques that counter subversion of secrecy policies (e.g., physically separate networks) are not always sufficient for countering the subversion of integrity policies.

For a relatively modest investment, an enemy can design, implement and field software to share in the control of a variety of different critical systems ranging from aircraft carriers to water treatment plants to electrical power distribution facilities.  The problem is not one of  “software safety” or “software reliability”.  A serious threat is deliberate design choices made by adversaries to alter the behavior of the system. Briefly consider the fundamentals of means, opportunity and motive:
  • Means:  writing compact, obscure, data-driven software that performs complex operations is a fairly common skill.
  • Opportunity:  commercial off-the-shelf software.  Go to work for a vendor and implement your own features. A recent version of the Excel spreadsheet program contains a flight simulator game triggered by an obscure keystroke sequence. No amount of testing performed by the vendor or the buyer could detect a small, well crafted, and obfuscated artifice.
  • Motive:  we hardly need reminding that there are relatively sophisticated groups willing to invest a lot to cause massive damage.
It may be that critical systems are controlled by carefully designed software built by reliable programmers.  But in many cases this high integrity software shares the network with software that literally could have come from anywhere. The most egregious examples are high integrity critical programs that run on top of bloated commercial operating systems having no end of security flaws and tens of thousands of different authors employed by scores of different enterprises.  Who wrote these operating systems? The game will include scenarios that illustrate the risks and countermeasures to the subversion threat to integrity policies.  These risks are mitigated by use of high assurance integrity enforcement mechanisms together with the use of high integrity applications for the input and modification of high integrity data..

Some take comfort in the following argument 1 :
The Department [of Defense] acquires most of its information systems from vendors providing commercial off-the-shelf (COTS) products.   Consequently, the Department has little or no knowledge of who developed the systems and, therefore, no measure of the trustworthiness, reliability or loyalties of those individuals.   Contrariwise, individual developers of COTS products who have malicious intentions would have an extraordinarily difficult task to target a particular customer because COTS products tend to be produced in large quantities and shipped to customers as an activity that is independent of the individual developer.   The developer with malicious intentions would have to deliver the same product to all customers while retaining the ability to isolate a particular customer for exploitation.
This “extraordinarily difficult task” is not really so difficult.  For example, a trap door can have a unique trigger (e.g., a unique character sequence) that ensures the malicious behavior is only active on the target systems.   Delivery of such a unique trigger (along with detailed control data) is made much easier by our willingness to let information “flow up” without regard to a well formed integrity policy.   For example, for all practical purposes, the Internet and the many “separate and secure networks” exist within the same integrity domain.   Much is made about military networks and other critical infrastructure networks being “isolated” from the Internet.    For the most part this is simply not true.   A variety of gateways exist that permit data (e.g., e-mail) to flow from less secure (e.g., the DoD NIPRNET) to more secure (e.g., the DoD SIPRNET) networks.  Even within a truly isolated environment, delivery of a unique targeted trigger would not be “extraordinarily difficult”.  For example, the enemy can introduce a new code word into data traffic that is known to be intercepted and/or processed by the targeted system.   And if all that still seems “extraordinarily difficult” the enemy can install a simple time bomb that renders all systems useless at a particular date.  And most attempts to recover by changing the system date would fail because all bootable media would include the same time bomb.

 1  DoD Insider Threat Mitigation, April 24, 2000    http://www.defenselink.mil/nii/org/sio/iptreport4_26dbl.doc

Tutorials