Malicious Code

What is Malicious Code?

Malicious code (also called vandals) is a new breed of Internet threat that cannot be efficiently controlled by conventional antivirus software alone. In contrast to viruses that require a user to execute a program in order to cause damage, vandals are auto-executable applications.
Malicious code can take the form of:

• Java Applets
• ActiveX Controls
• Scripting languages
• Browser plug-ins
• Pushed content

Once inside your network or workstation malicious code can enter network drives and propagate. They can also cause network and mail server overload by sending email messages, stealing data and passwords, deleting document files, email files or passwords, and even re-formatting hard drives.
Malware, short for malicious software, is software designed to infiltrate a computer without the owner's informed consent. The expression is a general term used by computer professionals to mean a variety of forms of hostile, intrusive, or annoying software or program code. The term "computer virus" is sometimes used as a catch-all phrase to include all types of malware, including true viruses.
Software is considered malware based on the perceived intent of the creator rather than any particular features. Malware includes computer viruses, worms, trojan horses, most rootkits, spyware, dishonest adware, crimeware and other malicious and unwanted software.
Malware is not the same as defective software, that is, software which has a legitimate purpose but contains harmful bugs.

Preliminary results from Symantec published in 2008 suggested that "the release rate of malicious code and other unwanted programs may be exceeding that of legitimate software applications. According to F-Secure, "As much malware was produced in 2007 as in the previous 20 years altogether. Malware’s most common pathway from criminals to users is through the Internet: primarily by e-mail and the World Wide Web
The prevalence of malware as a vehicle for organized Internet crime, along with the general inability of traditional anti-malware protection platforms to protect against the continuous stream of unique and newly produced professional malware, has seen the adoption of a new mindset for businesses operating on the Internet - the acknowledgment that some sizable percentage of Internet customers will always be infected for some reason or other, and that they need to continue doing business with infected customers. The result is a greater emphasis on back-office systems designed to spot fraudulent activities associated with advanced malware operating on customers computers.

Purposes

Many early infectious programs, including the first Internet Worm and a number of MS-DOS were written as experiments or pranks generally intended to be harmless or merely annoying rather than to cause serious damage to computers. In some cases the perpetrator did not realize how much harm their creations could do. Young programmers learning about viruses and the techniques wrote them for the sole purpose that they could or to see how far it could spread. As late as 1999, widespread viruses such as the Melissa virus appear to have been written chiefly as pranks.
Hostile intent related to vandalism can be found in programs designed to cause harm or data loss. Many DOS viruses, and the Windows ExploreZip worm, were designed to destroy files on a hard disk, or to corrupt the file system by writing invalid data. Network-borne worms such as the 2001 Code Red worm or the Ramen worm fall into the same category. Designed to vandalize web pages, these worms may seem like the online equivalent to graffiti tagging, with the author's alias or affinity group appearing everywhere the worm goes.
However, since the rise of widespread broadband Internet access, malicious software has come to be designed for a profit motive, either more or less legal (forced advertising) or criminal. For instance, since 2003, the majority of widespread viruses and worms have been designed to take control of users' computers for black-market exploitation. Infected "zombie computers" are used to send email spam, to host contraband data such as child pornography, or to engage in distributed denial-of-service attacks as a form of extortion.
Another strictly for-profit category of malware has emerged in spyware -- programs designed to monitor users' web browsing, display unsolicited advertisements, or redirect affiliate marketing revenues to the spyware creator. Spyware programs do not spread like viruses; they are generally installed by exploiting security holes or are packaged with user-installed software, such as peer-to-peer applications.

Infectious malware: viruses and worms

The best-known types of malware, viruses and worms, are known for the manner in which they spread, rather than any other particular behavior. The term computer virus is used for a program which has infected some executable software and which causes that software, when run, to spread the virus to other executable software. Viruses may also contain a payload which performs other actions, often malicious. A worm, on the other hand, is a program which actively transmits itself over a network to infect other computers. It too may carry a payload.
These definitions lead to the observation that a virus requires user intervention to spread, whereas a worm spreads automatically. Using this distinction, infections transmitted by email or Microsoft Word documents, which rely on the recipient opening a file or email to infect the system, would be classified as viruses rather than worms.
Some writers in the trade and popular press appear to misunderstand this distinction, and use the terms interchangeably.

Capsule history of viruses and worms

Before Internet access became widespread, viruses spread on personal computers by infecting programs or the executable boot sectors of floppy disks. By inserting a copy of itself into the machine code instructions in these executables, a virus causes itself to be run whenever the program is run or the disk is booted. Early computer viruses were written for the Apple II and Macintosh, but they became more widespread with the dominance of the IBM PC and MS-DOS system. Executable-infecting viruses are dependent on users exchanging software or boot floppies, so they spread heavily in computer hobbyist circles.

The first worms, network-borne infectious programs, originated not on personal computers, but on multitasking Unix systems. The first well-known worm was the Internet Worm of 1988, which infected SunOS and VAX BSD systems. Unlike a virus, this worm did not insert itself into other programs. Instead, it exploited security holes in network server programs and started itself running as a separate process. This same behavior is used by today's worms as well.
With the rise of the Microsoft Windows platform in the 1990s, and the flexible macro systems of its applications, it became possible to write infectious code in the macro language of Microsoft Word and similar programs. These macro viruses infect documents and templates rather than applications, but rely on the fact that macros in a Word document are a form of executable code.

Today, worms are most commonly written for the Windows OS, although a small number are also written for Linux and Unix systems. Worms today work in the same basic way as 1988's Internet Worm: they scan the network and leverage vulnerable computers to replicate.

Concealment: Trojan horses, rootkits, and backdoors

Trojan horses

For a malicious program to accomplish its goals, it must be able to do so without being shut down, or deleted by the user or administrator of the computer via which it is running. Concealment can also help get the malware installed in the first place. When a malicious program is disguised as something innocuous or desirable, users may be tempted to install it without knowing what it does. This is the technique of the Trojan horse or trojan.
Broadly speaking, a Trojan horse is any program that invites the user to run it, concealing a harmful or malicious payload. The payload may take effect immediately and can lead to many undesirable effects, such as deleting the user's files or further installing malicious or undesirable software. Trojan horses known as droppers are used to start off a worm outbreak, by injecting the worm into users' local networks.

One of the most common ways that spyware is distributed is as a Trojan horse, bundled with a piece of desirable software that the user downloads from the Internet. When the user installs the software, the spyware is installed alongside. Spyware authors who attempt to act in a legal fashion may include an end-user license agreement which states the behavior of the spyware in loose terms, and which the users are unlikely to read or understand.

Rootkits

Once a malicious program is installed on a system, it is essential that it stay concealed, to avoid detection and disinfection. The same is true when a human attacker breaks into a computer directly. Techniques known as rootkits allow this concealment, by modifying the host operating system so that the malware is hidden from the user. Rootkits can prevent a malicious process from being visible in the system's list of processes, or keep its files from being read. Originally, a rootkit was a set of tools installed by a human attacker on a Unix system where the attacker had gained administrator (root) access. Today, the term is used more generally for concealment routines in a malicious program.

Some malicious programs contain routines to defend against removal: not merely to hide themselves; but to repel attempts to remove them. An early example of this behavior is recorded in the Jargon File tale of a pair of programs infesting a Xerox CP-V timesharing system:
Each ghost-job would detect the fact that the other had been killed, and would start a new copy of the recently slain program within a few milliseconds. The only way to kill both ghosts was to kill them simultaneously (very difficult) or to deliberately crash the system.
Similar techniques are used by some modern malware, wherein the malware starts a number of processes which monitor and restore one another as needed.

Backdoors

A backdoor is a method of bypassing normal authentication procedures. Once a system has been compromised (by one of the above methods, or in some other way), one or more backdoors may be installed in order to allow easier access in the future. Backdoors may also be installed prior to malicious software, to allow attackers entry.
The idea has often been suggested that computer manufacturers preinstall backdoors on their systems to provide technical support for customers, but this has never been reliably verified. Crackers typically use backdoors to secure remote access to a computer, while attempting to remain hidden from casual inspection. To install backdoors crackers may use Trojan horses, worms, or other methods.

Malware for profit: spyware, botnets, keystroke loggers, and dialers

During the 1980s and 1990s, it was usually taken for granted that malicious programs were created as a form of vandalism or prank. More recently, the greater share of malware programs have been written with a financial or profit motive in mind. This can be taken as the malware authors' choice to monetize their control over infected systems: to turn that control into a source of revenue.
Spyware programs are commercially produced for the purpose of gathering information about computer users, showing them pop-up ads, or altering web-browser behavior for the financial benefit of the spyware creator. For instance, some spyware programs redirect search engine results to paid advertisements. Others, often called "stealware" by the media, overwrite affiliate marketing codes so that revenue is redirected to the spyware creator rather than the intended recipient.

Spyware programs are sometimes installed as Trojan horses of one sort or another. They differ in that their creators present themselves openly as businesses, for instance by selling advertising space on the pop-ups created by the malware. Most such programs present the user with an end-user license agreement which purportedly protects the creator from prosecution under computer contaminant laws. However, spyware EULAs have not yet been upheld in court.
Another way that financially-motivated malware creators can profit from their infections is to directly use the infected computers to do work for the creator. The infected computers are used as proxies to send out spam messages. A computer left in this state is often known as Computer_zombie. The advantage to spammers of using infected computers is they provide anonymity, protecting the spammer from prosecution. Spammers have also used infected PCs to target anti-spam organizations with distributed denial-of-service attacks.

In order to coordinate the activity of many infected computers, attackers have used coordinating systems known as botnets. In a botnet, the malware or malbot logs in to an Internet Relay Chat channel or other chat system. The attacker can then give instructions to all the infected systems simultaneously. Botnets can also be used to push upgraded malware to the infected systems, keeping them resistant to antivirus software or other security measures.
It is possible for a malware creator to profit by stealing sensitive information from a victim. Some malware programs install a key logger, which intercepts the user's keystrokes when entering a password, credit card number, or other information that may be exploited. This is then transmitted to the malware creator automatically, enabling credit card fraud and other theft. Similarly, malware may copy the CD key or password for online games, allowing the creator to steal accounts or virtual items.

Data-stealing malware

Data-stealing malware is a web threat that divests victims of personal and proprietary information with the intent of monetizing stolen data through direct use or underground distribution. Content security threats that fall under this umbrella include keyloggers, screen scrapers, spyware, adware, backdoors, and bots. The term does not refer to activities such as spam, phishing, DNS poisoning, SEO abuse, etc. However, when these threats result in file download or direct installation, as most hybrid attacks do, files that act as agents to proxy information will fall into the data-stealing malware category.

• Characteristics of data-stealing malware
• Does not leave traces of the event
• The malware is typically stored in a cache which is routinely flushed
• The malware may be installed via a drive-by-download process
• The website hosting the malware as well as the malware is generally temporary or rogue
• Frequently changes and extends its functions
• It is difficult for antivirus software to detect final payload attributes due to the combinations of malware components
• The malware uses multiple file encryption levels
• Thwarts Intrusion Detection Systems (IDS) after successful installation
• There are no perceivable network anomalies
• The malware hides in web traffic
• The malware is stealthier in terms of traffic and resource use
• Thwarts disk encryption
• Data is stolen during decryption and display
• The malware can record keystrokes, passwords, and screenshots
• Thwarts Data Loss Prevention (DLP)
• Leakage protection hinges on metadata tagging, not everything is tagged
• Miscreants can use encryption to port data

Examples of data-stealing malware

Bancos, an info stealer that waits for the user to access banking websites then spoofs pages of the bank website to steal sensitive information.

Gator, spyware that covertly monitors web-surfing habits, uploads data to a server for analysis then serves targeted pop-up ads.

LegMir, spyware that steals personal information such as account names and passwords related to online games.

Qhost, a Trojan that modifies the Hosts file to point to a different DNS server when banking sites are accessed then opens a spoofed login page to steal login credentials for those financial institutions.

Data-stealing malware incidents
Albert Gonzalez is accused of masterminding a ring to use malware to steal and sell more than 170 million credit card numbers in 2006 and 2007 -- the largest computer fraud in history. Among the firms targeted were (BJ’s Wholesale Club, TJX, DSW Shoe, OfficeMax, Barnes & Noble, Boston Market, Sports Authority and Forever 21).
A Trojan horse program stole more than 1.6 million records belonging to several hundred thousand people from Monster Worldwide Inc’s job search service. The data was used by cybercriminals to craft phishing emails targeted at Monster.com users to plant additional malware on users’ PCs.

Customers of Hannaford Bros. Co, a supermarket chain based in Maine, were victims of a data security breach involving the potential compromise of 4.2 million debit and credit cards. The company was hit by several class-action law suits.

The Torpig Trojan has compromised and stolen login credentials from approximately 250,000 online bank accounts as well as a similar number of credit and debit cards. Other information such as email, and FTP accounts from numerous websites, have also been compromised and stolen.

Vulnerability to malware
In this context, as throughout, it should be borne in mind that the “system” under attack may be of various types, e.g. a single computer and operating system, a network or an application.

Various factors make a system more vulnerable to malware:

• Homogeneity – e.g. when all computers in a network run the same OS, if you can exploit that OS, you can break into any computer running it.
• Defects – malware leveraging defects in the OS design.
• Unconfirmed code – code from a floppy disk, CD-ROM or USB device may be executed without the user’s agreement.
• Over-privileged users – some systems allow all users to modify their internal structures.
• Over-privileged code – some systems allow code executed by a user to access all rights of that user.

An oft-cited cause of vulnerability of networks is homogeneity or software monoculture. For example, Microsoft Windows or Apple Mac have such a large share of the market that concentrating on either could enable a cracker to subvert a large number of systems, but any total monoculture is a problem. Instead, introducing inhomogeneity (diversity), purely for the sake of robustness, could increase short-term costs for training and maintenance. However, having a few diverse nodes would deter total shutdown of the network, and allow those nodes to help with recovery of the infected nodes. Such separate, functional redundancy would avoid the cost of a total shutdown, would avoid homogeneity as the problem of "all eggs in one basket".

Most systems contain bugs, or loopholes, which may be exploited by malware. A typical example is the buffer-overrun weakness, in which an interface designed to store data, in a small area of memory, allows the caller to supply more data than will fit. This extra data then overwrites the interface's own executable structure (past the end of the buffer and other data). In this manner, malware can force the system to execute malicious code, by replacing legitimate code with its own payload of instructions (or data values) copied into live memory, outside the buffer area.

Originally, PCs had to be booted from floppy disks, and until recently it was common for this to be the default boot device. This meant that a corrupt floppy disk could subvert the computer during booting, and the same applies to CDs. Although that is now less common, it is still possible to forget that one has changed the default, and rare that a BIOS makes one confirm a boot from removable media.

In some systems, non-administrator users are over-privileged by design, in the sense that they are allowed to modify internal structures of the system. In some environments, users are over-privileged because they have been inappropriately granted administrator or equivalent status. This is a primarily a configuration decision, but on Microsoft Windows systems the default configuration is to over-privilege the user. This situation exists due to decisions made by Microsoft to prioritize compatibility with older systems above security configuration in newer systems and because typical applications were developed without the under-privileged users in mind. As privilege escalation exploits have increased this priority is shifting for the release of Microsoft Windows Vista. As a result, many existing applications that require excess privilege (over-privileged code) may have compatibility problems with Vista. However, Vista's User Account Control feature attempts to remedy applications not designed for under-privileged users through virtualization, acting as a crutch to resolve the privileged access problem inherent in legacy applications.

Malware, running as over-privileged code, can use this privilege to subvert the system. Almost all currently popular operating systems, and also many scripting applications allow code too many privileges, usually in the sense that when a user executes code, the system allows that code all rights of that user. This makes users vulnerable to malware in the form of e-mail attachments, which may or may not be disguised.

Given this state of affairs, users are warned only to open attachments they trust, and to be wary of code received from untrusted sources. It is also common for operating systems to be designed so that device drivers need escalated privileges, while they are supplied by more and more hardware manufacturers.

Eliminating over-privileged code

Over-privileged code dates from the time when most programs were either delivered with a computer or written in-house, and repairing it would at a stroke render most antivirus software almost redundant. It would, however, have appreciable consequences for the user interface and system management.
The system would have to maintain privilege profiles, and know which to apply for each user and program. In the case of newly installed software, an administrator would need to set up default profiles for the new code.
Eliminating vulnerability to rogue device drivers is probably harder than for arbitrary rogue executables. Two techniques, used in VMS, that can help are memory mapping only the registers of the device in question and a system interface associating the driver with interrupts from the device.

Other approaches are:

• Various forms of virtualization, allowing the code unlimited access only to virtual resources
• Various forms of sandbox or jail
• The security functions of Java, in java.security

Such approaches, however, if not fully integrated with the operating system, would reduplicate effort and not be universally applied, both of which would be detrimental to security.

Anti-malware programs

As malware attacks become more frequent, attention has begun to shift from viruses and spyware protection, to malware protection, and programs have been developed to specifically combat them.
Anti-malware programs can combat malware in two ways:
They can provide real time protection against the installation of malware software on a computer. This type of spyware protection works the same way as that of antivirus protection in that the anti-malware software scans all incoming network data for malware software and blocks any threats it comes across.

Anti-malware software programs can be used solely for detection and removal of malware software that has already been installed onto a computer. This type of malware protection is normally much easier to use and more popular. This type of anti-malware software scans the contents of the windows registry, operating system files, and installed programs on a computer and will provide a list of any threats found, allowing the user to choose which files to delete or keep, or to compare this list to a list of known malware components, removing files that match.

Real-time protection from malware works identically to real-time antivirus protection: the software scans disk files at download time, and blocks the activity of components known to represent malware. In some cases, it may also intercept attempts to install start-up items or to modify browser settings. Because many malware components are installed as a result of browser exploits or user error, using security software (some of which are anti-malware, though many are not) to "sandbox" browsers (essentially babysit the user and their browser) can also be effective in helping to restrict any damage done.

Writing Secure Programs

The main focus of most writing about security is network security and physical security. Not much is written about writing secure programs. A lot of what you need to write secure programs is common sense, but due to time constraints and design shortcuts, it is rarely used. Any good programmer knows the concepts but usually does not have the time to implement them; there is a lot of pressure to produce a lot of code and get the project done.

In the early 1970s, the concept of structured programming was alive. Not only was the program structured but the whole project had structure; there were technical specifications, design specifications, detail design specifications, design walk-throughs and code walk-throughs. This made projects bigger and longer, but when finished, the code was debugged easily and often worked with few changes. Some of these projects took many years to produce. However, there was little external influence from networks, the Web and time to market.

Today, a lot of the structured development process has literally disappeared. But security starts with the design of the program or application and depends on coding standards established by the organization where the work is being done.

It is very unlikely that any code will be 100% secure; no code ever is. But, what can be done to make sure code is solid and secure? This article offers some ideas on what to consider and explains three tools to help write secure code. When designing and implementing an embedded system, more care is required for the coding. With the assistance of the tools in the Resources section, a lot of coding errors can be checked. The ultimate judge of secure code is left up to the implementer of the code and the ability of the implementer to understand what is secure.

Check for Errors

Every function returns some type of status, returned either directly or as part of errno. Checking these should be simple. In C++, the exception-handling capability is easy to use but can be complicated to set up. Exception handling has improved greatly over the past few years, once the C++ standard was finalized. When practical, it should be used. Previous practice had been to ignore errors, because it was thought the data being passed was valid. This has been proven to be a bad assumption.

Data buffer overflows have led to many security fixes in the past years. When writing for an embedded system, checking for error returns is important. Decisions need to be made about whether the error is benign and can be ignored. If the error is not benign, maybe it can be corrected. If it cannot be corrected, does the system perform a soft reset or a hard reset? In some cases a soft reset, causing the action in error to be restarted, is all that is required. This is the basis for some fault-tolerant systems. Depending on the type of device, a hard reset may not be a bad thing. Other times, some form of recovery is a must.

Looking for Strings

Instead of using sprintf, strcpy and strcat, use functions like strncpy and strncat. These functions make sure the buffer does not overflow and discard any excess. Do not use fgets when reading data, as this allows overflows. These may seem like simple changes, but they are easy to forget, as is string handling, one of the most exploited areas of programs. Automated test programs check for these problems quite nicely, but the tests can be misleading. Some uses of the string function may be flagged as a problem but prove to be fine in the context where used. This is where the ability and knowledge of the implementer plays an important role. The logs generated by the tools need to be scanned to determine what code has been flagged and needs to be changed.

Memory Leaks and Buffer Overflows

Memory leaks in and of themselves do not necessarily create security risks. However, they can be exploited if the memory is shared by several procedures and structures.

Buffer overflows are by far the most common security issue. If a buffer is allocated on the stack, it can be overflowed to wipe out or change the return address of a function. When a function returns then, it returns to the new address instead of to the proper address. Some buffer attacks also can occur on the heap. These are more difficult to create, but they still can be done. Programs written in C are most vulnerable to these attacks, but any language that provides low-level memory access and pointer arithmetic can be problematic. Pointer arithmetic is one area that should have bounds checking.

The GNU C compiler has an extension available, which needs to be included when the compiler is built, that implements bounds checking. It is used as an option that adds code to the program. During testing, the code can be turned on and used. During deployment the code would not be present. The reason the code should be turned off is it prints messages when the bounds are breached. If the system in place is a workstation, the messages can be left on, but an embedded system typically has no console.

An idea that might occur here is all the buffers should be statically allocated; then the problem goes away. In truth, the notion that a buffer is of fixed length can be exploited. The data being moved to the buffer still can be longer than the buffer. When it is moved it overflows, and the same problem happens. To lower the risk, the data movement should move up only to the maximum allowed for the buffer. Dynamic reallocation of strings permits programs to handle inputs of arbitrary sizes. The problem with this is the program could run out of memory. On an embedded system, such a mistake is fatal. On a workstation, the virtual memory system may start to thrash and create a performance bottleneck. In C++, the std::string class has the dynamic growth approach. If the class' data is turned into a char * pointer, a buffer overflow can happen. Other string libraries may not have these problems, but the implementer needs to be aware of the limitations.

Validate the Input

If a program is receiving data on which it must operate, there should be some type of validation that the data is correct, does not exceed the maximum size and is free of non-valid types. For instance, if the data is limited to uppercase letters from A to Z, the function should reject anything else. It also should check to make sure the length of the data is valid. Many years ago, everyone thought of data as 80 characters, the size of a punch card. Today, data literally can be any size; it can be text or binary or encrypted. It still has some type of limit though. This should be checked, and if it fails, reject it.

Not only should you check for the maximum size of a record or piece of data but, in some cases, check for a minimum size. Strings should be checked for legal values or legal patterns. If the data being checked contains binary data that needs to be kept that way, it may be better to use the common escape character to signal that the data is binary. If the data is numeric, range checking should be done. If it is any positive integer, check if it is less than zero. If there is a maximum value, check for that. The file limits.h defines the maximum and minimum values for most values, so it is easy to check for system limits.

Program Security

This chapter has covered development issues in computer security: the kinds and effects of security flaws, and in malicious code, and the techniques that can help to control threats. Malicious code receives a great deal of attention in the media; the colorful terminology certainly draws people to stories about it, and the large numbers of affected systems ensure that major malicious code attacks get very wide visibility. But it is important for us to realize that the seriousness of the threat and the degree of vulnerability should also cause people to pay attention.

The total amount of damage already done is not measurable, but it is certainly large. Many successful attacks go undetected—for now, at least. With the explosive growth in connectivity to massive public networks such as the Internet, the exposure to threats is increasing dramatically. Yet the public continues to increase its reliance on computers and networks, ignoring the obvious danger.

Software development is often considered a solitary effort; a programmer sits with a specification or design and grinds out line after line of code. But in fact, software development is a collaborative effort, involving people with different skill sets who combine their expertise to produce a working product. Development requires people who can

• specify the system, by capturing the requirements and building a model of how the system should work from the users' point of view

• design the system, by proposing a solution to the problem described by the requirements and building a model of the solution

• implement the system, by using the design as a blueprint for building a working solution

• test the system, to ensure that it meets the requirements and implements the solution as called for in the design

• review the system at various stages, to make sure that the end products are consistent with the specification and design models

• document the system, so that users can be trained and supported

• manage the system, to estimate what resources will be needed for development and to track when the system will be done

• maintain the system, tracking problems found, changes needed, and changes made, and evaluating their effects on overall quality and functionality

One person could do all these things. But more often than not, a team of developers works together to perform these tasks. Sometimes a team member does more than one activity; a tester can take part in a requirements review, for example, or an implementer can write documentation. Each team is different, and team dynamics play a large role in the team's success.

We can examine both product and process to see how each contributes to quality and in particular to security as an aspect of quality. Let us begin with the product, to get a sense of how we recognize high-quality secure software.

Modularity, Encapsulation, and Information Hiding

Code usually has a long shelf-life, and it is enhanced over time as needs change and faults are found and fixed. For this reason, a key principle of software engineering is to create a design or code in small, self-contained units, called components or modules; when a system is written this way, we say that it is modular. Modularity offers advantages for program development in general and security in particular.

If a component is isolated from the effects of other components, then it is easier to trace a problem to the fault that caused it and to limit the damage the fault causes. It is also easier to maintain the system, since changes to an isolated component do not affect other components. And it is easier to see where vulnerabilities may lie if the component is isolated. We call this isolation encapsulation.

Information hiding is another characteristic of modular software. When information is hidden, each component hides its precise implementation or some other design decision from the others. Thus, when a change is needed, the overall design can remain intact while only the necessary changes are made to particular components.

Let us look at these characteristics in more detail.

Modularity

Modularization is the process of dividing a task into subtasks. This division is done on a logical or functional basis. Each component performs a separate, independent part of the task. Modularity is depicted. The goal is to have each component meet four conditions:

• single-purpose: performs one function

• small: consists of an amount of information for which a human can readily grasp both structure and content

• simple: is of a low degree of complexity so that a human can readily understand the purpose and structure of the module

• independent: performs a task isolated from other module

In particular, smallness is an important quality that can help security analysts understand what each component does. That is, in good software, design and program units should be only as large as needed to perform their required functions. There are several advantages to having small, independent components.

• Maintenance. If a component implements a single function, it can be replaced easily with a revised one if necessary. The new component may be needed because of a change in requirements, hardware, or environment. Sometimes the replacement is an enhancement, using a smaller, faster, more correct, or otherwise better module. The interfaces between this component and the remainder of the design or code are few and well described, so the effects of the replacement are evident.

• Understandability. A system composed of many small components is usually easier to comprehend than one large, unstructured block of code.

• Reuse. Components developed for one purpose can often be reused in other systems. Reuse of correct, existing design or code components can significantly reduce the difficulty of implementation and testing.

• Correctness. A failure can be quickly traced to its cause if the components perform only one task each.

• Testing. A single component with well-defined inputs, output, and function can be tested exhaustively by itself, without concern for its effects on other modules (other than the expected function and output, of course).

Security analysts must be able to understand each component as an independent unit and be assured of its limited effect on other components.

A modular component usually has high cohesion and low coupling. By cohesion, we mean that all the elements of a component have a logical and functional reason for being there; every aspect of the component is tied to the component's single purpose. A highly cohesive component has a high degree of focus on the purpose; a low degree of cohesion means that the component's contents are an unrelated jumble of actions, often put together because of time-dependencies or convenience.

Coupling refers to the degree with which a component depends on other components in the system. Thus, low or loose coupling is better than high or tight coupling, because the loosely coupled components are free from unwitting interference from other components.

Encapsulation

Encapsulation hides a component's implementation details, but it does not necessarily mean complete isolation. Many components must share information with other components, usually with good reason. However, this sharing is carefully documented so that a component is affected only in known ways by others in the system. Sharing is minimized so that the fewest interfaces possible are used. Limited interfaces reduce the number of covert channels that can be constructed.

Information Hiding

Developers who work where modularization is stressed can be sure that other components will have limited effect on the ones they write. Thus, we can think of a component as a kind of black box, with certain well-defined inputs and outputs and a well-defined function. Other components' designers do not need to know how the module completes its function; it is enough to be assured that the component performs its task in some correct manner.

This concealment is the information hiding. Information hiding is desirable, because developers cannot easily and maliciously alter the components of others if they do not know how the components work.

Peer Reviews

We turn next to the process of developing software. Certain practices and techniques can assist us in finding real and potential security flaws (as well as other faults) and fixing them before the system is turned over to the users. Of the many practices available for building what they call "solid software," Pfleeger et al. recommend several key techniques: [PFL01a]

• peer reviews
• hazard analysis
• testing
• good design
• prediction
• static analysis
• configuration management
• analysis of mistakes

Here, we look at each practice briefly, and we describe its relevance to security controls. We begin with peer reviews.

You have probably been doing some form of review for as many years as you have been writing code: desk-checking your work or asking a colleague to look over a routine to ferret out any problems. Today, a software review is associated with several formal process steps to make it more effective, and we review any artifact of the development process, not just code. But the essence of a review remains the same: sharing a product with colleagues able to comment about its correctness. There are careful distinctions among three types of peer reviews:

• Review: The artifact is presented informally to a team of reviewers; the goal is consensus and buy-in before development proceeds further.

• Walk-through: The artifact is presented to the team by its creator, who leads and controls the discussion. Here, education is the goal, and the focus is on learning about a single document.

• Inspection: This more formal process is a detailed analysis in which the artifact is checked against a prepared list of concerns. The creator does not lead the discussion, and the fault identification and correction are often controlled by statistical measurements.

A wise engineer who finds a fault can deal with it in at least three ways:

1. by learning how, when and why errors occur,

2. by taking action to prevent mistakes, and

3. by scrutinizing products to find the instances and effects of errors that were missed.

Peer reviews address this problem directly. Unfortunately, many organizations give only lip service to peer review, and reviews are still not part of mainstream software engineering activities.

But there are compelling reasons to do reviews. An overwhelming amount of evidence suggests that various types of peer review in software engineering can be extraordinarily effective. For example, early studies at Hewlett-Packard in the 1980s revealed that those developers performing peer review on their projects enjoyed a very significant advantage over those relying only on traditional dynamic testing techniques, whether black-box or white-box. Figure 3-19 compares the fault discovery rate (that is, faults discovered per hour) among white-box testing, black-box testing, inspections, and software execution. It is clear that inspections discovered far more faults in the same period of time than other alternatives. [GRA87] This result is particularly compelling for large, secure systems, where live running for fault discovery may not be an option.

Hazard Analysis

Hazard analysis is a set of systematic techniques intended to expose potentially hazardous system states. In particular, it can help us expose security concerns and then identify prevention or mitigation strategies to address them. That is, hazard analysis ferrets out likely causes of problems so that we can then apply an appropriate technique for preventing the problem or softening its likely consequences. Thus, it usually involves developing hazard lists, as well as procedures for exploring "what if" scenarios to trigger consideration of nonobvious hazards. The sources of problems can be lurking in any artifacts of the development or maintenance process, not just in the code, so a hazard analysis must be broad in its domain of investigation; in other words, hazard analysis is a system issue, not just a code issue. Similarly, there are many kinds of problems, ranging from incorrect code to unclear consequences of a particular action. A good hazard analysis takes all of them into account.

Although hazard analysis is generally good practice on any project, it is required in some regulated and critical application domains, and it can be invaluable for finding security flaws. It is never too early to be thinking about the sources of hazards; the analysis should begin when you first start thinking about building a new system or when someone proposes a significant upgrade to an existing system. Hazard analysis should continue throughout the system life cycle; you must identify potential hazards that can be introduced during system design, installation, operation, and maintenance.

Good Design

We saw earlier in this chapter that modularity, information hiding, and encapsulation are characteristics of good design. Several design-related process activities are particularly helpful in building secure software:

• using a philosophy of fault tolerance

• having a consistent policy for handling failures

• capturing the design rationale and history

• using design patterns

We describe each of these activities in turn.

Designs should try to anticipate faults and handle them in ways that minimize disruption and maximize safety and security. Ideally, we want our system to be fault free. But in reality, we must assume that the system will fail, and we make sure that unexpected failure does not bring the system down, destroy data, or destroy life. For example, rather than waiting for the system to fail (called passive fault detection), we might construct the system so that it reacts in an acceptable way to a failure's occurrence. Active fault detection could be practiced by, for instance, adopting a philosophy of mutual suspicion. Instead of assuming that data passed from other systems or components are correct, we can always check that the data are within bounds and of the right type or format. We can also use redundancy, comparing the results of two or more processes to see that they agree before using their result in a task.

If correcting a fault is too risky, inconvenient, or expensive, we can choose instead to practice fault tolerance: isolating the damage caused by the fault and minimizing disruption to users. Although fault tolerance is not always thought of as a security technique, it supports the idea, discussed in Chapter 8, that our security policy allows us to choose to mitigate the effects of a security problem instead of preventing it. For example, rather than install expensive security controls, we may choose to accept the risk that important data may be corrupted. If in fact a security fault destroys important data, we may decide to isolate the damaged data set and automatically revert to a backup data set so that users can continue to perform system functions.

Trusted Software

We say that software is trusted software if we know that the code has been rigorously developed and analyzed, giving us reason to trust that the code does what it is expected to do and nothing more. Typically, trusted code can be a foundation on which other, untrusted, code runs. That is, the untrusted system's quality depends, in part, on the trusted code; the trusted code establishes the baseline for security of the overall system. In particular, an operating system can be trusted software when there is a basis for trusting that it correctly controls the accesses of components or systems run from it. For example, the operating system might be expected to limit users' accesses to certain files. We look at trusted operating systems in more detail in Chapter 5.

To trust any program, we base our trust on rigorous analysis and testing, looking for certain key characteristics:

• Functional correctness: The program does what it is supposed to, and it works correctly.

• Enforcement of integrity: Even if presented erroneous commands or commands from unauthorized users, the program maintains the correctness of the data with which it has contact.

• Limited privilege: The program is allowed to access secure data, but the access is minimized and neither the access rights nor the data are passed along to other untrusted programs or back to an untrusted caller.

• Appropriate confidence level: The program has been examined and rated at a degree of trust appropriate for the kind of data and environment in which it is to be used.

Trusted software is often used as a safe way for general users to access sensitive data. Trusted programs are used to perform limited (safe) operations for users without allowing the users to have direct access to sensitive data.