The present invention relates generally to the security of computers and programs which execute on them, and more particularly to detecting and/or protecting against attacks which seek to divert or alter the flow of execution of a program. The invention is suited for, but not limited to, use with programs arranged for execution on an embedded controller. It may be utilised in relation to any computer system in which the integrity of the system can be jeopardised by, for example, by deliberate manipulation or by cosmic rays, electrostatic discharge or unexpected operating conditions.

A program consists of a sequence of instructions which will be executed in a predetermined order by a processor repeating a machine cycle which, for example, consists of three phases:

1. 1) Fetch a copy of an instruction from main memory; the address of the instruction is specified by the contents of the Program Counter (PC);
2. 2) decode the instruction that was fetched;
3. 3) execute it.

At the machine level, all instructions, addresses and data are stored and manipulated in binary form. An instruction comprises an opcode which specifies the type of operation to be performed and (usually) one or more operands. An opcode may, for example, specify an arithmetic or logical instruction such as ADD X, Y; or program control such as JUMP X, which causes address X to be loaded into the PC so that the instruction at address X will be fetched, decoded and executed during the next cycle. In this way, a program's flow may branch off down different paths. A control flow graph (CFG) can be used to capture all the possible paths that could be taken during normal i.e. non-compromised execution of the program.

However, an attacker can divert the expected flow of a program using a technique known as Return Oriented Programming (ROP).

In an ROP attack, the binary code is searched for short sequences which the attacker can chain together. These instruction sequences (known as 'gadgets') can then be used to perform defined tasks and introduce unintended behaviour into the program. The instruction sequences are typically chosen so that each gadget ends in a return instruction or equivalent which, if the attacker has control of the run-time stack, allows control to flow from one sequence to the next.

Defending against this type of attack is possible using software countermeasures.

An alternative solution has now been devised. The invention is defined in the appended claims.

The invention may provide a computer-implemented method of control flow enforcement, comprising the steps:

• analysing a reference control flow generated in respect of a portion of code to derive at least one control flow rule; and
• evaluating the at least one control flow rule during execution of the portion of code to facilitate detection of a deviation from the reference control flow.

In one embodiment, the method may comprise the step of providing a rule checking component configured to evaluate, apply or check the at least one rule during execution of the portion of code. The rule checking component may be implemented as, or within, a hardware unit. It may be implemented as or within a virtual machine. The rule checking component may comprise or be in communication with memory for storing the at least one control flow rule, and/or an intervention unit configured to generate a response upon detection of a deviation from the reference control flow.

The reference control flow may provide a means for predicting the expected execution of a program under normal operating conditions. It may express the manner in which the program is to operate, for example a series of expected function calls.

The portion of code may comprise a sequence of instructions. It may be an entire program or a part thereof. It may be at least a portion of a machine or low level language program. It may be static i.e. not changing over time, or non-static.

In one embodiment, a plurality of control flow rules is derived from the reference control flow. The control flow rule(s) may define run-time behaviour which is expected or legitimate according to the reference control flow. As the reference control flow may be generated for a particular portion of code, the plurality of control flow rules generated therefrom may be specific to the portion of code.

Evaluation of the at least one control flow rule may be triggered or determined by the contents of the program counter (PC), an instruction opcode, a flag and/or a register value. The rule may be selected from a plurality of rules based upon the contents of the PC and/or an instruction opcode. A rule may be selected and evaluated for each instruction that is executed. The rule may be defined based, for example, upon the current value of the program counter (PC), the previous value of the PC (pPC), the current instruction opcode (OP), the previously executed instruction opcode(s) (xOP) or the opcode before the current opcode (pOP). Other forms of rules may also be incorporated or devised.

The at least one control flow rule may impose a restriction on the way that the program is allowed to execute. It may, for example, limit the distance of a JUMP, forbid certain instructions (opcodes) or patterns of opcodes from being used. Additionally or alternatively, a rule may be configured such that it is only applied, for example, in respect of a specified region of code memory, or when a specified external signal is applied.

In one embodiment, the reference control flow and/or at least one control flow rule is generated prior to execution of the portion of code. The reference control flow may be generated by executing the portion of code under normal operating conditions to determine the set of legitimate paths that may be taken by the portion of code. The at least one control flow rule may then be generated from the reference control flow to provide a rule-based description or definition of the code's expected behaviour. The rule(s) can then be checked at run-time to evaluate whether or not the code is behaving as intended.

This can allow an embodiment of this disclosure to be used with an existing portion of code. Accordingly, an embodiment of this disclosure need not require the use of a specially adapted or designed instruction set, or a specially designed software development tool. The source code may not even be required. As long as the program can be executed under normal conditions so as to generate the reference flow, the rules can be generated and applied.

In one embodiment, the reference control flow is a control flow graph (CFG). Thus, an embodiment of this disclosure may enable an observed program flow to be assessed or evaluated against a reference program flow which may be encoded as a CFG. As the size of a CFG can be very large, typically even larger than the program binary itself, direct assessment of a program flow against the reference CFG can give rise to a variety of implementation difficulties such as storage overhead, memory bandwidth saturation and the need for a large comparison logic due to the amount of data that would need to be analysed. An embodiment of this disclosure may provide a mechanism for utilising the reference control flow while avoiding such problems.

In one embodiment, the method may further comprise the step of observing or monitoring the execution of the portion of code. This may be achieved by observing signals sent over a signal bus and/or observing internal signals from a processor.

The method may further comprise the step of generating a response upon detection of a deviation from the expected control flow. The response may be encoded as part of the rule. The response may comprise, for example: triggering an interrupt for a processor; halting a processor; generating a memory fault; logging an entry into a fault log; and/or generating an alert. The response may be selected from a plurality of possible responses which may be executed upon detection of a deviation from the expected control flow. The response may depend upon the nature and/or severity level of the deviation.

In one embodiment, the method may comprise the step of analysing the portion of code to derive the reference control flow and identify at least one possible attack mechanism for a security attack. The security attack may be a Return Oriented Programming attack. The attack mechanism may be a gadget. Additionally or alternatively, the method may comprise the step of testing a plurality of control flow rules against the reference control flow and recording the number of attack mechanisms (eg gadgets) eliminated by each rule in the plurality. Additionally or alternatively, it may comprise the step of identifying a subset of the plurality of rules to provide a reduced rule set. The reduced rule set may be selected so as to maximise the number of attack mechanisms addressed or eliminated by the rule set. This may provide the benefit of a trade-off between cost and performance.

Also in accordance with the invention, there may be provided an apparatus for performing any embodiment of the method described above.

The invention may provide a computer-implemented system for enforcing the control flow of a program. The system may comprise:

• memory for storing at least one control flow rule derived by analysis of a reference control flow generated in respect of a portion of code; and
• at least one processor configured to execute the portion of code; and a rule checking component arranged to evaluate the at least one control flow rule during execution of the portion of code to facilitate detection of a deviation from the reference control flow.

The system may comprise a hardware unit or virtual machine configured to implement the method described above. The system may comprise a rule checking component having, or in communication with, memory for storing the at least one control flow rule and/or an intervention unit configured to generate a response upon detection of a deviation from the reference control flow. The rule checking unit may be a hardware unit. It may be implemented using a programmable (logic) component. It may be a Field-Programmable Gate Array (FPGA). The FPGA may be provided in proximity to the processor. It may be arranged and configured to select which rule(s) to apply for an observed instruction.

The rule checking component may be configured to observe or monitor the execution of the portion of code. The reference control flow may be a control flow graph.

The system may comprise an intervention component arranged to generate a response upon detection of a deviation from the expected control flow. The response may comprise, for example, a processor interrupt; a halt signal for halting a processor; a signal for generating a memory fault; an entry in a fault log; and/or an alert. Other forms of response may be utilised.

The reference control flow and/or at least one control flow rule may be generated prior to execution of the portion of code. The rule checking component may be configured to evaluate the at least one control flow rule based upon, for example, the contents of the program counter, an instruction opcode, a flag and/or a register value. Other variations may be devised.

Accordingly, embodiments of this disclosure can provide a method and corresponding system for monitoring control flow during program execution and/or assessing control flow integrity.

It should be noted that any feature described above in relation to one aspect or embodiment of the invention may also be applicable to and incorporated into another other aspect or embodiment. For example, subject matter described in relation to a method of the invention may also be applicable to a system of the invention, and vice versa.

These and other aspects of the present invention will be apparent from and elucidated with reference to, the embodiment described herein. An embodiment of the present invention will now be described, by way of example only, and with reference to the accompany drawings, in which:

• Figure 1 illustrates an embodiment of this disclosure having a dedicated rule memory.
• Figure 2 illustrates an embodiment of this disclosure with rule memory located in system memory.

Embodiments of this disclosure can provide a rule-based solution enforcing program control flow. An expected control flow is pre-determined ie prior to execution of the program using a control flow graph (CFG) which is then analysed to provide a set of control flow rules relating to the expected behavior of the program under normal operating circumstances. Once the rules have been generated, they can be stored for subsequent retrieval and application.

During execution of the program by one or more processors, the program state can be observed and monitored. The rules are repeatedly checked to determine whether the program's unfolding execution conforms to the flow defined by the CFG. If at any point a rule is found to have been violated, an intervention of some type is generated to respond appropriately.

Thus, in broad terms, the embodiments of this disclosure can be said to comprise the following steps:

1. 1. determine how the program (or a portion of it) is expected to behave under normal circumstances;
2. 2. define and store a set of rules describing the expected behavior;
3. 3. execute the program on a processor and use the appropriate rule(s) to determine at each point in the execution whether the program is behaving as expected;
4. 4. generate a response if a rule has been violated.

Accordingly, embodiments of this disclosure can provide a mechanism for detecting abnormal control flows such as may be brought about by a ROP attack and eliminating or reducing its harmful effects. This enhances security and provides a more reliable computer system.

The rules set may be designed in different ways. For example, a rule may be devised to address each possible scenario which could arise during an attack at run time. However, while this approach can provide a functionally sophisticated defence mechanism, it can also result in a large, complex set of rules which, in turn, require more hardware and logic to implement.

Thus, another approach to designing the rule set is to devise a reduced set of rules which addresses an acceptable number of possible attack scenarios. Such an approach can be as follows:

1. 1. analyse the target program to arrive at both the CFG and a list of 'attackable' positions (gadgets) for a ROP; techniques and tools are known in the art for such purposes;
2. 2. devise a set of rules for handling the gadgets identified in step 1
3. 3. test at least some of the rules devised in step 2 against the CFG; for each tested rule, count the number of gadgets which are eliminated by that rule;
4. 4. identify a minimal subset of rules that maximize the number of eliminated gadgets.

This 'reduced rule set' approach provides the advantage that it allows a trade off or compromise between the cost and performance of the countermeasure.

Turning to Figure 1, an illustrative architecture is shown for implementing a program flow enforcement unit in accordance with an embodiment of this disclosure. In one embodiment, this architecture is implemented as a hardware unit. For example, a PFGA can be used to provide the required logic. However, other implementations may be utilised to similar or same effect. For example, the program flow enforcement solution could be implemented as part of a virtual machine.

In one or more embodiments, the system may comprise a rule checking unit which is in communication with a separate memory resource containing additional data and/or logic for specifying which rule is to be applied to which instruction. Alternatively, the additional data and/or logic may be incorporated into the rule set itself so that the separate memory is not required. In either architecture, however, there is a mechanism which specifies which rule or rules are to be applied to a particular instruction.

Figure 1 depicts an exemplary processor system comprising a processor (100) connected via a signal bus (101) to system memory (102) and other system components (103). Also depicted in Figure 1 is an embodiment of the present disclosure comprising a rule checking unit (200) which compares a set of rules stored in rule memory (201) to the observed behaviour of the processor (100). It does this by observing signals (202) sent over the signal bus (101) and/or observing internal signals (203) from the processor (100). When a violation of these rules is detected or observed it intervenes through an intervention unit (204).

Figure 2 depicts an alternative embodiment wherein the set of rules (201) is stored in system memory (102). During execution of the observed program these rules are retrieved from the system memory (102) and provided to the checking unit (200) via a rule retrieval unit (205). This arrangement can be beneficial because such a rule retrieval unit could take advantage of instruction pipelining on modern processors (100) to pre-fetch and cache the rules for to-be-executed instruction's addresses.

Below is an exemplary list of rules which could be checked by the rule checker. The following is not to be interpreted as an exhaustive list. The rule set may comprise other rules in addition to or instead of those mentioned below. The set may comprise any rules that can be derived from analysis of the CFG generated in respect of the software running on the processor system.

Rule 1: previous PC should = current PC - previous instruction size

This rule enforces linear program flow. During linear flow, the next instruction in the sequence is fetched, decoded and executed. The address where the next instruction is to be fetched from in main memory is specified by the contents of the PC. Therefore, if the CFG specifies that linear program flow is expected, then the previous value of the PC should equal the current value minus the size of the previous instruction in bytes.

The instruction size depends on the program and the processor architecture. While some architectures are designed around fixed-length instructions, others can include variable length instruction sets. Therefore, multiple instances or variations of this rule may be provided to account for architectures which allow for instructions of different sizes.

Rule 2: Always execute

This rule can be used to execute a specified instruction without question or checks. This may be appropriate in circumstances where there is no available information regarding the previously executed instruction or address. Such circumstances might include, for example, the first instruction to be executed after a processor reset, or when the instruction/address can be executed at any time e.g. the first instruction of an exception handler.

Rule 3: Never execute

This rule can be used to mark at least one address as non-executable. These might be addresses of literals (constant data blocks and addresses) that are intermixed with executable code, or parts of long instruction words that are not the start of the instruction (when the instruction spans over multiple addresses).

This technique allows much finer access specification than the usual access protection schemes that are based on address ranges.

Rule 4: Previous instruction is not an indirect jump

Indirect jumps that obtain their destination from writeable memory or from a register can be more easily exploited by ROP attacks. This rule helps to prevent that by marking all addresses/instructions that ought never to be accessed in this way according to the CFG. It also provides a mechanism for avoiding the need to check a potentially long list of allowed possibilities. Therefore, while this rule results in a less than accurate CFG check it still provides a very effective defence and detection technique.

Rule 5: Previous instruction is a static jump

At many points the program flow is not linear although still very predictable. This rule prevents arbitrary predecessors in the sense that only jumps with fixed destinations (encoded in the instruction) are allowed.

Rule 6: The previous instruction has a specific length

This rule is similar to rule 1, but also works when the program flow is not linear. Any predecessor with the wrong instruction length is rejected. It will be apparent to the skilled person that this rule is only useful if not all instructions have the same length.

Rule 7: The previous instruction was conditionally executed.

This rule applies to all instructions whose execution or effect depends on a certain condition, this includes conditional jumps and conditional operations such as ADDIF<flag>. This rule may be further enforced by double checking the condition under which the instruction should/should not have been executed.

Rule 8: The instruction is a memory access to a specific memory region

The region in question can be defined by lower and upper bounds which could be set globally, could depend on the address of the currently executed instruction, or could be defined as part of the rule. This rule can be used to enforce that the instruction is e.g. only allowed to access the stack space.

During use, as the observed program executes, instruction addresses and/or opcodes are examined by the rule checker 200. The address or opcode of an observed instruction determines which rule applies. Thus, for each observed signal (ie. instruction) there is always at least one rule that can be checked - even if it is the 'always execute' rule. Therefore, as the program execution progresses, an embodiment of this disclosure can continuously monitor and analyse the behaviour in comparison to the CFG and respond accordingly.

In one or more embodiments, only one rule may be checked for each instruction. Thus, by selectively enforcing the most stringent rule for a given instruction, the complete arsenal of rules can be used in an effective manner while the need for rule memory is reduced.

The embodiment described above may require no modification to the software creation process. The programmer can program freely without having to abide by certain coding rules or restrictions and the software can be created using a standard development toolchain.