R&D, protection of Windows processes

Tristan Leiter
Cybersecurity Blog

It is part of Navixia's values to be involved in the research and development of security products while promoting cybersecurity academic training in Switzerland.

This is why Jämes Ménétrey, a master student at the HES-SO, University of Applied Science and Arts Western Switzerland, joined us during a semester to complete his master thesis.

His dissertation work, entitled "Software Supervisor: A Security Mechanism That Protects Executing Programs From Malicious Manipulation ", consists in a state of the art research project on software protection methods including the development of a tool designed to isolate certain critical Windows processes in view of preventing their malicious manipulation.

What is it all about?

A process is a program running on an operating system. One of its specific features is to ensure the program's isolation. This allows the operating system to determine what resources, such as memory, are allocated to the program then to release them when the program terminates or encounters an error that ends the process.

However, isolating a process does not in turn isolate the information it contains. A process can interact with another one, for example to extract sensitive data from its memory, without needing to obtain its consent first. Processes are important notions in program management within the operating system, but they do not assure data confidentiality.

Concrete case: access to a critical process

To understand what impact process permeability can have in terms of memory access, let us look into a meaningful example, the KeePass password manager. This program creates a cryptographically secure password container on a computer-based data storage. Executing the KeePass process opens a container and the user is prompted to enter a password (or another comparable authentication method) to decrypt its content. Once this action has been performed, the user is allowed to view the container's contents, without re-entering a password or decrypting the contents again since they are stored in memory. Of course, KeePass does not store the decryption method in its original form but retains an equivalent information that allows the user to retrieve the container's passwords.

This method of operation makes the KeePass process critical, since another process has the ability to read its memory in order to extract the information it contains, such as the container's main password (or its cryptographic representation), which KeePass stores as long as the container is open. In practice, a malicious process infects the KeePass process and invokes the password access functions that KeePass itself uses when handling user requests. This technique is used by password-stealing programs when extracting sensitive information from KeePass. (KeeThief, KeeFarce).

In addition, the official KeePass documentation informs users about its KeePass memory protection procedure (check the Process Memory Protection section). This section indicates in particular that for some operations, KeePass is forced to expose sensitive data in the process's memory in unencrypted form. This happens, for instance, when a user wants to display a password in clear in the KeePass interface. The extraction of sensitive data is made even easier if the process's memory is copied in order to analyze its content in an offline step (forensic analysis).

How do we limit this sensitive data's exposure?

Jämes Ménétrey's master's thesis introduces a new concept designed to limit interaction between two processes in Microsoft's operating system: the enclave system. An enclave is defined as a protective environment surrounding one or more processes. Processes inside an enclave can interact with processes outside this protected zone but the other way around is not allowed, unless a communication channel has been explicitly opened. As a result, a process can no longer read another process's memory if it is located inside an enclave.

Enclaves are created, managed and wiped out using a driver running in the Windows kernel and developed as part of this research project. This driver is able to filter interactions between two processes. It maintains a set of data structures that first determine whether a given process is located within an enclave, then block any interaction with other processes. The driver also extends this notion of enclaves to the operating system's file system which prevents a process from manipulating sensitive files, such as a secure password container. This ensures that a secure container can only be reached by a specific program, which is also protected.

Access rights to enclaves

This research defines two types of enclaves:

  1. Protection enclaves, which contain critical processes and data (e.g. KeePass.exe) and
  2. Dependency enclaves, which contain the dependencies needed to run critical processes (such as kernel32.dll).

It proved necessary to introduce these two concepts because dependencies are attack vectors for protected processes and must therefore be included in the protection. However, dependencies are not critical data per se. Their level of protection is lower than that of critical data. Access rights related to the various process interactions are illustrated in the image below. These rules can be summarized as follows:

  • A process located outside the protection system can read the content of a dependency, but not modify it. It has no access to the processes and data located inside the protection enclave.
  • A process located inside the protection system can read the content of a dependency, but not modify it. It can access the resources contained in the same enclave. It can also access processes outside the protection system.

In addition, a process can belong to several enclaves, allowing critical data to be shared between several processes.

Access rights for processes as regards protected resources within an enclave system.

Centralized enclave management

The enclave management system was centralized as a web application, making it possible to manage a fleet of enclave-holding machines. An important part of the project relates to the handling of data within blocked interactions. Any process trying to manipulate a protected process generates a log, then this information is sent back to the server managing the enclaves. Administrators can view these alerts via a web interface. This enables them to detect malicious activity related to both process interaction and access to protected files inside the enclaves. Other functionalities are also available, such as suspending one or more enclaves when protected programs need to get updated.

Architecture and technologies used

The tool developed in this research is called Navixia Shield. It comprises three parts:

  • An ASP.NET Core web interface
  • A .NET agent installed on each enclaves-holding server. This agent is designed to propagate the enclaves' configuration to the driver and send their activity logs. The agents interact with the server in real time using a web socket.
  • An agent-connected C driver handles enclave management and retrieval of related information, such as activity logs of blocked interactions.

It has been evaluated that the use of a higher level programming language, in this case C#, facilitated the development of such platforms as the web interface or the agent, with the added flexibility of an excellent compatibility with lower level languages (in this case C, which was used to develop the driver). Consequently, a communication protocol was developed, allowing easy transfer of data between technologies. A data transmission overview is shown in the following function (C#), which is located in the agent that talks to the driver:

// Sends a given buffer to the driver and receives a buffer in response.
public static unsafe TOutBuffer FilterSendMessage<TInBuffer, TOutBuffer>
  (SafeHandle handle, in TInBuffer buffer)
    where TInBuffer : unmanaged
    where TOutBuffer : unmanaged
    TOutBuffer outBuffer;

    // The input buffer is fixed in memory, so we can dereference it without the fear of being annoyed by the garbage collector
    fixed (void* inBufferPtr = &buffer)
        // Send and receive the buffers to the driver
        var result = NativeApi.FilterSendMessage(
            (uint) sizeof(TInBuffer),
            (uint) sizeof(TOutBuffer),
            out _);

        if (result != (uint) HResult.SuccessOk)
            throw new Win32Exception((int) result, "Cannot send a message [..].");

    return outBuffer;

The above source code is a function designed to transform a C# data structure into another data structure. Their types are defined using TInBuffer and TOutBuffer parameters, whose distinguishing feature is to be unmanaged. This means that structures passed as arguments are contiguous in memory and can therefore be copied as blocks, without risking to get a pointer to a C# object. In that way, structures can be sent and received to/from the driver and this is the way information circulates between the agent and the driver. Here is one of the many examples that demonstrate this language's usability and interoperability with other systems.

On the driver's side, interactions between processes are notably regulated by callbacks objects, i.e. a filter making it possible to add restrictions when a process tries to access another process. The driver is of the type minifilter file system, which also allows it to filter access to the file system. This approach makes it possible to prohibit access to files that are located inside an enclave.

Overview of the administration interface

The screenshots below give an insight into the admin interface used to manage enclaves on multiple devices.

Devices management

Enclave management

Enclave detail

Are you interested in this security solution?

The prototype developed for this master's thesis was sufficient to demonstrate that this protection system is functional and provides enough information to detect and slow down an attack on critical processes. However, being rather complex, it is not available as a Navixia commercial solution because it still requires a fair amount of development and verification. Nevertheless, should you be interested to work with us on this project, please do not hesitate to contact us through our contact form.

Additional information about the master's thesis

This research project was carried out by Jämes Ménétrey as part of the Master of Science in Engineering (MSE) training at the University of Applied Sciences Western Switzerland (HES-SO) during the winter semester 2019-2020. He was supervised by Tristan Leiter, Security Architect at Navixia and by Prof. Dr. Sylvain Pasini at the HEIG-VD. He obtained the maximum mark of 6 for the project's design, implementation and thesis defense.


No comments made yet. Be the first to submit a comment

Leave your comment