This is the second post in a series about the X-Road logs. The first part was about different log types (technical logs, business logs, audit logs) and the X-Road logs in general. The second part concentrates on the X-Road business log which contains all the messages processed by a Security Server – the message log.
Background
The original idea behind the message log was to store a tamper-proof machine-readable evidence of every message processed by a Security Server. By guaranteeing non-repudiation of log entries using digital signatures it was possible to provide an undeniable evidence of each transaction. Storing the logs in a unified machine-readable format made it possible to use them in automated processes. In a wider picture, the logs would allow reducing manual work, increase the level of automation in various processes and make things easier for both users of different information systems and citizens.
Another important aspect was to implement some commonly needed features such as logging of business events in an off-to-shelf component that everyone would use. In this way, there was no need to implement the same feature for all the information systems separately. Of course, potential benefits of this approach depend on the starting point of the ecosystem as nowadays logging of business events is required from all the production level information systems. However, the format of logs is not unified between different information systems and not all the systems guarantee non-repudiation of data.
Message log today
Originally, the aim of the X-Road was to provide a secure and standardized way to exchange data that guarantees non-repudiation of the data and provides the evidence in unified machine-readable format. Today, in 2018, the core functionality of the X-Road version 6 can still be described using the same words even if version numbers and technical implementation details have changed many times over the years.
The X-Road version 6 guarantees non-repudiation of the data sent via the X-Road using time-stamping and digital signatures. All the evidence is stored in the message log database from where it is archived to disk using associated signature containers (ASiC) for eIDAS. Security Server owner can access active log records stored in the message log database using a web service interface. Once log records have been archived, accessing them requires shell access to Security Server. No external parties have access to the message log. The X-Road itself is used as a data exchange layer in automated processes between different organizations and information systems, but the message log is not currently used for automation purposes. If something needs to be checked from the message log, manual work from Security Server administrator is always required.
Nowadays Security Server provides a feature that makes it possible to disable logging of message payload that contains the actual business data. This means that message payload is dropped before logging and only message headers with an empty payload are logged. However, time-stamping and signing of messages are always done using the original message which means that it is impossible to verify the signature afterwards as it is created using the original message and message log contains only message headers. Message hash in the signature and message hash calculated using the logged message will never match as the logged message does not contain the payload. This means that all the evidential value of the message log is lost, and it can be used for reporting and statistical purposes only.
To log or not to log?
Why to disable logging of message payload if the evidential value of the message log is lost? The answer lies on the logical architecture and the type of data that is exchanged. Is Security Server used for exchanging personal data or other sensitive data? Is Security Server seen as a part of the information system that is using it to exchange data or is Security Server seen as a separate, external information system that is integrated with the information system that is using it to exchange data?
Type of data that is exchanged is important, because there are rules and restrictions regarding how personal data and other sensitive data must be handled and processed. In case of personal data, depending on the jurisdiction, the message log may form a person registry when message payload logging is enabled. This means that Security Server must be compliant with technical and non-technical requirements regarding processing of personal data which might differ between different countries and ecosystems. In addition, the interpretation of different legal requirements might vary as well.
When Security Server is seen as a part of an information system containing a person registry, Security Server is one of the system components and therefore personal data stored in the message log remains inside the system boundaries. It is enough that Security Server meets the technical requirements and all the applicable maintenance and operating processes are followed. Instead, when Security Server is seen as a separate, external information system, message log may become an additional person registry and the purposes of processing personal data of the information system that is using the Security Server to exchange data cannot be applied to it anymore. In this scenario disabling logging of message payload can be used to prevent the creation of an additional person registry.
What is the most logical interpretation regarding Security Server’s role with respect to the information system? If we think about Security Server as a message mediator or a message proxy it is easy to see it as a part of the information system rather than an external system. Let's think about the question in more general level. For example, how modern microservice based systems are structured – usually they consist of multiple independent services that communicate with each other through APIs. All the individual microservices are part of the same system and there’s only a single person registry even if not all personal data is stored in the same physical storage. In addition, all the enterprise level information systems consist of multiple components that are located in different physical or virtual hosts, and usually they’re seen as a one system. So why should Security Server be any different?
What next?
Have the original ideas how the message log could be used for process automation become true and is the message log’s full potential already reached? Answer to both questions is no, which means that the X-Road could provide even more value to its users than it currently does. Message log records could be used in process automation and for implementing new business features. When logging of message payload is enabled the required data is already there in the message log, but utilizing it for new use cases will require some additional development to make it more accessible than it is now. However, the required effort is small compared to the potential value that could be created.
In the third part of the series about the X-Road logs I’m going to discuss how to provide access to the logs of who has accessed my data and when. Until then.