# Data is Moody: Discovering Data Modification Rules from Process Event Logs
This repository contains the source code and some supplementary material for the paper *Data is Moody: Discovering Data Modification Rules from Process Event Logs*.

## Abstract
Although event logs are a powerful source to gain insight into the behavior of the underlying business process,
existing work primarily focuses on finding patterns in the activity sequences of an event log,
while ignoring event attribute data.
Event attribute data has mostly been used to predict
event occurrences and process outcome,
but the state of the art neglects to mine succinct and interpretable rules describing
how event attribute data changes during process execution.
Subgroup discovery and rule-based classification approaches lack the ability to capture the
sequential dependencies present in event logs,
and thus lead to unsatisfactory results with limited insight into the process behavior.

Given an event log, we aim to find accurate yet succinct and interpretable if-then rules
how the process modifies data.
We formalize the problem in terms of the Minimum Description Length (MDL) principle,
by which we choose the model with the best lossless description of the data.
Additionally, we propose the greedy MOODY algorithm to efficiently search for rules.
By extensive experiments on both synthetic and real-world data,
we show MOODY indeed finds compact and interpretable rules,
needs little data for accurate discovery, and is robust to noise.

## MDL encoding differences
The MDL encoding of numerical values in this implementation differs slightly from what we described in the paper.
In particular, when specifying numerical ranges for an update (`v in [a, b]` or `v = v + [a, b]`), our implementation assumes a uniform distribution inside the interval.
In contrast, the encoding described in the paper would respect different probabilities of the histogram bins that fall inside the interval.

## Getting started
To reproduce the experiments in the paper or run your own experiments, you need to install [Docker](https://www.docker.com/) first.
I also recommend using [Visual Studio Code](https://code.visualstudio.com/) to develop and run the code since it has integrated support for [development containers](https://code.visualstudio.com/docs/devcontainers/containers).
For guidance on how to build the containers and MOODY's implementation, take a look at the [.gitlab-ci.yml](/.gitlab-ci.yml) file.

## How to get the real-world datasets
Due to the size limit of 10 MB for supplementary material on Microsoft CMT, we could not add the
real-world datasets to our submission. If you want to reproduce our experiments on real-world data,
please download the file `Sepsis Cases - Event Log.xes` from
https://doi.org/10.4121/uuid:915d2bfb-7e84-49ad-a286-dc35f063a460 and add it to the folder
`experiments/datasets/`.
Also download the file `Road_Traffic_Fine_Management_Process.xes` from
https://doi.org/10.4121/uuid:270fd440-1057-4fb9-89a9-b699b47990f5 and add it to the folder
`experiments/datasets/`.

## Technical architecture of experiments
![Visualization of docker containers and interfaces](experiment-architecture.png "Experiment architecture")
To run the experiments without interferring software dependencies, MOODY itself as well as each method we compared MOODY against run in a separate Docker container which we illustrate above.
Each method provides its functionality through a [Cap'n Proto](https://capnproto.org/) Remote Procedure Call (RPC) interface on different ports.
We run the actual experiment code in another Docker container that connects to these RPC interfaces.
This setup is also configured in the [docker-compose.yml](docker/docker-compose.yml) file.

## Repository structure
For a brief overview of where to find what in this repository, use the following table:

| Directory | Description |
| --------- | ----------- |
| `cpp-tool/` | C++ source code for the MOODY algorithm | 
| `docker/` | Dockerfiles for all containers used to develop and experiment with MOODY and its baselines (e.g. `Cpp.Dockerfile` is used to build and run MOODY) | 
| `experiments/` | Python source code and Jupyter Notebooks used to experiment with MOODY and its baselines |

We describe the detailed structure of the C++ source code in the following table:

| Directory | Description |
| --------- | ----------- |
| `cpp-tool/capnproto/` | Cap'n Proto interface of MOODY and implementation of its RPC server |
| `cpp-tool/classes/` | C++ classes containing all of MOODY's functionality |
| `cpp-tool/classes/datamodel/` | C++ classes modeling the data structures that MOODY operates on (i.e. event logs, models of data modification rules) |
| `cpp-tool/classes/preprocessing/` | C++ classes mostly used for preprocessing event logs before running the search for data modification rules. This includes estimating the accuracy and distribution of variables, as well as detecting cyclic models. |
| `cpp-tool/classes/scoring/` | C++ classes used to compute the (estimated) MDL score that MOODY searches over |
| `cpp-tool/classes/search/` | C++ classes for finding good models and generating rule candidates |
| `cpp-tool/libraries/` | Source code of libraries used for MOODY |
| `cpp-tool/tests/` | Unit tests for MOODY's core functionality |

For more details on the directory for our experiments, consider this table:

| Directory | Description |
| --------- | ----------- |
| `experiments/SSDpp/` | Source code to use [SSD++](https://doi.org/10.1007/s10618-022-00856-x) in our experiments |
| `experiments/TURS/` | Source code to use [TURS](http://arxiv.org/abs/2206.08804) in our experiments |
| `experiments/artificial-data/` | Source code and results of our experiments on synthetic event logs |
| `experiments/preliminary/` | Source code of initial experiments to test our implementation, hyperparameters and other properties of MOODY |
| `experiments/real-data/` | Source code and results of our experiments on public real-world event logs |
