Structural Positional Encoding for knowledge integration in transformer-based medical process monitoring

Christopher Irwin,Marco Dossena,Giorgio Leonardi,Stefania Montani
2024-03-13
Abstract:Predictive process monitoring is a process mining task aimed at forecasting information about a running process trace, such as the most correct next activity to be executed. In medical domains, predictive process monitoring can provide valuable decision support in atypical and nontrivial situations. Decision support and quality assessment in medicine cannot ignore domain knowledge, in order to be grounded on all the available information (which is not limited to data) and to be really acceptable by end users.
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: in medical process monitoring, how to provide decision - making support for complex and atypical medical situations through Predictive Process Monitoring (PPM) technology. Specifically, the author focuses on how to use the Transformer architecture and Structural Positional Encoding (SPE) to improve the prediction accuracy of future medical activities. ### Main problems 1. **Atypical situations in the medical field**: - Clinical guidelines are usually designed based on ideal patients and ideal conditions, but in practical applications, factors such as resource limitations and patients' complex conditions (such as comorbidities or rare disease variants) may make it difficult to directly apply the guidelines. - Doctors may lack sufficient background knowledge to correctly interpret and apply the guidelines, especially in complex situations. 2. **Limitations of existing methods**: - Most of the existing Predictive Process Monitoring methods rely on traditional machine - learning techniques (such as transition systems, Hidden Markov Models, etc.), and these methods have deficiencies in dealing with long - sequence dependencies and complex relationships. - Although deep - learning methods (such as RNN, LSTM, GRU) can capture long - term dependencies in time series, they still lack effective integration of domain - specific knowledge. ### Solutions The author proposes a Predictive Process Monitoring method based on the Transformer architecture and introduces Structural Positional Encoding (SPE) to incorporate domain - specific knowledge (such as medical ontology) into the model. The main contributions of this method include: - **Transformer architecture**: Use the self - attention mechanism to capture the complex dependency relationships between different activities in the medical process. - **Structural Positional Encoding (SPE)**: Encode activities through graph structures (such as ontology), enabling the model to better understand the relationships between activities, thereby improving prediction accuracy. ### Experimental verification The author conducted experiments in the field of stroke management. The results show that the method using SPE significantly improves the prediction performance of the model, especially showing stability under different embedding dimensions. This indicates that combining domain knowledge can significantly improve the effect of Predictive Process Monitoring and provide stronger support for medical decision - making. ### Conclusion This research shows that in medical process monitoring, combining the Transformer architecture and domain - specific knowledge (achieved through SPE) can significantly improve prediction accuracy, providing new ideas for decision - support in complex medical scenarios. Future research will further explore the application of different embedding techniques and expand to other medical fields.