Abstract:Context - The exponential growth of data is becoming a significant concern. Managing this data has become incredibly challenging, especially when dealing with various sources in different formats and speeds. Moreover, Ensuring data quality has become increasingly crucial for effective decision-making and operational processes. Data Architecture is crucial in describing, collecting, storing, processing, and analyzing data to meet business needs. Providing an abstract view of data-intensive applications is essential to ensure that the data is transformed into valuable information. We must take these challenges seriously to ensure we can effectively manage and use the data to our advantage. Objective - To establish an architecture framework that enables a comprehensive description of the data architecture and effectively streamlines data quality monitoring. Method - The architecture framework utilizes Model Driven Engineering (MDE) techniques. Its backing of data-intensive architecture descriptions empowers with an automated generation for data quality checks. Result - The Framework offers a comprehensive solution for data-intensive applications to model their architecture efficiently and monitor the quality of their data. It automates the entire process and ensures precision and consistency in data. With DAT, architects and analysts gain access to a powerful tool that simplifies their workflow and empowers them to make informed decisions based on reliable data insights. Conclusion - We have evaluated the DAT on more than five cases within various industry domains, demonstrating its exceptional adaptability and effectiveness.
What problem does this paper attempt to address?
The paper mainly explores how to effectively manage and monitor data quality by establishing a data-intensive application architecture framework. With the exponential growth of data volume, managing data from different sources, formats, and speeds becomes extremely challenging, while ensuring data quality is crucial for decision making and operational processes. The study proposes an architecture framework that utilizes Model-Driven Engineering (MDE) technology, which supports automated generation of data quality checks, aiming to provide a comprehensive solution for data architects and analysts to efficiently model application architectures and monitor data quality.
In the paper, the authors first introduce the importance of data, the necessity of data architecture, and the current challenges in data management. Then, the research methodology is described in detail, including the adoption of qualitative research, case company analysis, research technology, and data analysis methods. The paper also mentions several related works, demonstrating the application and effectiveness of the proposed framework in various industry cases, and evaluates it.
The main contribution of the paper is the development of a data architecture modeling tool called DAT for data-driven applications. DAT supports a comprehensive description from architecture design to data quality assurance, automating the entire process to ensure data accuracy and consistency. In addition, the paper discusses how to use DAT to model data analytics architectures, especially in data-driven applications such as data warehouses and big data analytics.
The final part of the paper compares related works, evaluates the results, and proposes future directions. Overall, this research aims to address the complexity of data management by providing structured methods and tools to improve data quality and management efficiency in data-intensive applications.