Abstract:The term data quality refers to measuring the fitness of data regarding the intended usage. Poor data quality leads to inadequate, inconsistent, and erroneous decisions that could escalate the computational cost, cause a decline in profits, and cause customer churn. Thus, data quality is crucial for researchers and industry practitioners. Different factors drive the assessment of data quality. Data context is deemed one of the key factors due to the contextual diversity of real-world use cases of various entities such as people and organizations. Data used in a specific context (e.g., an organization policy) may need to be more efficacious for another context. Hence, implementing a data quality assessment solution in different contexts is challenging. Traditional technologies for data quality assessment reached the pinnacle of maturity. Existing solutions can solve most of the quality issues. The data context in these solutions is defined as validation rules applied within the ETL (extract, transform, load) process, i.e., the data warehousing process. In contrast to traditional data quality management, it is impossible to specify all the data semantics beforehand for big data. We need context-aware data quality rules to detect semantic errors in a massive amount of heterogeneous data generated at high speed. While many researchers tackle the quality issues of big data, they define the data context from a specific standpoint. Although data quality is a longstanding research issue in academia and industries, it remains an open issue, especially with the advent of big data, which has fostered the challenge of data quality assessment more than ever. This paper provides a scoping review to study the existing context-aware data quality assessment solutions, starting with the existing big data quality solutions in general and then covering context-aware solutions. The strength and weaknesses of such solutions are outlined and discussed. The survey showed that none of the existing data quality assessment solutions could guarantee context awareness with the ability to handle big data. Notably, each solution dealt only with a partial view of the context. We compared the existing quality models and solutions to reach a comprehensive view covering the aspects of context awareness when assessing data quality. This led us to a set of recommendations framed in a methodological framework shaping the design and implementation of any context-aware data quality service for big data. Open challenges are then identified and discussed.

Big Data Quality: A Survey

Big data quality framework: a holistic approach to continuous quality management

Assessing Data Quality Within Available Context

Big Data Quality: A systematic literature review and future research directions

Data Quality Assessment: Challenges and Opportunities

A Survey of Big Data Research

The Challenges of Data Quality and Data Quality Assessment in the Big Data Era

Quality Analytics in a Big Data supply chain: Commodity data analytics for quality engineering

Data Quality in Edge Machine Learning: A State-of-the-Art Survey

Data Quality Awareness: A Journey from Traditional Data Management to Data Science Systems

A survey of big data management: Taxonomy and state-of-the-art

On the Meaningfulness of “Big Data Quality” (Invited Paper)

A survey on dataset quality in machine learning

Big Data Analytics: A Survey

A Survey on Data Quality Dimensions and Tools for Machine Learning

Overview and Importance of Data Quality for Machine Learning Tasks

Data Deluge Dynamics: Tracing the Evolution and Ramifications of Big Data Phenomenon

Big data: From beginning to future

A Theoretical Framework for AI-driven data quality monitoring in high-volume data environments

Big Data: Survey, Technologies, Opportunities, and Challenges

Context-aware big data quality assessment: a scoping review