Abstract:With the daily increase of data production and collection, Hadoop is a platform for processing big data on a distributed system. A master node globally manages running jobs, whereas worker nodes process partitions of the data locally. Hadoop uses MapReduce as an effective computing model. However, Hadoop experiences a high level of security vulnerability over hybrid and public clouds. Specially, several workers can fake results without actually processing their portions of the data. Several redundancy-based approaches have been proposed to counteract this risk. A replication mechanism is used to duplicate all or some of the tasks over multiple workers (nodes). A drawback of such approaches is that they generate a high overhead over the cluster. Additionally, malicious workers can behave well for a long period of time and attack later. This paper presents a novel model to enhance the security of the cloud environment against untrusted workers. A new component called malicious workers' trap (MWT) is developed to run on the master node to detect malicious (noncollusive and collusive) workers as they convert and attack the system. An implementation to test the proposed model and to analyze the performance of the system shows that the proposed model can accurately detect malicious workers with minor processing overhead compared to vanilla MapReduce and Verifiable MapReduce (V-MR) model [1]. In addition, MWT maintains a balance between the security and usability of the Hadoop cluster.

Eavesdropper: A Framework for Detecting the Location of the Processed Result in Hadoop.

Haddle: A Framework for Investigating Data Leakage Attacks in Hadoop

A Distributed Data Mining System Framework for Mobile Internet Access Log Based on Hadoop.

Security Threats to Hadoop: Data Leakage Attacks and Investigation.

SEINA: A Stealthy and Effective Internal Attack in Hadoop Systems

Adaptive Failure Detection Via Heartbeat under Hadoop.

Vhadoop: A Scalable Hadoop Virtual Cluster Platform for MapReduce-Based Parallel Machine Learning with Performance Consideration

A security framework in G-Hadoop for big data computing across distributed Cloud data centres

Invader Job: A Kind Of Malicious Failure Job On Hadoop Yarn

Desgin and Implemention of Distributed Intrusion Detection System Based on Hadoop Cluster

Distributed Shielded Execution for Transmissible Cyber Threats Analysis

A Novel Hadoop Security Model for Addressing Malicious Collusive Workers

Hadoop-Based Healthcare Information System Design and Wireless Security Communication Implementation

Towards Real-Time Privacy Preservation: A Streaming Location Anonymous Method Based On Distributed Framework

Privacy-Preserving Machine Learning Algorithms for Big Data Systems

An Auxiliary Decision-Making System for Electric Power Intelligent Customer Service Based on Hadoop

Detection of Information leakage in cloud

Call Trace and Memory Access Pattern based Runtime Insider Threat Detection for Big Data Platforms

Online System Problem Detection by Mining Patterns of Console Logs

SeEagle: Semantic-Enhanced Anomaly Detection for Securing Eagle

SEYARN: Enhancing Security of YARN Clusters Based on Health Check Service