Abstract:Automated monitoring of dark web (DW) platforms on a large scale is the first step toward developing proactive Cyber Threat Intelligence (CTI). While there are efficient methods for collecting data from the surface web, large-scale dark web data collection is often hindered by anti-crawling measures. In particular, text-based CAPTCHA serves as the most prevalent and prohibiting type of these measures in the dark web. Text-based CAPTCHA identifies and blocks automated crawlers by forcing the user to enter a combination of hard-to-recognize alphanumeric characters. In the dark web, CAPTCHA images are meticulously designed with additional background noise and variable character length to prevent automated CAPTCHA breaking. Existing automated CAPTCHA breaking methods have difficulties in overcoming these dark web challenges. As such, solving dark web text-based CAPTCHA has been relying heavily on human involvement, which is labor-intensive and time-consuming. In this study, we propose a novel framework for automated breaking of dark web CAPTCHA to facilitate dark web data collection. This framework encompasses a novel generative method to recognize dark web text-based CAPTCHA with noisy background and variable character length. To eliminate the need for human involvement, the proposed framework utilizes Generative Adversarial Network (GAN) to counteract dark web background noise and leverages an enhanced character segmentation algorithm to handle CAPTCHA images with variable character length. Our proposed framework, DW-GAN, was systematically evaluated on multiple dark web CAPTCHA testbeds. DW-GAN significantly outperformed the state-of-the-art benchmark methods on all datasets, achieving over 94.4% success rate on a carefully collected real-world dark web dataset. We further conducted a case study on an emergent Dark Net Marketplace (DNM) to demonstrate that DW-GAN eliminated human involvement by automatically solving CAPTCHA challenges with no more than three attempts. Our research enables the CTI community to develop advanced, large-scale dark web monitoring. We make DW-GAN code available to the community as an open-source tool in GitHub.

A Novel Cross-Domain Adaptation Framework for Unsupervised Criminal Jargon Detection Via Pre-Trained Contextual Embedding of Darknet Corpus

Chinese underground market jargon analysis based on unsupervised learning

Camouflage is NOT Easy: Uncovering Adversarial Fraudsters in Large Online App Review Platform

Unsupervised Domain Adaptation for Crime Risk Prediction Across Cities

Towards Dark Jargon Interpretation in Underground Forums

DarknetSec: A novel self-attentive deep learning method for darknet traffic classification and application identification

Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation

Darknet Public Hazard Entity Recognition Based on Deep Learning.

Transferring Audio Deepfake Detection Capability Across Languages

Link Prediction-Based Multi-Identity Recognition of Darknet Vendors.

Darknet Data Mining -- A Canadian Cyber-crime Perspective

A Probabilistic Generative Model for Mining Cybercriminal Networks from Online Social Media

A systematic empirical study on word embedding based methods in discovering Chinese black keywords

Forensic investigation of the dark web on the Tor network: pathway toward the surface web

What Happens Behind the Scene? Towards Fraud Community Detection in E-Commerce from Online to Offline

Discovering Emerging Threats in the Hacker Community: A Nonparametric Emerging Topic Detection Framework

A Multi-View Attention-Based Deep Learning System for Online Deviant Content Detection

dStyle-GAN: Generative Adversarial Network based on Writing and Photography Styles for Drug Identification in Darknet Markets

Counteracting Dark Web Text-Based CAPTCHA with Generative Adversarial Learning for Proactive Cyber Threat Intelligence

A Novel Darknet Traffic Classification Method Based on Knowledge Graph with Dynamic Embedding Learning.

An Efficient Deep Unsupervised Domain Adaptation for Unknown Malware Detection