Abstract:Mobile applications (or apps) are rapidly growing in number and variety. These apps provide useful features, but also bring certain privacy and security risks. For example, malicious authors may attach destructive payloads to legitimate apps to create so-called "piggybacked" apps and advertise them in various app markets to infect unsuspecting users. To detect them, existing approaches typically employ pair-wise comparison, which unfortunately has limited scalability. In this paper, we present a fast and scalable approach to detect these apps in existing Android markets. Based on the fact that the attached payload is not an integral part of a given app's primary functionality, we propose a module decoupling technique to partition an app's code into primary and non-primary modules. Also, noticing that piggybacked apps share the same primary modules as the original apps, we develop a feature fingerprint technique to extract various semantic features (from primary modules) and convert them into feature vectors. We then construct a metric space and propose a linearithmic search algorithm (with O(n log n) time complexity) to efficiently and scalably detect piggybacked apps. We have implemented a prototype and used it to study 84,767 apps collected from various Android markets in 2011. Our results show that the processing of these apps takes less than nine hours on a single machine. In addition, among these markets, piggybacked apps range from 0.97% to 2.7% (the official Android Market has 1%). Further investigation shows that they are mainly used to steal ad revenue from the original developers and implant malicious payloads (e.g., for remote bot control). These results demonstrate the effectiveness and scalability of our approach.

An Automatic and Scalable Application Crawler for Large-Scale Mobile Internet Content Retrieval.

UMCR: User Interaction-Driven Mobile Content Retrieval.

Crawler Detection in Location-Based Services Using Attributed Action Net

Fast, Scalable Detection of "piggybacked" Mobile Applications

What Makes a Great Mobile App? A Quantitative Study Using a New Mobile Crawler

AMACS: Automated Mobile Application Content Sensing

App Crawler : Opening Mobile App Black Boxes

SmartCrawler: A Two-Stage Crawler for Efficiently Harvesting Deep-Web Interfaces

A cognitive crawler using structure pattern for incremental crawling and content extraction

Scale-Adaptable Recrawl Strategies For Dht-Based Distributed Web Crawling System

Automatic Browsing of Large Pictures on Mobile Devices.

A Brief History of Web Crawlers

A Large-Scale Mobile Traffic Dataset For Mobile Application Identification

Automatically Crawling Dynamic Web Applications Via Proxy-Based JavaScript Injection and Runtime Analysis.

A Measurement Study of Wechat Mini-Apps

The Application of Web Crawler in City Image Research

A peer-to-peer based passive web crawling system

An Efficient Valid Page Crawling Approach for Websites with Dynamic Scripts

BUbiNG: Massive Crawling for the Masses