Annotating Needles in the Haystack Without Looking

Weinan Zhang,Amr Ahmed,Jie Yang,Vanja Josifovski,Alex J. Smola

DOI: https://doi.org/10.1145/2783258.2788580

2015-01-01

Abstract:Business-to-consumer (B2C) emails are usually generated by filling structured user data (e.g.purchase, event) into templates. Extracting structured data from B2C emails allows users to track important information on various devices.

What problem does this paper attempt to address?

Annotating Needles In The Haystack Without Looking: Product Information Extraction From Emails

Weinan Zhang,Amr Ahmed,Jie Yang,Vanja Josifovski,Alex J. Smola

DOI: https://doi.org/10.1145/2783258.2788580

2015-01-01

Abstract:Business-to-consumer (B2C) emails are usually generated by filling structured user data (e.g. purchase, event) into templates. Extracting structured data from B2C emails allows users to track important information on various devices.However, it also poses several challenges, due to the requirement of short response time for massive data volume, the diversity and complexity of templates, and the privacy and legal constraints. Most notably, email data is legally protected content, which means no one except the receiver can review the messages or derived information.In this paper we first introduce a system which can extract structured information automatically without requiring human review of any personal content. Then we focus on how to annotate product names from the extracted texts, which is one of the most difficult problems in the system. Neither general learning methods, such as binary classifiers, nor more specific structure learning methods, such as Conditional Random Field (CRF), can solve this problem well.To accomplish this task, we propose a hybrid approach, which basically trains a CRF model using the labels predicted by binary classifiers (weak learners). However, the performance of weak learners can be low, therefore we use Expectation Maximization (EM) algorithm on CRF to remove the noise and improve the accuracy, without the need to label and inspect specific emails. In our experiments, the EM-CRF model can significantly improve the product name annotations over the weak learners and plain CRFs.
Finding Camouflaged Needle in a Haystack?

Guoxiu He,Yangyang Kang,Zhe Gao,Zhuoren Jiang,Changlong Sun,Xiaozhong Liu,Wei Lu,Qiong Zhang,Luo Si

DOI: https://doi.org/10.1145/3331184.3331197

2019-01-01

Abstract:It is an important and urgent research problem for decentralized eCommerce services, e.g., eBay, eBid, and Taobao, to detect illegal products, e.g., unclassified pornographic products. However, it is a challenging task as some sellers may utilize and change camouflaged text to deceive the current detection algorithms. In this study, we propose a novel task to dynamically locate the pornographic products from very large product collections. Unlike prior product classification efforts focusing on textual information, the proposed model, BerryPIcking TRee MoDel (BIRD), utilizes both product textual content and buyers' seeking behavior information as berrypicking trees. In particular, the BIRD encodes both semantic information with respect to all branches sequence and the overall latent buyer intent during the whole seeking process. An extensive set of experiments have been conducted to demonstrate the advantage of the proposed model against alternative solutions. To facilitate further research of this practical and important problem, the codes and buyers' seeking behavior data have been made publicly available1.
Automatic Extraction Of Commodity Attributes On Webpages Based On Hierarchical Structure

Zhi Yu,Meiyan Li,Wei Wang,Can Wang

DOI: https://doi.org/10.1142/9789814689007_0046

2015-01-01

Abstract:Everygood in e-commerce websites contains a lot of commodity attributes; even some of them are hidden behind the user's dynamic interaction. Finding them out for further analysis is a challenging topic. How to extract the commodity attributes effectively has been an attracting research topic in this area recently. Previous works cost too much calculation to deal this problem effectively, since they involve complex operations such as semantic analysis to extract hidden attributes. In this article, we analyze the relationship between different URL terms and commodity attributes, discuss with the hierarchical structure of the query results and propose a new Commodity Attributes Extraction algorithm based on Single-Layer method to find hidden attributes denoted by the URL terms. Using these URL terms, we can find out hidden attributes from a new query result page easily by analyzing its URL. Experiment result demonstrates the effectiveness of our algorithm.
Crawler Detection in Location-Based Services Using Attributed Action Net

Wei Xia,Fei Zhao,Haishuai Wang,Peng Zhang,Anhui Wang,Kang Li

DOI: https://doi.org/10.1145/3459637.3481907

2021-01-01

Abstract:Malicious Web crawlers threaten information system due to heavily taking up bandwidth resources and stealing private user data. Ele.me, a prevalent on-demand food delivery platform in China, suffers from the negative impact of crawlers. The crawler detection systems face two major challenges: spatial patterns of the crawler behaviors and limited labeled data for training. In this paper, we present efficient solutions to tackle these challenges. Specifically, we propose a new Attributed Action Net (AANet for short) model to detect Location-Based Services~(LBS) crawlers and a three-stage learning framework to train the model. AANet consists of three different embedding modules, including the action token sequence, temporal-spatial attributes of users, and the context information of the raw data. We have deployed the model at Ele.me, and both offline experiments and online A/B tests show that the proposed method is superior to the state-of-the-art models for sequence data classification on the food delivery platform.
Personalization in Email Marketing: The Role of Non-Informative Advertising Content

Navdeep S. Sahni,S. Christian Wheeler,Pradeep K. Chintagunta

DOI: https://doi.org/10.2139/ssrn.2725251

2016-01-01

SSRN Electronic Journal

Abstract:In collaboration with three companies selling a diverse set of products, we conducted randomized field experiments in which experimentally tailored email ads were sent to millions of individuals. We found consistently that personalizing the emails by adding consumer-specific information (e.g., name, employer name) benefited the advertisers. Importantly, such content is not likely to be informative about the advertised product or the company. In our main experiment, we found that adding the name of the message recipient to the email's subject-line increased the probability of the recipient opening it by 20% (from 9.05% to 10.80%), which translated to an increase in sales leads by 31% (from 0.39% to 0.51%) and a reduction in the number of individuals unsubscribing from the email campaign by 17% (from 1.2% to 1.0%). We present similar experiments conducted with other companies, which show that the effects we document extend from objectives ranging from acquiring new customers to retaining customers who have purchased from the company in the past. The effects also extend to other content of similar nature. Our investigation of several possible mechanisms suggests that such content increases the attention consumers pay to the other content in the rest of the advertising message. Our paper quantifies the benefits from personalization, and contributes to understanding the role of non-informative advertising content. It contributes to the psychology-based research in marketing by establishing the robustness of lab findings in field settings. It has clear implications for the firms that are designing their advertising campaigns.
Natural language processing on customer note data

Andrew Hilditch,David Webb,Jozef Baca,Tom Armitage,Matthew Shardlow,Peter Appleby

2023-05-03

Abstract:Automatic analysis of customer data for businesses is an area that is of interest to companies. Business to business data is studied rarely in academia due to the sensitive nature of such information. Applying natural language processing can speed up the analysis of prohibitively large sets of data. This paper addresses this subject and applies sentiment analysis, topic modelling and keyword extraction to a B2B data set. We show that accurate sentiment can be extracted from the notes automatically and the notes can be sorted by relevance into different topics. We see that without clear separation topics can lack relevance to a business context.

Computation and Language
"Know" Your Mobile Customers: A Design Approach to an Android-Based Mobile Analytics Tool.

Qiqi Jiang,Lele Kang

2018-01-01

Abstract:There is no doubt that we are living in the mobile age. Massive commercial data from mobile users provide huge potential opportunities for both merchants and researchers. However, compared with Web analytics, how to analyze the mobile data is still under debate. Some companies have adopted similar measurements or metrics to those that have been used in Web analytics tools to depict mobile activities. However, such measurements might bias the interpretation because the behaviors on mobile devices are fundamentally different from behaviors in an online context. Hence, we propose a prototype of an Android-based mobile analytics tool with novel mechanisms for better observing user behaviors in the mobile context, by which the individual behavioral data can be captured at the activity level. The detailed design, such as prototype structure, data structure, specific classes or methods, and sample codes, will be provided. In addition, some potential research opportunities will be elaborated. Furthermore, the practical value of our proposed prototype will be discussed.
Annotating High-Level Structures of Short Stories and Personal Anecdotes

Boyang Li,Beth Cardier,Tong Wang,Florian Metze

DOI: https://doi.org/10.48550/arXiv.1710.06917

2017-10-08

Computation and Language

Abstract:Stories are a vital form of communication in human culture; they are employed daily to persuade, to elicit sympathy, or to convey a message. Computational understanding of human narratives, especially high-level narrative structures, remain limited to date. Multiple literary theories for narrative structures exist, but operationalization of the theories has remained a challenge. We developed an annotation scheme by consolidating and extending existing narratological theories, including Labov and Waletsky's (1967) functional categorization scheme and Freytag's (1863) pyramid of dramatic tension, and present 360 annotated short stories collected from online sources. In the future, this research will support an approach that enables systems to intelligently sustain complex communications with humans.
Semantic annotation of consumer health questions

Halil Kilicoglu,Asma Ben Abacha,Yassine Mrabet,Sonya E. Shooshan,Laritza Rodriguez,Kate Masterton,Dina Demner-Fushman

DOI: https://doi.org/10.1186/s12859-018-2045-1

IF: 3.307

2018-02-06

BMC Bioinformatics

Abstract:BackgroundConsumers increasingly use online resources for their health information needs. While current search engines can address these needs to some extent, they generally do not take into account that most health information needs are complex and can only fully be expressed in natural language. Consumer health question answering (QA) systems aim to fill this gap. A major challenge in developing consumer health QA systems is extracting relevant semantic content from the natural language questions (question understanding). To develop effective question understanding tools, question corpora semantically annotated for relevant question elements are needed. In this paper, we present a two-part consumer health question corpus annotated with several semantic categories: named entities, question triggers/types, question frames, and question topic. The first part (CHQA-email) consists of relatively long email requests received by the U.S. National Library of Medicine (NLM) customer service, while the second part (CHQA-web) consists of shorter questions posed to MedlinePlus search engine as queries. Each question has been annotated by two annotators. The annotation methodology is largely the same between the two parts of the corpus; however, we also explain and justify the differences between them. Additionally, we provide information about corpus characteristics, inter-annotator agreement, and our attempts to measure annotation confidence in the absence of adjudication of annotations.ResultsThe resulting corpus consists of 2614 questions (CHQA-email: 1740, CHQA-web: 874). Problems are the most frequent named entities, while treatment and general information questions are the most common question types. Inter-annotator agreement was generally modest: question types and topics yielded highest agreement, while the agreement for more complex frame annotations was lower. Agreement in CHQA-web was consistently higher than that in CHQA-email. Pairwise inter-annotator agreement proved most useful in estimating annotation confidence.ConclusionsTo our knowledge, our corpus is the first focusing on annotation of uncurated consumer health questions. It is currently used to develop machine learning-based methods for question understanding. We make the corpus publicly available to stimulate further research on consumer health QA.

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
A STUDY OF B2C CONSUMER BEHAVIOR MODEL

Min-yang Wang,Hui-Ming Kuo,Sheue-Ling Hwang

DOI: https://doi.org/10.1080/10170660509509307

2005-01-01

Journal of the Chinese Institute of Industrial Engineers

Abstract:The purpose of this research is to verify a preliminary B2C consumer behavior model that is constructed previously by brainstorming, structured interview, and on-line observation. Twelve subjects voluntarily participated in the verification experiment. Thinking aloud method was used for the experiment. Subjects were asked to do online shopping in an experimental environment and their behavioral processes were recorded by video cameras for later analysis using observation software—“The Observer”. Three types of products were provided in the experimental environment: book, cell phone, and notebook PC. Experimental results indicated that the B2C consumer behavior in electronic commerce could be described completely by the proposed Ten-Step Model. The complexity of the behavior model and time spent in each step depend on the characteristics of the products. It is suggested that websites information and the customer-supporting interfaces should be customized according to the types of products and the factors that affect consumer behavior to best assist customers in EC.
Categorizing Online Shopping Behavior from Cosmetics to Electronics: An Analytical Framework

Sohini Roychowdhury,Wenxi Li,Ebrahim Alareqi,Akhilesh Pandita,Ao Liu,Joakim Soderberg

DOI: https://doi.org/10.48550/arXiv.2010.02503

2020-10-06

Abstract:A success factor for modern companies in the age of Digital Marketing is to understand how customers think and behave based on their online shopping patterns. While the conventional method of gathering consumer insights through questionnaires and surveys still form the bases of descriptive analytics for market intelligence units, we propose a machine learning framework to automate this process. In this paper we present a modular consumer data analysis platform that processes session level interaction records between users and products to predict session level, user journey level and customer behavior specific patterns leading towards purchase events. We explore the computational framework and provide test results on two Big data sets-cosmetics and consumer electronics of size 2GB and 15GB, respectively. The proposed system achieves 97-99% classification accuracy and recall for user-journey level purchase predictions and categorizes buying behavior into 5 clusters with increasing purchase ratios for both data sets. Thus, the proposed framework is extendable to other large e-commerce data sets to obtain automated purchase predictions and descriptive consumer insights.

Machine Learning,Computers and Society
An Analysis of Structured Data on the Web

Nilesh Dalvi,Ashwin Machanavajjhala,Bo Pang

DOI: https://doi.org/10.48550/arXiv.1203.6406

2012-03-29

Abstract:In this paper, we analyze the nature and distribution of structured data on the Web. Web-scale information extraction, or the problem of creating structured tables using extraction from the entire web, is gathering lots of research interest. We perform a study to understand and quantify the value of Web-scale extraction, and how structured information is distributed amongst top aggregator websites and tail sites for various interesting domains. We believe this is the first study of its kind, and gives us new insights for information extraction over the Web.

Databases
Intelligent Parsing: An Automated Parsing Framework for Extracting Design Semantics from E-commerce Creatives

Guandong Li,Xian Yang

DOI: https://doi.org/10.48550/arXiv.2312.17283

2023-12-28

Computer Vision and Pattern Recognition

Abstract:In the industrial e-commerce landscape, creative designs such as banners and posters are ubiquitous. Extracting structured semantic information from creative e-commerce design materials (manuscripts crafted by designers) to obtain design semantics represents a core challenge in the realm of intelligent design. In this paper, we propose a comprehensive automated framework for intelligently parsing creative materials. This framework comprises material recognition, preprocess, smartname, and label layers. The material recognition layer consolidates various detection and recognition interfaces, covering business aspects including detection of auxiliary areas within creative materials and layer-level detection, alongside label identification. Algorithmically, it encompasses a variety of coarse-to-fine methods such as Cascade RCNN, GFL, and other models. The preprocess layer involves filtering creative layers and grading creative materials. The smartname layer achieves intelligent naming for creative materials, while the label layer covers multi-level tagging for creative materials, enabling tagging at different hierarchical levels. Intelligent parsing constitutes a complete parsing framework that significantly aids downstream processes such as intelligent creation, creative optimization, and material library construction. Within the practical business applications at Suning, it markedly enhances the exposure, circulation, and click-through rates of creative materials, expediting the closed-loop production of creative materials and yielding substantial benefits.
ANSWER: Annotation Software for Electronic Reporting

Chelsea Raulerson,Guillaume Jimenez,Benjamin Wakeland,Erika Villa,Jeffrey Sorelle,James Malter,Jeffrey Gagan,Brandi Cantarel

DOI: https://doi.org/10.1200/CCI.21.00113

Abstract:Purpose: To better use genetic testing, which is used by clinicians to explain the molecular mechanism of disease and to suggest clinical actionability and new treatment options, clinical next-generation sequencing (NGS) laboratories must send the results into reports in PDF and discrete data element format (HL7). Although most clinical diagnostic tests have set molecular markers tested and have a set range of values or a binary result (positive or negative), the NGS genetic test could examine hundreds or thousands of genes with no predefined list of variants. Although there are some commercial and open-source tools for clinically reporting genomics results for oncology testing, they often lack necessary features. Methods: Using several available software tools for data storage including MySQL and MongoDB, database querying with Python, and a web-based user application using JAVA and JAVA script, we have developed a tool to store and query complex genomics and demographics data, which can be manually curated and reported by the user. Results: We have developed a tool, Annotation SoftWare for Electronic Reporting (ANSWER), that can allow molecular pathologists to (1) filter variants to find those meeting quality control metrics in the genes that are clinically actionable by diagnosis; (2) visualize variants using data generated in the bioinformatics analysis; (3) create annotations that can be reused in future reports with association specific to the gene, variant, or diagnosis; (4) select variants and annotations that should be reported to match the details of the case; and (5) generate a report that includes demographics, reported variants, clinical actionability annotation, and references that can be exported into PDF or HL7 format, which can be electronically sent to an electronic health record. Conclusion: ANSWER is a tool that can be installed locally and is designed to meet the clinical reporting needs of a clinical oncology NGS laboratory for reporting.
You Are What You Buy: Personal Information Extraction From Anonymized Data

Thomas Cilloni,Charles Fleming,Charles Walter

DOI: https://doi.org/10.1109/access.2024.3365190

IF: 3.9

2024-03-02

IEEE Access

Abstract:The exponential growth of data in the information age poses several threats to the privacy and safety of digital service users. Existing legislation, such as the GDPR in Europe and the CCPA in California, defines frameworks and guidelines to promote personal privacy but leaves freedom in the choice of means to achieve privacy. Data anonymization techniques remove information that can be used to identify individuals from the dataset, either through suppression, generalization, anatomization, permutation, or perturbation. Information suppression remains the most common, safe, and widely applicable anonymization method, though at a high data utility cost. In this paper, we argue that even information suppression may not be sufficient in some cases. We study the case of a dataset that describes the shopping habits of a grocery store's customers. All identifiers and quasi-identifiers are removed from the dataset by suppression. However, by augmenting the data in a novel multi-step, iterative process, and building a neural network enriched with prior knowledge, we show that most sensitive information can be retrieved with an accuracy of 80%.

computer science, information systems,telecommunications,engineering, electrical & electronic
Data Embroidery with Black-and-White Textures

Tingying He,Petra Isenberg,Tobias Isenberg

DOI: https://doi.org/10.48550/arXiv.2309.02864

2023-09-06

Abstract:We investigated data embroidery with black-and-white textures, identifying challenges in the use of textures for machine embroidery based on our own experience. Data embroidery, as a method of physically representing data, offers a unique way to integrate personal data into one's everyday fabric-based objects. Owing to their monochromatic characteristics, black-and-white textures promise to be easy to employ in machine embroidery. We experimented with different textured visualizations designed by experts and, in this paper, we detail our workflow and evaluate the performance and suitability of different textures. We then conducted a survey on vegetable preferences within a family and created a canvas bag as a case study, featuring the embroidered family data to show how embroidered data can be used in practice.

Human-Computer Interaction
Leam: An Interactive System for In-situ Visual Text Analysis

Sajjadur Rahman,Peter Griggs,Çağatay Demiralp

DOI: https://doi.org/10.48550/arXiv.2009.03520

2020-09-08

Abstract:With the increase in scale and availability of digital text generated on the web, enterprises such as online retailers and aggregators often use text analytics to mine and analyze the data to improve their services and products alike. Text data analysis is an iterative, non-linear process with diverse workflows spanning multiple stages, from data cleaning to visualization. Existing text analytics systems usually accommodate a subset of these stages and often fail to address challenges related to data heterogeneity, provenance, workflow reusability and reproducibility, and compatibility with established practices. Based on a set of design considerations we derive from these challenges, we propose Leam, a system that treats the text analysis process as a single continuum by combining advantages of computational notebooks, spreadsheets, and visualization tools. Leam features an interactive user interface for running text analysis workflows, a new data model for managing multiple atomic and composite data types, and an expressive algebra that captures diverse sets of operations representing various stages of text analysis and enables coordination among different components of the system, including data, code, and visualizations. We report our current progress in Leam development while demonstrating its usefulness with usage examples. Finally, we outline a number of enhancements to Leam and identify several research directions for developing an interactive visual text analysis system.

Databases,Computation and Language,Human-Computer Interaction
A Study of Electronic Annotation on Web Documents

Pei-Luen Patrick Rau,Sho-Hsen Chen

DOI: https://doi.org/10.1177/154193120204600517

2002-01-01

Abstract:This study develops an electronic annotation system, allowing users to annotate on hypertexts, to build up knowledge structure, and to browse instructions provided by the system administrator or the instructor electronically. The electronic annotation system is a distributed World Wide Web application based on HTTP access and allows annotations on HTML documents. The major functions of the electronic annotation system include highlighting texts, inserting and editing annotations, and organizing and presenting annotations hierarchically. The five interactive components of the electronic annotation system are Main Tool Bar, Hypertext, Annotation Editor, Hierarchy Viewer, and Instruction Viewer. A user test was conducted to investigate the effect of the location of electronic annotating (Annotation Editor) on reading performance in terms of recall and degree of satisfaction.
Improving Web Search Ranking by Incorporating Structured Annotation of Queries.

Xiao Ding,Zhicheng Dou,Bing Qin,Ting Liu,Ji-Rong Wen

DOI: https://doi.org/10.18653/v1/d13-1045

2013-01-01

Abstract:Web users are increasingly looking for structured data, such as lyrics, job, or recipes, using unstructured queries on the web. However, retrieving relevant results from such data is a challenging problem due to the unstructured language of the web queries. In this paper, we propose a method to improve web search ranking by detecting Structured Annotation of queries based on top search results. In a structured annotation, the original query is split into different units that are associated with semantic attributes in the corresponding domain. We evaluate our techniques using real world queries and achieve significant improvement.
Not Enough Labeled Data? Just Add Semantics: A Data-Efficient Method for Inferring Online Health Texts

Joseph Gatto,Sarah M. Preum

DOI: https://doi.org/10.48550/arXiv.2309.09877

2023-09-18

Abstract:User-generated texts available on the web and social platforms are often long and semantically challenging, making them difficult to annotate. Obtaining human annotation becomes increasingly difficult as problem domains become more specialized. For example, many health NLP problems require domain experts to be a part of the annotation pipeline. Thus, it is crucial that we develop low-resource NLP solutions able to work with this set of limited-data problems. In this study, we employ Abstract Meaning Representation (AMR) graphs as a means to model low-resource Health NLP tasks sourced from various online health resources and communities. AMRs are well suited to model online health texts as they can represent multi-sentence inputs, abstract away from complex terminology, and model long-distance relationships between co-referring tokens. AMRs thus improve the ability of pre-trained language models to reason about high-complexity texts. Our experiments show that we can improve performance on 6 low-resource health NLP tasks by augmenting text embeddings with semantic graph embeddings. Our approach is task agnostic and easy to merge into any standard text classification pipeline. We experimentally validate that AMRs are useful in the modeling of complex texts by analyzing performance through the lens of two textual complexity measures: the Flesch Kincaid Reading Level and Syntactic Complexity. Our error analysis shows that AMR-infused language models perform better on complex texts and generally show less predictive variance in the presence of changing complexity.

Computation and Language

Annotating Needles in the Haystack Without Looking

Annotating Needles In The Haystack Without Looking: Product Information Extraction From Emails

Finding Camouflaged Needle in a Haystack?

Automatic Extraction Of Commodity Attributes On Webpages Based On Hierarchical Structure

Crawler Detection in Location-Based Services Using Attributed Action Net

Personalization in Email Marketing: The Role of Non-Informative Advertising Content

Natural language processing on customer note data

"Know" Your Mobile Customers: A Design Approach to an Android-Based Mobile Analytics Tool.

Annotating High-Level Structures of Short Stories and Personal Anecdotes

Semantic annotation of consumer health questions

A STUDY OF B2C CONSUMER BEHAVIOR MODEL

Categorizing Online Shopping Behavior from Cosmetics to Electronics: An Analytical Framework

An Analysis of Structured Data on the Web

Intelligent Parsing: An Automated Parsing Framework for Extracting Design Semantics from E-commerce Creatives

ANSWER: Annotation Software for Electronic Reporting

You Are What You Buy: Personal Information Extraction From Anonymized Data

Data Embroidery with Black-and-White Textures

Leam: An Interactive System for In-situ Visual Text Analysis

A Study of Electronic Annotation on Web Documents

Improving Web Search Ranking by Incorporating Structured Annotation of Queries.

Not Enough Labeled Data? Just Add Semantics: A Data-Efficient Method for Inferring Online Health Texts