Abstract:Web archives, a key area of digital preservation, meet the needs of journalists, social scientists, historians, and government organizations. The use cases for these groups often require that they guide the archiving process themselves, selecting their own original resources, or seeds, and creating their own web archive collections. We focus on the collections within Archive-It, a subscription service started by the Internet Archive in 2005 for the purpose of allowing organizations to create their own collections of archived web pages, or mementos. Understanding these collections could be done via their user-supplied metadata or via text analysis, but the metadata is applied inconsistently between collections and some Archive-It collections consist of hundreds of thousands of seeds, making it costly in terms of time to download each memento. Our work proposes using structural metadata as an additional way to understand these collections. We explore structural features currently existing in these collections that can unveil curation and crawling behaviors. We adapt the concept of the collection growth curve for understanding Archive-It collection curation and crawling behavior. We also introduce several seed features and come to an understanding of the diversity of resources that make up a collection. Finally, we use the descriptions of each collection to identify four semantic categories of Archive-It collections. Using the identified structural features, we reviewed the results of runs with 20 classifiers and are able to predict the semantic category of a collection using a Random Forest classifier with a weighted average F1 score of 0.720, thus bridging the structural to the descriptive. Our method is useful because it saves the researcher time and bandwidth. Identifying collections by their semantic category allows further downstream processing to be tailored to these categories.

Digital humanities and web archives: Possible new paths for combining datasets

Web archives as a data resource for digital scholars

Archival strategies for contemporary collecting in a world of big data: Challenges and opportunities with curating the UK web archive

Archives, linked data and the digital humanities: increasing access to digitised and born-digital archives via the semantic web

Big Data Science Over the Past Web

Digital Data Archives as Knowledge Infrastructures: Mediating Data Sharing and Reuse

Methods and Approaches to Using Web Archives in Computational Communication Research

Lost but not forgotten: finding pages on the unarchived web

Full-Text and URL Search Over Web Archives

Web Archives Preserve Our Digital Collective Memory

Web archives for data collection: An ethics case study

The Many Shapes of Archive-It

Archaeology in the Digital Age: From Paper to Databases

Digital Archives as Big Data

The Archives Unleashed Project: Technology, Process, and Community to Improve Scholarly Access to Web Archives

From Web Archive to WebDigest: Concept and Examples

Understanding Web Archiving Services and Their (Mis)Use on Social Media

Creating Structure in Web Archives With Collections: Different Concepts From Web Archivists

Just Google It - Digital Research Practices of Humanities Scholars

Digital Collections as Data: A New Direction for Libraries

Using the wayback machine to mine websites in the social sciences: A methodological resource