The Storage Optimization of XML Metadata in Digital Library

李然,张云霞,汪卫,施伯乐

DOI: https://doi.org/10.3969/j.issn.1002-137X.2002.11.025

2002-01-01

Computer Science

Abstract:On how to storage XML metadata in digital library and make optimization according to the statistics gener ated by the Database engine,this paper introduces a feasible solution which involves view materialization and colonygenetics algorism.

What problem does this paper attempt to address?

A digital library architecture supporting massive small files and efficient replica maintenance.

Chunhui Shen,Weiming Lu,Jiangqin Wu,Baogang Wei

DOI: https://doi.org/10.1145/1816123.1816195

2010-01-01

Abstract:ABSTRACTIn this paper, we presented a service infrastructure based on distributed file system for massive storage in digital library. In addition, we addressed the small-file problem by merging small files into big ones, and proposed a novel dynamic replica number adjustment scheme to ensure the maximal availability and reliability in a limited storage space.
A Scalable, Adaptive, Self-management and Fault-Tolerant Architecture for Digital Library

Liangju Zheng,Weiming Lu,Meng Zhang,Baofeng Zhang,Baogang Wei

DOI: https://doi.org/10.1007/978-3-642-34752-8_15

2012-01-01

Abstract:With the rapid growth of data, two major challenges are taking place in digital library technologies. One is the transition from database to Data Ocean; the other is the transition from digital library to Smart Library. In order to address these challenges, we propose a scalable, adaptive, self-management, fault-tolerant architecture for various services in the digital library. Users can deploy services quite easily on the architecture, and it also enables the deployed services load balancing, scalable, adaptive and fault-tolerant. The practice in CADAL demonstrates the feasibility and efficiency of our architecture.
A Technique to Generate and Optimize the Materialized Model for XML Data

Liang ZHANG,Ran LI,Wei WANG,Bai-Le SHI

DOI: https://doi.org/10.1360/jos180323

2007-01-01

Journal of Software

Abstract:One way to improve the performance of XML (extensible markup language) management systems is to materialize part of the XML documents and store them aside in cache memory. In this paper, a method is presented to characterize query sets of XML data as schema graph, which is a technique to generate materialized plan based on the distribution of user's queries. Experimental results demonstrate its performance gain in XML cache
A Survey on Storage Strategies of XML Database

MEN Ai-Hua,FENG Jian-Hua,ZHOU Li-Zhu

DOI: https://doi.org/10.3969/j.issn.1002-137X.2005.09.002

2005-01-01

Computer Science

Abstract:XML is a subset of SGML, and is a specific SGML markup language in its essence. XML has become the new criteria for the data expression and data exchange on Internet and is regarded as the most promising formation way for semi-structured data. The main function of XML is to manage the information data rather than the style of the data. The clear labor division of XML will lead to the appearance of more efficient Web programming, quicker explorer, more consistent data expression and more convenient data exchange. The lower layer storage design has great influence on the upper layer inquiry processing and optimization, so which is the best way to store XML files has become an im- portant issue. The article introduces several storage strategies for XML database, providing careful description and a- nalysis for each of them, and compares their functions and their advantages and disadvantages.
XML-based Complex Lab Data Representing, Storing and Publishing

Shao Zhijiang,Wang Yongming

DOI: https://doi.org/10.1109/wcica.2004.1342196

2004-01-01

Abstract:The problem of heterogeneous data integration in industrial supervisory control is presented and analyzed. The demand and difficulties of complex lab data representing, storing and publishing are outlined and analyzed in this paper. It is pointed out that a self-descriptive data structure with clear organization of contents and easy connection to human-machine interface is needed. A solution based on XML (eXtended Mark-up Language) is proposed with combination of different layers of data storage, data logic and presentation, human-machine interface. An XML-based system is developed to meet the demands of laboratory data publishing with complex structure. Construction of user interface from XML schema, conversion of lab data into XML, algorithm of storing XML into RDBMS (relation database manager system), query and reconstruction of XML documents from the RDBMS are touched upon. The data structure is described in details and its unique advantage is presented.
Indexing Techniques For Query Of Xml Documents

Yuan Wang,Jianlin Sun,Jinxiang Dong

2001-01-01

Abstract:The popularity of XML in WWW calls for further research on query optimization for XML documents. In this paper, we introduce a universal indexing technique for XML documents based on DOM API and the XQuery Language. Then the design of a new kind of path indexing technique is presented, not only notably reducing the cost of query processing and facilitating its building and maintenance which would be provided later in this paper but also contributes to cost model and query visualization. Finally, we conclude the paper and address the future work.
Optimized Query Translation Strategy for XML Stored in Relational Database

Hongzhi Wang,Jianzhong Li,Zhenying He

DOI: https://doi.org/10.1007/978-3-540-27772-9_38

2004-01-01

Abstract:XML is an important standard of data exchange and representation. XML database also becomes important as web database. As a mature database system, using relational database to support XML data may bring some advantages. In this paper, based on a relational storage structure support XPath query effectively, an optimized query translation strategy is presented as well as optimization techniques. This method converts XQuery to variable forest. Optimization operations are applied on the variable forest. The optimization is based on the semantics of XML document such as schema of original XML document. The VF can be converted to SQL easily. This method translates XQuery to a set of SQL statements and generates final result through the result of them. Our method adapts to the XML database with relational storage supporting effective query process to XML documents with various schemas and size.
Query Performance Optimization in XML Data Warehouses

Hadj Mahboubi,Jérôme Darmont

DOI: https://doi.org/10.4018/978-1-61692-016-6.ch014

2017-01-27

Abstract:XML data warehouses form an interesting basis for decision-support applications that exploit complex data. However, native-XML database management systems (DBMSs) currently bear limited performances and it is necessary to research for ways to optimize them. In this chapter, we present two such techniques. First, we propose a join index that is specifically adapted to the multidimensional architecture of XML warehouses. It eliminates join operations while preserving the information contained in the original warehouse. Second, we present a strategy for selecting XML materialized views by clustering the query workload. To validate these proposals, we measure the response time of a set of decision-support XQueries over an XML data warehouse, with and without using our optimization techniques. Our experimental results demonstrate their efficiency, even when queries are complex and data are voluminous.

Databases
Application of XML Tree with Incomplete Information in Query Efficiency Improvement

肖金升,宋军,施伯乐

DOI: https://doi.org/10.3969/j.issn.1000-3428.2003.20.018

2003-01-01

Abstract:A lot of research has been done to improve the query efficiency of research engines. In the digital library project, people uses a method of taking full use of the previous results of users to improve the query efficiency. It uses the XML tree with incomplete information to record the results of the previous queries and uses the results to answer the following queries. The XML tree with incomplete information will be modified after each query.
Optimization Technology of Domain-special Data Transmission Based on XML

ZHAO Chong-chong,WANG Xin,HU Chang-jun

DOI: https://doi.org/10.3969/j.issn.1002-137X.2009.08.044

2009-01-01

Computer Science

Abstract:XML are widely used in various areas due to the usability,comprehensibility and cross-platform attribute.Although the XML technique can deal with the cross-platform transmission of domain data efficiently,the performance of this technique is limited by the fact that a great deal of additional data may be transmitted to keep integrality.This transmission problem not only increases the pressure of network,but also delays the response of data reserve.This paper proposed an optimized transmission technique for domain-special data based on XML.The thesis of this technique is to keep the attribute of XML,and to organize the elements of XML file in term of certain logical relationship and domain knowledge,i.e.based on certain rules,the XML data is divided into a series of classes to be transmitted without entire XML file.This approach can effectively resolve the problem arosen by the fact that the partial modification of XML data files lower the performance of system.Rules and method of the various granularity partitions were given.Application condition and efficiency of the method were given too.Proposed methods which have been applied in petroleum field can improve the efficiency of transmission.
XML2HBase: Storing and querying large collections of XML documents using a NoSQL database system

Liang Bao,Jin Yang,Chase Q. Wu,Haiyang Qi,Xin Zhang,Shunda Cai

DOI: https://doi.org/10.1016/j.jpdc.2021.11.003

IF: 4.542

2022-03-01

Journal of Parallel and Distributed Computing

Abstract:Many big data applications such as smart transportation, healthcare, and e-commerce need to store and query large collections of small XML documents, which has become a fundamental problem. However, existing solutions are inadequate to deliver satisfactory query performance in such circumstances. In this paper, we propose a framework named XML2HBase to address this problem using HBase, a widely deployed NoSQL database. Within this framework, we design a novel encoding scheme called Pathed-Dewey Order and a two-layer mapping method to store XML documents in HBase tables. XML queries, which are represented as XPath expressions, are evaluated through their translation into queries over HBase tables. Based on an in-depth analysis of the characteristics of the proposed approach, we design and integrate four optimization strategies to reduce storage space and query response time. Extensive experiments on two well-known XML benchmarks demonstrate the superior performance of XML2HBase over three state-of-the-art methods.

computer science, theory & methods
Highly Efficient Processing of XML Path/twig Queries Using Index Caches.

Shichuan Li,Dongqing Yang,Tengjiao Wang,Yue Wang

DOI: https://doi.org/10.1109/fskd.2012.6233729

2012-01-01

Abstract:XML has been becoming the de facto standard for data exchange on the World Wide Web. Now, while XML data files are become more, people have developed a variety of storage methods of the XML file, such as the use of relational databases, native XML databases, and so on. Meanwhile, people have a higher demand on the technology to efficient processing of large XML data sets, now the main direction of research is XML index technology, on the one hand, the twig matching process is more important. Many researchers have done a lot of research in this area. This paper proposed an XML Index Cache designed to improve the efficiency of matching XML indexes, including the simple path matching and twig matching. This paper also presents the structure of XML Index Cache technology and related algorithms. Finally, we do the experiments in CoSQL RX 1.0 that is dual engine database system which was independently developed by the laboratory of Peking University.
Metadata designing and realizing in land use management information system.

XU Cai-jiang,CHEN Zhi-rong,LIU Ren-yi,YANG Xiao-mei

DOI: https://doi.org/10.3321/j.issn:1008-9497.2006.06.024

2006-01-01

Abstract:Metadata make the spatial data more effective and have great benefits in data managing,using and sharing.The method how to design and realize land use metadata management subsystem has been especially discussed.Relationship of metadata inscape is first analyzed,then the module of managing metadata is designed and the process of designing metadata subsystem is analyzed.The XML(Extensible Markup Language) document is used as the way of organizing the content of land us metadata and the new technology XSLT(Extensible StyleSheet Language Transformations) is used to transform the document created by XML.The result is successfully integrated in the system of LanEx.The method that the paper adopted is of typical and general.
Lazy XML Updates: Laziness as a Virtue of Update and Structural Join Efficiency.

Barbara Catania,Wen Qiang Wang,Beng Chin Ooi,Xiaoling Wang

DOI: https://doi.org/10.1145/1066157.1066216

2005-01-01

Abstract:XML documents are normally stored as plain text files. Hence, the natural and most convenient way to update XML documents is to simply edit the text files. But efficient query evaluation algorithms require XML documents to be indexed. Every element is given a unique identifier based on its location in the document or its preorder-traversal order, and this identifier is later used as (part of) the key in the index. Reassigning orders of possibly a large number of elements is therefore necessary when the original XML documents are updated. Immutable dynamic labeling schemes have been proposed to solve this problem, that, however, require very long labels and may decrease query performance. If we consider a real-world scenario, we note that many relatively small ad-hoc XML segments are inserted/deleted into/from an existing XML database. In this paper, we start from this consideration and we propose a new lazy approach to handle XML updates that also improves query performance. The lazy approach: (i) completely avoids reassigning existing element orders after updates; (ii) improves query processing by taking advantages from segments. Experimental results show that our approach is much more efficient in handling updates than using immutable labeling and, at the same time, it also improves the performance of recently defined structural join algorithms.
Optimizing the storage of massive electronic pedigrees in HDFS

Yin Zhang,Weili Han,Wei Wang,Chang Lei

DOI: https://doi.org/10.1109/IOT.2012.6402306

2012-01-01

IoT

Abstract:Benefiting from trustworthily tracking of the processes in the production, processing, storage, transportation and sale phases, an electronic pedigree system becomes an important technology of the Internet of Things. In an electronic pedigree system, small-sized but huge volume of electronic pedigrees in the XML format will be generated, stored, and retrieved. Unfortunately, study of these massive electronic pedigrees' storage in an electronic pedigree system, which is in the form of small XML files, is rarely concerned. We, therefore, try to leverage Hadoop to solve the storage problem of massive electronic pedigrees, by the optimization of storing and accessing massive small XML files in HDFS. First, all correlated small XML files of the same envelope are merged into a larger file to reduce the metadata occupation at NameNode. Second, a prefetching mechanism and a remerging mechanism are used to improve the efficiency of accessing small XML files. Finally, we implement a prototype to evaluate the effectiveness and efficiency comparing with the origin HDFS. The results show that the optimized approach is able to reduce the memory consumption of NameNodes by up to 50%, improve performance of storing by up to 91%, and accelerate accessing by up to 88% in Hadoop.
Measuring tag weighing in XML keyword search

Qingtao Wu,Ying Lou,Ruijuan Zheng,Mingchuan Zhang,Wangyang Wei

2012-01-01

Journal of Information and Computational Science

Abstract:Keyword search over XML databases has attracted a lot of attention recently. Existing approaches mainly adopt the concept of lowest common ancestor (LCA), which excessively emphasizes the structural relationships among nodes in XML data. However, semantics of XML data are not fully exploited. The semantics is implied by XML tags. Inspired from relational databases and information entropy, we study how to use the tag weighing to improve the quality of XML keyword search results. We propose the notion of Tag Weighing (TW), which can reflect the importance of value nodes that contained by the tags. Finally, we implement the proposed techniques in our ranking schema. The extensive experiments demonstrate effectiveness of our approach. Copyright © 2012 Binary Information Press.
Efficient Processing of Secured XML Metadata

Ling Feng,Willem Jonker

DOI: https://doi.org/10.1007/978-3-540-39962-9_72

2003-01-01

Abstract:Metadata management is a key issue in intelligent Web-based environments. It plays an important role in a wide spectrum of areas, ranging from semantic explication, information handling, knowledge management, multimedia processing to personalized service delivery. As a result, security issues around metadata management needs to be addressed in order to build trust and confidence to ambient environments. The aim of this paper is to bring together the worlds of security and XML-formatted metadata management in such a way that, on the one hand the requirement on secure metadata management is satisfied, while on the other other hand the efficiency on metadata processing can still be guaranteed. To this end, we develop an effective approach to enable efficient search on encrypted XML metadata. The basic idea is to augment encrypted XML metadata with encodings which characterize the topology and content of every tree-structured XML metadata, and then filter out candidate data for decryption and query execution by examining query conditions against these encodings. We describe a generic framework consisting of three phases, namely, query preparation, query pre-processing and query execution, to implement the proposed search strategy.
An Optimization of Twig Query Processing on XML Data with Dirty Tags

JIANG Guohua,JIANG Shouxu,LI Jianzhong

DOI: https://doi.org/10.3969/j.issn.2095-2163.2011.04.012

2011-01-01

Abstract:Dirty data such as incorrect data,inconsistent data,imprecise data bring challenges to effective query processing on XML.The paper focuses on processing twig queries on XML with dirty tags.And it separately proposes the principle,pseudo code,correctness proof and complexity analysis of the optimization,examples are also given in the paper.Ultimately,experimental evaluation of efficiency is made on the optimization.
Towards optimal workload-aware XML to relational schema mapping

Xiaoling Wang,Jinfeng Luan,Guimei Liu,Aoying Zhou

DOI: https://doi.org/10.1007/s10479-008-0361-y

IF: 4.82

2008-01-01

Annals of Operations Research

Abstract:Storing XML documents in relational databases has drawn much attention in recent years because it can leverage existing investments in relational database technologies. Different algorithms have been proposed to map XML DTD/Schema to relational schema in order to store XML data in relational databases. However, most work defines mapping rules based on heuristics without considering application characteristics, hence fails to produce efficient relational schema for various applications. In this paper, we propose a workload-aware approach to generate relational schema from XML data and user specified workload. Our approach adopts the genetic algorithm to find optimal mappings. An elegant encoding method and related operations are proposed to manipulate mappings using bit strings. Various techniques for optimization can be applied to the XML to relational mapping problem based on this representation. We implemented the proposed algorithm and our experiment results showed that our algorithm was more robust and produced better mappings than existing work.
Metadata management system: design and implementation

Shien‐Chiang Yu,Kun‐Yung Lu,Ruey‐Shun Chen

DOI: https://doi.org/10.1108/02640470310470525

2003-04-01

The Electronic Library

Abstract:Commonly, an organizational information system may have various data types and directory formats. It usually employs different metadata formats to represent the documents. Although the metadata system can cover the different formats of documents, there still exist the integration problems in various metadata systems. This may lower the performance of data processing and impede information sharing. Research focuses on the model of using multi‐XML schema to construct an XML system framework. Through a complete hierarchical tree structure definition of inner elements, the proposed model can settle the weakness of traditional object‐oriented languages in information sharing; it can also eliminate the constraints of storage and management among heterogeneous metadata while processing different metadata information.

information science & library science

The Storage Optimization of XML Metadata in Digital Library

A digital library architecture supporting massive small files and efficient replica maintenance.

A Scalable, Adaptive, Self-management and Fault-Tolerant Architecture for Digital Library

A Technique to Generate and Optimize the Materialized Model for XML Data

A Survey on Storage Strategies of XML Database

XML-based Complex Lab Data Representing, Storing and Publishing

Indexing Techniques For Query Of Xml Documents

Optimized Query Translation Strategy for XML Stored in Relational Database

Query Performance Optimization in XML Data Warehouses

Application of XML Tree with Incomplete Information in Query Efficiency Improvement

Optimization Technology of Domain-special Data Transmission Based on XML

XML2HBase: Storing and querying large collections of XML documents using a NoSQL database system

Highly Efficient Processing of XML Path/twig Queries Using Index Caches.

Metadata designing and realizing in land use management information system.

Lazy XML Updates: Laziness as a Virtue of Update and Structural Join Efficiency.

Optimizing the storage of massive electronic pedigrees in HDFS

Measuring tag weighing in XML keyword search

Efficient Processing of Secured XML Metadata

An Optimization of Twig Query Processing on XML Data with Dirty Tags

Towards optimal workload-aware XML to relational schema mapping

Metadata management system: design and implementation