CCKS 2022 - Evaluation Track

Ningyu Zhang,Meng Wang,Tianxing Wu,Wei Hu,Shumin Deng

DOI: https://doi.org/10.1007/978-981-19-8300-9

2022-01-01

Abstract:CCKS 2022 Evaluation proceedings on platforms and resources for testing knowledge and semantic computing technologies, algorithms and systems.

What problem does this paper attempt to address?

Design and Development of a Sharable Clinical Decision Support System Based on a Semantic Web Service Framework

Yi-Fan Zhang,Ling Gou,Yu Tian,Tian-Chang Li,Mao Zhang,Jing-Song Li

DOI: https://doi.org/10.1007/s10916-016-0472-y

IF: 4.92

2016-01-01

Journal of Medical Systems

Abstract:Clinical decision support (CDS) systems provide clinicians and other health care stakeholders with patient-specific assessments or recommendations to aid in the clinical decision-making process. Despite their demonstrated potential for improving health care quality, the widespread availability of CDS systems has been limited mainly by the difficulty and cost of sharing CDS knowledge among heterogeneous healthcare information systems. The purpose of this study was to design and develop a sharable clinical decision support (S-CDS) system that meets this challenge. The fundamental knowledge base consists of independent and reusable knowledge modules (KMs) to meet core CDS needs, wherein each KM is semantically well defined based on the standard information model, terminologies, and representation formalisms. A semantic web service framework was developed to identify, access, and leverage these KMs across diverse CDS applications and care settings. The S-CDS system has been validated in two distinct client CDS applications. Model-level evaluation results confirmed coherent knowledge representation. Application-level evaluation results reached an overall accuracy of 98.66 % and a completeness of 96.98 %. The evaluation results demonstrated the technical feasibility and application prospect of our approach. Compared with other CDS engineering efforts, our approach facilitates system development and implementation and improves system maintainability, scalability and efficiency, which contribute to the widespread adoption of effective CDS within the healthcare domain.
Extension knowledge spaces and extension knowledge grid model(Ⅰ):extension knowledge spaces

SHEN Bin,YAO Min,JIANG Zhi-wei,YI Wen-sheng

DOI: https://doi.org/10.3969/j.issn.1007-0249.2006.04.029

2006-01-01

Abstract:This paper proposes the concept of extension knowledge space(EKS),builds the corresponding operations for EKS,and then establishes the models of semantic web of extension knowledge space(SWEKS)and extension knowledge grid(EKG)based on EKS.We analyze the construction processes of these models in detail.A series of corresponding problems is also discussed,for example,the computing method for correlative function of EKS,corresponding actions of EKS operations in SWEKS and EKG services etc.They have great significance for large scale extension problem computing.
Overview of the CCKS 2019 Knowledge Graph Evaluation Track: Entity, Relation, Event and QA

Xianpei Han,Zhichun Wang,Jiangtao Zhang,Qinghua Wen,Wenqi Li,Buzhou Tang,Qi Wang,Zhifan Feng,Yang Zhang,Yajuan Lu,Haitao Wang,Wenliang Chen,Hao Shao,Yubo Chen,Kang Liu,Jun Zhao,Taifeng Wang,Kezun Zhang,Meng Wang,Yinlin Jiang,Guilin Qi,Lei Zou,Sen Hu,Minhao Zhang,Yinnian Lin

DOI: https://doi.org/10.48550/arXiv.2003.03875

2020-03-09

Abstract:Knowledge graph models world knowledge as concepts, entities, and the relationships between them, which has been widely used in many real-world tasks. CCKS 2019 held an evaluation track with 6 tasks and attracted more than 1,600 teams. In this paper, we give an overview of the knowledge graph evaluation tract at CCKS 2019. By reviewing the task definition, successful methods, useful resources, good strategies and research challenges associated with each task in CCKS 2019, this paper can provide a helpful reference for developing knowledge graph applications and conducting future knowledge graph researches.

Artificial Intelligence,Computation and Language
Knowledge Graph and Semantic Computing. Knowledge Computing and Language Understanding

Jun Zhao,Frank van Harmelen,Jie Tang,Xianpei Han,Quan Wang,Xianyong Li

DOI: https://doi.org/10.1007/978-981-13-3146-6

2019-01-01

Abstract:CCKS 2018 proceedings on knowledge graph, information extraction, question answering, knowledge representation and reasoning, linked data, machine reading, representation learning, knowledge storage and indexing, representation learning, semantic search.
CIKM'15 Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

Haishuai Wang,Peng Zhang,Ivor Tsang,Ling Chen,Chengqi Zhang,CIKM' Conference Committee

2015-01-01

Abstract:The 24th ACM International Conference on Information and Knowledge Management (CIKM 2015) will be held from Oct 19-23, 2015 in Melbourne, Australia. CIKM is a top-tier ACM conference in the areas of information retrieval, knowledge management and databases. Since 1992, it has successfully brought together leading researchers and developers from the three communities, with the purpose of identifying challenging problems facing the development of advanced knowledge and information systems, and shaping future research directions through the publication of high quality, applied and theoretical research findings. With CIKM 2015, we will continue the tradition of promoting collaboration among multiple areas and are arranging an exciting program for the conference, including workshops, tutorials, industry speakers, keynote speakers, research papers, and posters.
Knowledge Graph and Semantic Computing. Language, Knowledge, and Intelligence: Second China Conference, CCKS 2017, Chengdu, China, August 26–29, 2017, Revised Selected Papers

Juanzi Li,Ming Zhou,Guilin Qi,Ni Lao,Tong Ruan,Jianfeng Du

DOI: https://doi.org/10.1007/978-981-10-7359-5

2018-01-01

Abstract:This book constitutes the refereed proceedings of the Second China Conference on Knowledge Graph and Semantic Computing, CCKS 2017, held in Chengdu, China, in August 2017. The 11 revised full papers and 6 revised short papers presented were carefully reviewed and selected from 85 submissions. The papers cover wide research fields including the knowledge graph, the Semantic Web, linked data, NLP, knowledge representation, graph databases.
JCDL2022 Workshop: Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2022)

Chengzhi Zhang,Philipp Mayr,Wei Lu,Yi Zhang

DOI: https://doi.org/10.1145/3529372.3530917

2022-01-01

Abstract:The 3rd Workshop on Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE 2022) was held online at the ACM/IEEE Joint Conference on Digital Libraries (JCDL) 2022. The goal of this workshop series (https://eekeworkshop.github.io/) is to engage the related communities in open problems in the extraction and evaluation of knowledge entities from scientific documents. Participants are encouraged to identify knowledge entities, explore feature of various entities, analyze the relationship between entities, and construct the extraction platform or knowledge base. CCS CONCEPTS • Information systems • Information retrieval • Retrieval tasks and goals • Information extraction
JCDL2022 Workshop

Chengzhi Zhang,Philipp Mayr,Wei Lu,Yi Zhang

DOI: https://doi.org/10.1145/3529372.3530917

2022-01-01

Abstract:The 3rd Workshop on Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE 2022) was held online at the ACM/IEEE Joint Conference on Digital Libraries (JCDL) 2022. The goal of this workshop series (https://eekeworkshop.github.io/) is to engage the related communities in open problems in the extraction and evaluation of knowledge entities from scientific documents. Participants are encouraged to identify knowledge entities, explore feature of various entities, analyze the relationship between entities, and construct the extraction platform or knowledge base.
PKUSpace: A Collaborative Platform for Scientific Researching

Ming Zhang,Dong-qing Yang,Zhi-hong Deng,Ying Feng,Wen-qing Wang,Pei-xiang Zhao,Sai Wu,Shu-an Wang,Shi-wei Tang,Ming Zhang

DOI: https://doi.org/10.1007/978-3-540-27859-7_16

2004-01-01

Abstract:This Paper introduces PKUSpace - a collaborative platform for scientific learning and researching. Supported by efficient automated metadata extraction and powerful text classification modules, PKUSpace provides users with a very helpful and interactive procedure to upload e-resources. We use heuristic methods and regular expression matching techniques to mine different patterns and extract relevant metadata. Upon the well-organized e-sources, we builds a uniform retrieval interface with multi-function to provide taxonomy navigation, full-text search, structural search in the metadata level and meta-searching module with novel result-merging method. Combining two on-line scoring methods: scoring with exponential distribution and metadata scoring, we achieve an effective merging result of matched documents. Furthermore, PKUSpace provides an academic forum for students and professional users in every community to discuss their ideas and research work freely.
Multi-Module System for Open Domain Chinese Question Answering over Knowledge Base

Yang Yiying,He Xiahui,Zhou Kaijie,Wei Zhongyu

2019-01-01

Abstract: For the task of open domain Knowledge Based Question Answering in CCKS2019, we propose a method combining information retrieval and semantic parsing. This multi-module system extracts the topic entity and the most related relation predicate from a question and transforms it into a Sparql query statement. Our method obtained the F1 score of 70.45% on the test data.
Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

Jianzhong Li,X. Sean Wang,Minos Garofalakis,Ian Soboroff,Torsten Suel,Min Wang

DOI: https://doi.org/10.1145/2661829

2014-01-01

Abstract:On behalf of the conference organizing committee, it is our great pleasure to welcome you to the 23rd ACM International Conference on Information and Knowledge Management (CIKM 2014). CIKM is a top-tier ACM conference in Databases, Information Retrieval, and Knowledge Management. The purpose of the conference is to identify challenging problems facing the development of future knowledge and information systems, and to shape future research directions through the publication of high quality, applied and theoretical research findings. The 23rd edition of CIKM continues the tradition of promoting collaboration among multiple areas. The conference this year has attracted 838 valid full paper submissions, 260 valid poster submissions, 73 valid demonstration submissions, and 15 valid workshop proposals. Among them, we accepted 175 full papers, 57 posters, 29 demonstrations, and 9 workshops. On top of the regular track, the conference has an outstanding keynote program, an exciting industry event, and six tutorials of contemporary topics. This time again, the conference is standing out among the many of its peers in its ability to attract researchers to exchange ideas and to interact, and the sheer numbers are a testament of the vitality of the three research areas and their interactions. We are honored to present three distinguished keynote speakers: Jeff Dean of Google, Qi Lu of Microsoft, and Gerhard Weikum of Max-Planck Institute for Informatics. We are also honored to present six industry event invitees to share their deep knowledge and insights: Soumen Chakrabarti, Chih Jen Lin, Wei-Ying Ma, Alex J. Smola, Limsoon Wong, and Tong Zhang. At CIKM 2014, we inaugurate a programming competition, sponsored by Baidu Inc. The competition has attracted 571 teams to sign up from around the world, with top-ranked teams achieving significant results on the task given.
Big Data and Social Computing

Xiaofeng Meng,Qi Xuan,Yang,Yue Yang,Zi-Ke Zhang

DOI: https://doi.org/10.1007/978-981-19-7532-5

2022-01-01

Abstract:BDSC 2022 proceedings on urban computing and social governance; artificial intelligence and cognitive science; social network and group behavior
Foundations and Practical Applications of: Cognitive Systems and Information Processing

Dewen Hu,Huaping Liu

2014-01-01

Abstract:This book is part of the Proceedings of the Seventh International Conference on Intelligent Systems and Knowledge Engineering (ISKE2012) and the First International Conference on Cognitive Systems and Information Processing (CSIP2012) held in Beijing, China, during December 15–17, 2012. ISKE is a prestigious annual conference on Intelligent Systems and Knowledge Engineering with the past events held in Shanghai (2006, 2011), Chengdu (2007), Xiamen (2008), Hasselt, Belgium (2009), and Hangzhou (2010). Over the past few years, ISKE has matured into a well-established series of International Conferences on Intelligent Systems and Knowledge Engineering and related fields over the world. CSIP2012 is the first conference sponsored by Tsinghua University and Science China Press, and technically sponsored by IEEE Computational Intelligence Society, Chinese Association for Artificial …
An Evaluation of Chinese Human-Computer Dialogue Technology

Zixian Feng,Caihai Zhu,Weinan Zhang,Zhigang Chen,Wanxiang Che,Minlie Huang,Linlin Li

DOI: https://doi.org/10.1162/dint_a_00090

2021-01-01

Data Intelligence

Abstract:There is a growing interest in developing human-computer dialogue systems which is an important branch in the field of artificial intelligence (AI). However, the evaluation of large-scale Chinese human-computer dialogues is still a challenging task. To attract more attention to dialogue evaluation work, we held the fourth Evaluation of Chinese Human-Computer Dialogue Technology (ECDT). It consists of few-shot learning in spoken language understanding (SLU) (Task 1) and knowledge-driven multi-turn dialogue competition (Task 2), the data sets of which are provided by Harbin Institute of Technology and Tsinghua University. In this paper, we will introduce the evaluation tasks and data sets in detail. Meanwhile, we will also analyze the evaluation results and the existing problems in the evaluation.
Knowledge-Enhanced Retrieval: A Scheme for Question Answering

Fake Lin,Weican Cao,Wen Zhang,Liyi Chen,Yuan Hong,Tong Xu,Chang Tan

DOI: https://doi.org/10.1007/978-981-19-0713-5_12

2022-01-01

Abstract:Chinese Knowledge Base Question Answering (CKBQA), as a significant task in natural language processing, has drawn massive attention from both industry and academia. However, previous studies mainly concentrated on multi-hop questions, which may limit the performance of tackling complex natural questions with various forms. To that end, in this paper, we propose a comprehensive technical framework called Knowledge-Enhanced Retrieval Question Answering (KERQA) for tackling complex questions, which could precisely extract the gold answers to these questions from a large-scale knowledge graph. Specifically, our proposed KERQA follows the pipeline with five modules, including the Question Classification module to categorize questions, the Named Entity Recognition module to extract mentions, and the Entity Linking module to match entities in the knowledge graph (KG). Along this line, we further design the Path Generation module to associate the paths in the KG with predefined templates, as well as the Path Ranking module to capture the best path. Extensive validations demonstrate the effectiveness of our KERQA framework, which achieved an F1 score of 78.78% on the final leaderboard of the CCKS 2021 KBQA contest.
An Evaluation System for Algorithms of Automated Text Categorization on Chinese

XU Wei,DONG Yuan,ZHANG Su-Qin

DOI: https://doi.org/10.3969/j.issn.1002-137X.2007.08.048

2007-01-01

Computer Science

Abstract:Automated text categorization on Chinese helps people make more effective use of the growing Chinese information on the Internet.Current algorithms have different performance according to the environment.Evaluation to the algorithms tells the feature of certain algorithm.At present,there is no evaluation system form algorithms of automated text categorization on Chinese.This paper introduces such a evaluating system.An open research platform based on it is also introduced,with results of some algorithms.
TREC iKAT 2023: The Interactive Knowledge Assistance Track Overview

Mohammad Aliannejadi,Zahra Abbasiantaeb,Shubham Chatterjee,Jeffery Dalton,Leif Azzopardi

2024-02-23

Abstract:Conversational Information Seeking has evolved rapidly in the last few years with the development of Large Language Models providing the basis for interpreting and responding in a naturalistic manner to user requests. iKAT emphasizes the creation and research of conversational search agents that adapt responses based on the user's prior interactions and present context. This means that the same question might yield varied answers, contingent on the user's profile and preferences. The challenge lies in enabling Conversational Search Agents (CSA) to incorporate personalized context to effectively guide users through the relevant information to them. iKAT's first year attracted seven teams and a total of 24 runs. Most of the runs leveraged Large Language Models (LLMs) in their pipelines, with a few focusing on a generate-then-retrieve approach.

Information Retrieval,Artificial Intelligence,Computation and Language
Panel: Computational Methods about Knowledge Graph

Jie Tang,Jure Leskovec,Kuansan Wang,Deborah McGuinness,Hassan Sawaf

DOI: https://doi.org/10.1145/3308560.3317712

2019-05-13

Abstract:This panel will focus on the cutting-edge computation methods, which can be applied to knowledge graph, such as latest NLP technologies to extract entities and relationships to build knowledge graphs, machine learning or deep learning methods on mining knowledge graph, and intelligent search or recommendations powered by knowledge graphs. Panelists are: Professor and the Vice Chair of the Department of Computer Science and Technology of Tsinghua University. I obtained my Ph.D. in DCST of Tsinghua University in 2006. My research interests include artificial intelligence, data mining, social networks, machine learning and knowledge graph, with an emphasis on designing new algorithms for mining social and knowledge networks. Associate Professor of Computer Science at Stanford University. My research focuses on mining and modeling large social and information networks, their evolution, and diffusion of information and influence over them. Problems I investigate are motivated by large scale data, the web and on-line media Managing Director of MSR Outreach, an organization with the mission to serve the research community. In addition to applying the intelligent technologies to make Bing and Cortana smarter in gathering and serving academic knowledge, we are also starting an experimental website, academic.microsoft.com (powered by Academic API), and mobile apps dedicated to exploring new service scenarios for active researchers like myself A leading expert in knowledge representation and reasoning languages and systems and has worked in ontology creation and evolution environments for over 20 years. Most recently, McGuinness is best known for her leadership role in semantic web research, and for her work on explanation, trust, and applications of semantic web technology, particularly for scientific applications. Director of Artificial Intelligence at Amazon Web Services, passionate about opening up new markets and opportunities with smart application of Artificial Intelligence and application-driven research in AI, in particular in language technology, speech processing, computer vision and computational reasoning.
JCDL2023 Workshop: Joint Workshop of the 4th Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2023) and the 3rd AI + Informetrics (AII2023)

Chengzhi Zhang,Yi Zhang,Philipp Mayr,Wei Lu,Arho Suominen,Haihua Chen,Ying Ding

DOI: https://doi.org/10.1109/jcdl57899.2023.00070

2024-01-01

Abstract:The 4th Workshop on Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE 2023) and the 3rd AI + Informetrics (AII2023) will be held in New Mexico, USA and online at the ACM/IEEE Joint Conference on Digital Libraries (JCDL) 2023. The goal of these two workshop series (https://eeke-workshop.github.io/ and https://ai-informetrics.github.io/) are to engage the related communities in open problems in the extraction and evaluation of knowledge entities from scientific documents and the modeling and applications of AI + Informetrics for broad interests in science of science, science, technology, & innovation (ST&I), etc. This joint workshop consists of keynotes, oral presentations, and posters, and will attract not only academic researchers and librarians but also decision makers from governments and practical sectors.
Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Ee Peng Lim,Marianne Winslett,Mark Sanderson,Ada Fu,Jimeng Sun,Shane Culpepper,Eric Lo,Joyce C. Ho,Debora Donato,Rakesh Agrawal,Yu Zheng,Carlos Castillo,Aixin Sun,Vincent S. Tseng,Chenliang Li

2017-01-01

Abstract:Since 1992, the ACM International Conference on Information and Knowledge Management (CIKM) has brought together leading researchers and developers from the knowledge management, information retrieval, and data management communities to discuss cutting-edge research on advanced knowledge and information systems. We are pleased to present the 26th edition of CIKM on 6-10 November, 2017, at the Pan Pacific Singapore hotel, with the special theme of Smart Cities, Smart Nations. This year our attendees will enjoy four keynote speakers: Rajeev Rastogi (Amazon), Qiang Yang (HKUST), Rada Mihalcea (Michigan), and K Ananth Krishnan (Tata Consultancy Services). In 6-7 parallel sessions, our program includes presentations of 171 full research papers, 119 short research papers, and 30 demonstrations of new research advances. The program's focus this year can be seen at a glance in the word cloud at right, constructed from the titles of all accepted research papers. Also on offer are eight tutorials on timely research topics, and six collocated workshops on topics ranging from history to transportation, biomedicine to bias. We are excited about our greatly expanded data analytics competition this year, the CIKM AnalytiCup. During the past nine months, over 1500 teams from all over the world have vied to win over $60,000 in AnalytiCup prizes and travel money by solving real-world analytics problems posed by our corporate sponsors Alibaba/Shenzhen Meteorological Bureau, DataSpark, and Lazada. A fourth competition, a weekend-long hackathon sponsored by DHL, takes place immediately before the conference. The finalists from all four competitions come together on 6 November for a final showdown in front of corporate judges. Solution summaries from finalist teams in the first three competitions can be found in these proceedings. Also new this year are several other events aimed directly at practitioners. During the main conference, we are offering hands-on tutorials on the hot topics of scalable deep learning and scalable data science. The Case Studies track, intended to highlight the experiences and lessons learned by early adopters, debuts this year with 23 studies of technology adoption in interesting applications. And immediately before the main conference, CIKMconnect brings together students and industry for posters, technical discussions, recruiting events, and networking. It takes a village to produce a major conference! Our program committee chairs, senior PC and PC members valiantly and gracefully handled a record total number of submissions: 855 full research papers, 419 short research papers, 80 demos, and 103 case studies. Each submission was reviewed by three program committee members, each a recognized expert in the field, and an independent committee selected the full paper awards recipients.

CCKS 2022 - Evaluation Track

Design and Development of a Sharable Clinical Decision Support System Based on a Semantic Web Service Framework

Extension knowledge spaces and extension knowledge grid model(Ⅰ):extension knowledge spaces

Overview of the CCKS 2019 Knowledge Graph Evaluation Track: Entity, Relation, Event and QA

Knowledge Graph and Semantic Computing. Knowledge Computing and Language Understanding

CIKM'15 Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

Knowledge Graph and Semantic Computing. Language, Knowledge, and Intelligence: Second China Conference, CCKS 2017, Chengdu, China, August 26–29, 2017, Revised Selected Papers

JCDL2022 Workshop: Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2022)

JCDL2022 Workshop

PKUSpace: A Collaborative Platform for Scientific Researching

Multi-Module System for Open Domain Chinese Question Answering over Knowledge Base

Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

Big Data and Social Computing

Foundations and Practical Applications of: Cognitive Systems and Information Processing

An Evaluation of Chinese Human-Computer Dialogue Technology

Knowledge-Enhanced Retrieval: A Scheme for Question Answering

An Evaluation System for Algorithms of Automated Text Categorization on Chinese

TREC iKAT 2023: The Interactive Knowledge Assistance Track Overview

Panel: Computational Methods about Knowledge Graph

JCDL2023 Workshop: Joint Workshop of the 4th Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2023) and the 3rd AI + Informetrics (AII2023)

Proceedings of the 2017 ACM on Conference on Information and Knowledge Management