Abstract:Radiologists today play a key role in making diagnostic decisions and labeling images for training A.I. algorithms. Low inter-reader reliability (IRR) can be seen between experts when interpreting challenging cases. While teams-based decisions are known to outperform individual decisions, inter-personal biases often creep up in group interactions which limit non-dominant participants from expressing true opinions. To overcome the dual problems of low consensus and inter-personal bias, we explored a solution modeled on biological swarms of bees. Two separate cohorts; three radiologists and five radiology residents collaborated on a digital swarm platform in real time and in a blinded fashion, grading meniscal lesions on knee MR exams. These consensus votes were benchmarked against clinical (arthroscopy) and radiological (senior-most radiologist) observations. The IRR of the consensus votes was compared to the IRR of the majority and most confident votes of the two <a class="link-external link-http" href="http://cohorts.The" rel="external noopener nofollow">this http URL</a> radiologist cohort saw an improvement of 23% in IRR of swarm votes over majority vote. Similar improvement of 23% in IRR in 3-resident swarm votes over majority vote, was observed. The 5-resident swarm had an even higher improvement of 32% in IRR over majority vote. Swarm consensus votes also improved specificity by up to 50%. The swarm consensus votes outperformed individual and majority vote decisions in both the radiologists and resident cohorts. The 5-resident swarm had higher IRR than 3-resident swarm indicating positive effect of increased swarm size. The attending and resident swarms also outperformed predictions from a state-of-the-art A.I. algorithm. Utilizing a digital swarm platform improved agreement and allows participants to express judgement free intent, resulting in superior clinical performance and robust A.I. training labels.

Conversational Swarms of Humans and AI Agents enable Hybrid Collaborative Decision-making

Conversational Swarm Intelligence amplifies the accuracy of networked groupwise deliberations

Conversational Swarm Intelligence (CSI) Enhances Groupwise Deliberation

Towards Collective Superintelligence: Amplifying Group IQ using Conversational Swarms

Towards Collective Superintelligence, a Pilot Study

ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams

Keeping Humans in the Loop: Pooling Knowledge through Artificial Swarm Intelligence to Improve Business Decision Making

Utilizing a digital swarm intelligence platform to improve consensus among radiologists and exploring its applications

AI can help humans find common ground in democratic deliberation

How Being Outvoted by AI Teammates Impacts Human-AI Collaboration

Human-AI Collaboration in a Cooperative Game Setting

Can Robotic Cues Manipulate Human Decisions? Exploring Consensus Building via Bias-Controlled Non-linear Opinion Dynamics and Robotic Eye Gaze Mediated Interaction in Human-Robot Teaming

Revisiting Citizen Science Through the Lens of Hybrid Intelligence

Cognitive swarming: an approach from the theoretical neuroscience of hippocampal function

Artificial Intelligence-Augmented Brainstorming: How Humans and Ai Beat Humans Alone

Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making

Conversational Crowdsensing: A Parallel Intelligence Powered Novel Sensing Approach

A2C: A Modular Multi-stage Collaborative Decision Framework for Human-AI Teams

Entertaining and Opinionated but Too Controlling: A Large-Scale User Study of an Open Domain Alexa Prize System

Using Artificial Intelligence to Accelerate Collective Intelligence: Policy Synth and Smarter Crowdsourcing

On the evolution of adaptable and scalable mechanisms for collective decision-making in a swarm of robots