Abstract:The public release of Large Language Models (LLMs) has sparked tremendous interest in how humans will use Artificial Intelligence (AI) to accomplish a variety of tasks. In our study conducted with Boston Consulting Group, a global management consulting firm, we examine the performance implications of AI on realistic, complex, and knowledge-intensive tasks. The pre-registered experiment involved 758 consultants comprising about 7% of the individual contributor-level consultants at the company. After establishing a performance baseline on a similar task, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. We suggest that the capabilities of AI create a “jagged technological frontier” where some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI. For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities, consultants using AI were significantly more productive (they completed 12.2% more tasks on average, and completed task 25.1% more quickly), and produced significantly higher quality results (more than 40% higher quality compared to a control group). Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores. For a task selected to be outside the frontier, however, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI. Further, our analysis shows the emergence of two distinctive patterns of successful AI use by humans along a spectrum of human-AI integration. One set of consultants acted as “Centaurs,” like the mythical halfhorse/half-human creature, dividing and delegating their solution-creation activities to the AI or to themselves. Another set of consultants acted more like “Cyborgs,” completely integrating their task flow with the AI and continually interacting with the technology.

Micro-work, artificial intelligence and the automotive industry

Making Data: The Work Behind Artificial Intelligence

Artificial intelligence at work: The problem of managerial control from call centers to transport platforms

Global Inequalities in the Production of Artificial Intelligence: A Four-Country Study on Data Work

The unwitting labourer: extracting humanness in AI training

Patchwork: The Hidden, Human Labor of AI Integration within Essential Work

Waiting for robots: the ever-elusive myth of automation and the global exploitation of digital labor

Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers

Microstock images of artificial intelligence: How AI creates its own conditions of possibility

Digital Labor and the Inconspicuous Production of Artificial Intelligence

Automation, AI & Work

Artificial intelligence, workers, and future of work skills

Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

“Manufacturing Life” in Real Work Processes? New Manufacturing Environments with Micro- and Nanorobotics

AI technologies and employment: micro evidence from the supply side

Friend or Foe? Teaming Between Artificial Intelligence and Workers with Variation in Experience

On the role of Artificial Intelligence methods in modern force-controlled manufacturing robotic tasks

Learning Occupational Task-Shares Dynamics for the Future of Work

Humans + AI: Exploring the Collaboration Between AI and Human Labor in the Workplace

Various perspectives of labor and human resources challenges and changes due to automation and artificial intelligence.

Making AI Machines Work for Humans in FoW