Abstract:The public release of Large Language Models (LLMs) has sparked tremendous interest in how humans will use Artificial Intelligence (AI) to accomplish a variety of tasks. In our study conducted with Boston Consulting Group, a global management consulting firm, we examine the performance implications of AI on realistic, complex, and knowledge-intensive tasks. The pre-registered experiment involved 758 consultants comprising about 7% of the individual contributor-level consultants at the company. After establishing a performance baseline on a similar task, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. We suggest that the capabilities of AI create a “jagged technological frontier” where some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI. For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities, consultants using AI were significantly more productive (they completed 12.2% more tasks on average, and completed task 25.1% more quickly), and produced significantly higher quality results (more than 40% higher quality compared to a control group). Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores. For a task selected to be outside the frontier, however, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI. Further, our analysis shows the emergence of two distinctive patterns of successful AI use by humans along a spectrum of human-AI integration. One set of consultants acted as “Centaurs,” like the mythical halfhorse/half-human creature, dividing and delegating their solution-creation activities to the AI or to themselves. Another set of consultants acted more like “Cyborgs,” completely integrating their task flow with the AI and continually interacting with the technology.

The Impact of AI Technology on the Productivity of Gig Economy Workers

Measuring the Impact of AI on Information Worker Productivity

An Analytical study on Navigating Sustainability Challenges and Opportunities in the era of AI and the Gig Economy

Ai, Skill, and Productivity: The Case of Taxi Drivers

Take It, Leave It, or Fix It: Measuring Productivity and Trust in Human-AI Collaboration

Leveraging In-Store Technology and AI: Increasing Customer and Employee Efficiency and Enhancing their Experiences

Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

Enhancing Online-to-offline Delivery Efficiency Facilitated by Industry 4.0: a Personnel Configuration Perspective.

The Impact of Conservative Estimated Time of Arrival Algorithm on Food Delivery Rider Behaviors

Impact of artificial intelligence adoption on online returns policies

Algorithmic management in the gig economy: A systematic review and research integration

AI and Jobs: Has the Inflection Point Arrived? Evidence from an Online Labor Platform

Proliferation of AI Tools: A Multifaceted Evaluation of User Perceptions and Emerging Trend

Challenge or Hindrance? The Dual Impact of Algorithmic Control on Gig Workers' Prosocial Service Behaviors

The Impact of Robot and AI on Human Workforce

Gig to the left, algorithms to the right: A case study of the dark sides in the gig economy

Assessing the Impact of Artificial Intelligence Tools on Employee Productivity: Insights from a Comprehensive Survey Analysis

The Relationship Between AI and Unemployment Rate and the Solution

How much does AI impact development speed? An enterprise-based randomized controlled trial

Strategic Responses to Technological Change: Evidence from ChatGPT and Upwork

Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction