Abstract:Large Language Models (LLMs) have revolutionized various aspects of engineering and science. Their utility is often bottlenecked by the lack of interaction with the external digital environment. To overcome this limitation and achieve integration of LLMs and Artificial Intelligence (AI) into real-world applications, customized AI agents are being constructed. Based on the technological trends and techniques, we extract a high-level approach for constructing these AI agents, focusing on their underlying architecture. This thesis serves as a comprehensive guide that elucidates a multi-faceted approach for empowering LLMs with the capability to leverage Application Programming Interfaces (APIs). We present a 7-step methodology that begins with the selection of suitable LLMs and the task decomposition that is necessary for complex problem-solving. This methodology includes techniques for generating training data for API interactions and heuristics for selecting the appropriate API among a plethora of options. These steps eventually lead to the generation of API calls that are both syntactically and semantically aligned with the LLM's understanding of a given task. Moreover, we review existing frameworks and tools that facilitate these processes and highlight the gaps in current attempts. In this direction, we propose an on-device architecture that aims to exploit the functionality of carry-on devices by using small models from the Hugging Face community. We examine the effectiveness of these approaches on real-world applications of various domains, including the generation of a piano sheet. Through an extensive analysis of the literature and available technologies, this thesis aims to set a compass for researchers and practitioners to harness the full potential of LLMs augmented with external tool capabilities, thus paving the way for more autonomous, robust, and context-aware AI agents.

Using LLM-Generated Draft Replies to Support Human Experts in Responding to Stakeholder Inquiries in Maritime Industry: A Real-World Case Study of Industrial AI

An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project

AI Language Models Could Both Help and Harm Equity in Marine Policymaking: The Case Study of the BBNJ Question-Answering Bot

A Survey on Human-Centric LLMs

The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?

LLMs as Academic Reading Companions: Extending HCI Through Synthetic Personae

Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course

Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics

LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Survey

Leveraging Large Language Models for Tradespace Exploration

Can We Trust AI Agents? An Experimental Study Towards Trustworthy LLM-Based Multi-Agent Systems for AI Ethics

LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System

Agentic LLMs in the Supply Chain: Towards Autonomous Multi-Agent Consensus-Seeking

Practical Applications of Large Language Models for Health Care Professionals and Scientists

The Accuracy of Domain Specific and Descriptive Analysis Generated by Large Language Models

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication

Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models

Human-Centric Autonomous Systems with LLMs for User Command Reasoning

Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs

Synergizing Human Expertise and AI Efficiency with Language Model for Microscopy Operation and Automated Experiment Design

LLMs are Imperfect, Then What? An Empirical Study on LLM Failures in Software Engineering