Abstract:Recent advancements in Large Language Models (LLMs) have demonstrated exceptional capabilities in natural language understanding and generation. While these models excel in general complex reasoning tasks, they still face challenges in mathematical problem-solving and logical reasoning. To address these limitations, researchers have explored function calling abilities, allowing LLMs to execute provided functions and utilize their outputs for task completion. However, concentrating on specific tasks can be very inefficient for large-scale LLMs to be used, because of the expensive cost of training and inference stages they need in terms of computational resources. This study introduces a novel framework for training smaller language models in function calling, focusing on specific logical and mathematical reasoning tasks. The approach aims to improve performances of small-scale models for these tasks using function calling, ensuring a high level of accuracy. Our framework employs an agent that, given a problem and a set of callable functions, queries the LLM by injecting a description and examples of the usable functions into the prompt and managing their calls in a step-by-step reasoning chain. This process is used to create a dataset of correct and incorrect reasoning chain chat completions from a large-scale LLM. This dataset is used to train a smaller LLM using Reinforcement Learning from Human Feedback (RLHF), specifically employing the Direct Preference Optimization (DPO) technique. Experimental results demonstrate how the proposed approach balances the trade-off between model size and performance, improving the ability of function calling for reasoning tasks, in smaller models.

Targeted training for numerical reasoning with large language models

Scaling Relationship on Learning Mathematical Reasoning with Large Language Models

Democratizing Reasoning Ability: Tailored Learning from Large Language Model

Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systems

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought

NUMCoT: Numerals and Units of Measurement in Chain-of-Thought Reasoning using Large Language Models

Efficient Sequential Decision Making with Large Language Models

Rational Metareasoning for Large Language Models

An Effective Framework to Help Large Language Models Handle Numeric-involved Long-context Tasks

Injecting Numerical Reasoning Skills into Language Models

Mixed Distillation Helps Smaller Language Model Better Reasoning

Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation

Improving Small-Scale Large Language Models Function Calling for Reasoning Tasks

Reasoning with Large Language Models, a Survey

Interpreting and Improving Large Language Models in Arithmetic Calculation

Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data

TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning

Effective Distillation of Table-based Reasoning Ability from LLMs