Function Calling in Spring AI: Connecting LLMs to Your Java Services

How Function Calling Works

The flow has four steps:

1. Declare — You describe the tool's name, purpose, and parameters in a schema
2. Request — Claude/GPT-4 sees the schema and decides which tool(s) to call and with what arguments
3. Execute — Your Java method runs with the model-supplied arguments
4. Return — The result is sent back to the model, which incorporates it into its final response

Spring AI handles steps 1–2 and 4 automatically when you use @Tool. You only write the Java logic in step 3.

Your First Tool: The @Tool Annotation

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.stereotype.Component;

@Component
public class WeatherTools {

    private final WeatherApiClient weatherClient;

    @Tool(description = "Get the current weather for a city. Returns temperature in Celsius, "
        + "weather condition (sunny, cloudy, rain), and wind speed in km/h.")
    public WeatherResult getCurrentWeather(String city) {
        return weatherClient.getWeather(city);
    }

    @Tool(description = "Get a 5-day weather forecast for a city.")
    public List<ForecastDay> getForecast(String city) {
        return weatherClient.getForecast(city, 5);
    }

    record WeatherResult(String city, double temperature,
                          String condition, double windSpeed) {}
    record ForecastDay(String date, double high, double low, String condition) {}
}

Wiring Tools into ChatClient

@Service
public class AiAssistantService {

    private final ChatClient chatClient;

    public AiAssistantService(ChatClient.Builder builder, WeatherTools weatherTools) {
        this.chatClient = builder
            .defaultSystem("You are a helpful travel assistant. Use available tools "
                + "to provide accurate, real-time information.")
            .defaultTools(weatherTools)    // all @Tool methods on this bean are available
            .build();
    }

    public String answer(String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
        // If the model decides to call getCurrentWeather("Paris"),
        // Spring AI invokes the method automatically and sends the result back.
    }
}

Parameter Descriptions with @ToolParam

For tools with multiple parameters, use @ToolParam to give each parameter context the model needs to fill it correctly:

import org.springframework.ai.tool.annotation.ToolParam;

@Tool(description = "Search our product catalogue. Returns matching products with prices.")
public List<Product> searchProducts(
        @ToolParam(description = "Search query, e.g. 'wireless headphones under 5000'")
        String query,

        @ToolParam(description = "Category filter: electronics, clothing, books, or null for all")
        String category,

        @ToolParam(description = "Maximum price in INR. Use null for no limit.")
        Integer maxPriceInr,

        @ToolParam(description = "Number of results to return. Default is 5, max is 20.")
        int limit) {

    return productRepository.search(query, category, maxPriceInr, limit);
}

Database Queries as Tools

Giving the LLM access to database queries is one of the most powerful patterns — natural language to SQL, effectively:

@Component
public class OrderTools {

    private final OrderRepository orderRepo;
    private final CustomerRepository customerRepo;

    @Tool(description = "Look up order details by order ID. Returns order status, items, and total.")
    public OrderDetails getOrder(String orderId) {
        return orderRepo.findById(orderId)
            .map(OrderDetails::from)
            .orElseThrow(() -> new ToolExecutionException("Order not found: " + orderId));
    }

    @Tool(description = "Get all orders for a customer in a date range.")
    public List<OrderSummary> getCustomerOrders(
            @ToolParam(description = "Customer email address") String email,
            @ToolParam(description = "Start date in ISO format (YYYY-MM-DD)") String fromDate,
            @ToolParam(description = "End date in ISO format (YYYY-MM-DD)") String toDate) {

        Customer customer = customerRepo.findByEmail(email)
            .orElseThrow(() -> new ToolExecutionException("Customer not found: " + email));

        LocalDate from = LocalDate.parse(fromDate);
        LocalDate to = LocalDate.parse(toDate);

        return orderRepo.findByCustomerAndDateBetween(customer, from, to)
            .stream()
            .map(OrderSummary::from)
            .collect(Collectors.toList());
    }
}

Error Handling in Tools

When a tool throws, the error is sent back to the model so it can respond gracefully. Throw meaningful exceptions:

@Tool(description = "Cancel an order. Only possible if order is in PENDING or PROCESSING status.")
public CancellationResult cancelOrder(String orderId) {
    Order order = orderRepo.findById(orderId)
        .orElseThrow(() -> new ToolExecutionException(
            "Order " + orderId + " not found."));

    if (order.getStatus() == OrderStatus.DELIVERED) {
        throw new ToolExecutionException(
            "Cannot cancel order " + orderId + ": already delivered. "
          + "Ask customer to initiate a return instead.");
    }

    if (order.getStatus() == OrderStatus.SHIPPED) {
        throw new ToolExecutionException(
            "Cannot cancel order " + orderId + ": already shipped. "
          + "Customer must refuse delivery or return once received.");
    }

    order.cancel();
    orderRepo.save(order);
    return new CancellationResult("success", order.getRefundAmount());
}

record CancellationResult(String status, BigDecimal refundAmount) {}

Per-Request Tool Injection

Not every user should access every tool. Inject different tool sets per request based on the authenticated user's role:

@PostMapping("/chat")
public String chat(@RequestBody ChatRequest req,
                   Principal principal) {

    User user = userService.findByUsername(principal.getName());

    // Build tool list based on user's role
    List<Object> tools = new ArrayList<>();
    tools.add(productTools);      // all users can search products
    tools.add(orderReadTools);    // all users can check order status

    if (user.hasRole("AGENT")) {
        tools.add(orderWriteTools);   // only support agents can cancel orders
        tools.add(customerTools);     // only agents can look up customer data
    }
    if (user.hasRole("ADMIN")) {
        tools.add(analyticsTools);    // only admins see business metrics
    }

    return chatClient.prompt()
        .user(req.message())
        .tools(tools.toArray())     // override defaultTools for this request
        .call()
        .content();
}

FunctionCallback: Programmatic Tool Definition

When you can't annotate a method (external library, dynamic tools, complex JSON schema), use FunctionCallback directly:

import org.springframework.ai.model.function.FunctionCallback;

@Bean
public FunctionCallback calculatorTool() {
    return FunctionCallback.builder()
        .name("calculate")
        .description("Evaluate a mathematical expression. "
            + "Supports +, -, *, /, ^, sqrt(), sin(), cos(). "
            + "Returns the numeric result.")
        .inputType(CalculatorInput.class)
        .function(input -> {
            double result = mathEngine.evaluate(input.expression());
            return new CalculatorOutput(result);
        })
        .build();
}

record CalculatorInput(
    @JsonProperty(required = true) String expression) {}

record CalculatorOutput(double result) {}

Real-World Pattern: AI Customer Support Agent

Here's a complete customer support agent with multiple tools:

@Component
public class CustomerSupportAgent {

    private final ChatClient chatClient;

    public CustomerSupportAgent(
            ChatClient.Builder builder,
            OrderTools orderTools,
            ShippingTools shippingTools,
            RefundTools refundTools,
            KnowledgeBaseTools kbTools) {

        this.chatClient = builder
            .defaultSystem("""
                You are a customer support agent for Acme Shop.
                - Always look up the order before giving status information
                - For refunds, explain the policy first, then process if eligible
                - If you can't resolve an issue, escalate: say "Let me connect you with a specialist"
                - Be concise and helpful; don't make the customer repeat information
                """)
            .defaultTools(orderTools, shippingTools, refundTools, kbTools)
            .defaultAdvisors(new MessageChatMemoryAdvisor(new InMemoryChatMemory()))
            .build();
    }

    public String handle(String sessionId, String customerMessage) {
        return chatClient.prompt()
            .user(customerMessage)
            .advisors(a -> a.param(
                MessageChatMemoryAdvisor.CHAT_MEMORY_CONVERSATION_ID_KEY, sessionId))
            .call()
            .content();
    }
}