Managed Inference and Agents API with Claude 3.5 Haiku

Table of Contents [expand]

Last updated February 12, 2026

Claude 3.5 Haiku is a text-to-text large language model (LLM) in Anthropic’s Claude 3 family. This model is optimized for cost-efficiency and solid performance at a lower price point than Claude 3.5 Sonnet. It supports conversational chat and tool-calling capabilities, and is slightly more advanced than the claude-3-haiku model.

  • Model ID: claude-3-5-haiku
  • Region: us

When to Use This Model

Claude 3.5 Haiku is ideal for straightforward chat interactions, lightweight code generation, and simpler workflows.

Usage

Claude 3.5 Haiku follows our Claude v1/chat/completions API schema.

To provision access to the model, attach a Managed Inference and Agents add-on add-on to your app $APP_NAME:

heroku addons:create heroku-inference:standard -a $APP_NAME

Using config variables, you can invoke the model in various ways:

Rate Limits

  • Maximum requests per minute: 200
  • Maximum tokens per minute: 800,000

Prompt Caching

Prompt caching is supported for system prompts and tools. The minimum tokens required for prompt caching is 2,048.

Example curl Request

Get started quickly with an example request:

export INFERENCE_KEY=$(heroku config:get -a $APP_NAME INFERENCE_KEY)
export INFERENCE_URL=$(heroku config:get -a $APP_NAME INFERENCE_URL)

curl $INFERENCE_URL/v1/chat/completions \
-H "Authorization: Bearer $INFERENCE_KEY" \
-d @- <<EOF
{
  "model": "claude-3-5-haiku",
  "messages": [
    { "role": "user", "content": "Hello!" },
    { "role": "assistant", "content": "Hi there! How can I assist you today?" },
    { "role": "user", "content": "What's the weather like in Portland, Oregon right now?" }
  ],
  "temperature": 0.5,
  "max_tokens": 100,
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Fetches the current weather for a given city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The name of the city to get weather for."
            }
          },
          "required": ["city"]
        }
      }
    }
  ],
  "tool_choice": "auto",
  "top_p": 0.9
}
EOF