Anthropic has unveiled the Message Batches API, allowing developers to send message queries in batches (up to 10,000 queries per batch) to lower the cost of using the Claude model by up to 50% compared to the Standard API usage.
Sending queries to Claude via the Message Batches API does not result in immediate processing (depending on server availability), but guarantees a response within 24 hours (often faster in practice), making it suitable for tasks that do not require real-time answers and can afford to wait, saving costs in the process.
The Message Batches API is compatible with Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Haiku, accessible through Anthropic API, Amazon Bedrock, and soon to be available on Google Cloud Vertex AI.
Source: Anthropic
TLDR: Anthropic introduces Message Batches API for developers to send query messages in batches, reducing costs and enabling delayed responses from the Claude model.
Leave a Comment