Getting Started with Any-LLM¶

Any-LLM is a unified interface that lets you work with language models from any provider using a consistent API. Whether you're using OpenAI, Anthropic, Google, local models, or open-source alternatives, any-llm makes it easy to switch between them without changing your code.

Why Any-LLM?¶

• Provider Agnostic: One API for all LLM providers
• Easy Switching: Change models with a single line
• Cost Comparison: Compare costs across providers
• Streaming Support: Real-time responses from any model
• Type Safe: Full TypeScript/Python type support

Installation¶

In [ ]:

%pip install any-llm-sdk[all] nest-asyncio -q

# nest_asyncio allows us to use 'await' directly in Jupyter notebooks
# This is needed because any-llm uses async functions for API calls
import nest_asyncio

nest_asyncio.apply()

%pip install any-llm-sdk[all] nest-asyncio -q # nest_asyncio allows us to use 'await' directly in Jupyter notebooks # This is needed because any-llm uses async functions for API calls import nest_asyncio nest_asyncio.apply()

Setting Up API Keys¶

Different providers require different API keys. Let's set them up properly:

In [ ]:

import os
from getpass import getpass


def setup_api_key(key_name: str, provider: str) -> None:
    """Set up API key for the specified provider."""
    if key_name not in os.environ:
        print(f"🔑 {key_name} not found in environment")
        api_key = getpass(f"Enter your {provider} API key (or press Enter to skip): ")
        if api_key:
            os.environ[key_name] = api_key
            print(f"✅ {key_name} set for this session")
        else:
            print(f"⏭️  Skipping {provider}")
    else:
        print(f"✅ {key_name} found in environment")


# Set up keys for different providers
print("Setting up API keys...\n")
setup_api_key("OPENAI_API_KEY", "OpenAI")
setup_api_key("ANTHROPIC_API_KEY", "Anthropic")

#  You could add more using :
# setup_api_key("GOOGLE_API_KEY", "Google")
# setup_api_key("MISTRAL_API_KEY", "Mistral")

import os from getpass import getpass def setup_api_key(key_name: str, provider: str) -> None: """Set up API key for the specified provider.""" if key_name not in os.environ: print(f"🔑 {key_name} not found in environment") api_key = getpass(f"Enter your {provider} API key (or press Enter to skip): ") if api_key: os.environ[key_name] = api_key print(f"✅ {key_name} set for this session") else: print(f"⏭️ Skipping {provider}") else: print(f"✅ {key_name} found in environment") # Set up keys for different providers print("Setting up API keys...\n") setup_api_key("OPENAI_API_KEY", "OpenAI") setup_api_key("ANTHROPIC_API_KEY", "Anthropic") # You could add more using : # setup_api_key("GOOGLE_API_KEY", "Google") # setup_api_key("MISTRAL_API_KEY", "Mistral")

List Models Across Providers¶

any_llm can list all available models for an LLM provider - in this case, we are listing out models supported by OpenAI and Anthropic.

In [ ]:

from any_llm import AnyLLM, LLMProvider

for provider in [LLMProvider.OPENAI, LLMProvider.ANTHROPIC]:
    client = AnyLLM.create(provider=provider)
    models = client.list_models()
    print(f"Provider: {provider}")
    print(", ".join([model.id for model in models]))
    print()

from any_llm import AnyLLM, LLMProvider for provider in [LLMProvider.OPENAI, LLMProvider.ANTHROPIC]: client = AnyLLM.create(provider=provider) models = client.list_models() print(f"Provider: {provider}") print(", ".join([model.id for model in models])) print()

Expected output¶

Provider: openai gpt-4o-mini, gpt-4-0613, gpt-4, gpt-3.5-turbo, gpt-5-search-api-2025-10-14, gpt-realtime-mini, gpt-realtime-mini-2025-10-06, sora-2, sora-2-pro, davinci-002, babbage-002, gpt-3.5-turbo-instruct, gpt-3.5-turbo-instruct-0914...

Provider: anthropic claude-haiku-4-5-20251001, claude-sonnet-4-5-20250929, claude-opus-4-1-20250805, claude-opus-4-20250514, claude-sonnet-4-20250514, claude-3-7-sonnet-20250219, claude-3-5-haiku-20241022, claude-3-haiku-20240307

Generate Text¶

Let's use one model from each provider to generate text for the same prompt.

In [ ]:

from any_llm import acompletion
from any_llm.types.completion import ChatCompletion

prompt = "Write a Haiku on the solar system."

# OpenAI
model = "openai:gpt-4o-mini"
result = await acompletion(
    model=model,
    messages=[
        {"role": "user", "content": prompt},
    ],
)
assert isinstance(result, ChatCompletion)

print(f"Model: {result.model}")
print(f"Response:\n{result.choices[0].message.content}\n")

# Anthropic
model = "anthropic:claude-haiku-4-5-20251001"
result = await acompletion(
    model=model,
    messages=[
        {"role": "user", "content": prompt},
    ],
)

assert isinstance(result, ChatCompletion)

print(f"Model: {result.model}")
print(f"Response:\n{result.choices[0].message.content}")

from any_llm import acompletion from any_llm.types.completion import ChatCompletion prompt = "Write a Haiku on the solar system." # OpenAI model = "openai:gpt-4o-mini" result = await acompletion( model=model, messages=[ {"role": "user", "content": prompt}, ], ) assert isinstance(result, ChatCompletion) print(f"Model: {result.model}") print(f"Response:\n{result.choices[0].message.content}\n") # Anthropic model = "anthropic:claude-haiku-4-5-20251001" result = await acompletion( model=model, messages=[ {"role": "user", "content": prompt}, ], ) assert isinstance(result, ChatCompletion) print(f"Model: {result.model}") print(f"Response:\n{result.choices[0].message.content}")

Expected Output¶

Note: The haiku content will be different each time since it's generated by the LLM. This example shows the output format.

Model: gpt-4o-mini-2024-07-18 Response:

Planets spin and dance,
In the vast cosmic embrace,
Stars whisper their tales.

Model: claude-haiku-4-5-20251001 Response:

Eight worlds circle round,
Sun's gravity holds them close—
Dance through endless void.