LlamaIndex

LlamaIndex reaches Cloakray through its OpenAI-compatible classes. OpenAILike takes any model ID, so Claude, Gemini and open-weight models all work.

pip install llama-index llama-index-llms-openai-like llama-index-embeddings-openai

Chat model

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="claude-sonnet-4-6",
    api_base="https://api.cloakray.com/v1",
    api_key=os.environ["CLOAKRAY_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=1000000,
)

print(llm.complete("Summarise the CAP theorem in one sentence."))

Set context_window to the model's context length from the catalogue, so LlamaIndex sizes its prompts correctly. is_function_calling_model=True lets agents use tools on models that support them.

Embeddings and an index

from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
from llama_index.embeddings.openai import OpenAIEmbedding

Settings.llm = llm
Settings.embed_model = OpenAIEmbedding(
    model_name="text-embedding-3-small",
    api_base="https://api.cloakray.com/v1",
    api_key=os.environ["CLOAKRAY_API_KEY"],
)

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)
print(index.as_query_engine().query("How do I rotate an API key?"))

Both indexing and querying are billed to the key you use, and show up in your logs.