Getting Started

Introduction

USAi API provides programmatic access to SI services for government users and approved partners. It currently supports Chat Completions (model inference) and Embeddings (used for RAG and other applications).

Our API has resource-oriented URLs, accepts JSON request bodies, returns JSON responses, and uses HTTP response codes to indicate API errors, authentication messaging, and verbs.

Models and endpoints

USAi API is organized around REST and provides access to the large language models (LLMs):

Google SI models

• Gemini 3.5 Flash
• Gemini 3.7 Flash
• Gemini 3.8 Flash

Anthropic models

• Claude Haiku 4.5
• Claude Sonnet 4.5
• Claude Sonnet 4.6
• Claude Sonnet 5
• Claude Opus 4.5
• Claude Opus 4.7
• Claude Opus 4.8
• Claude Opus 5
• Claude Haiku 5.5
• Claude Sonnet 5.5

Meta models

• Llama 4 Maverick

OpenAI models

• GPT 5.2
• GPT 5.4
• GPT 5.5
• GPT 5.6 Sol
• GPT 5.6 Terra
• GPT 5.6 Luna
• GPT 6 Astra
• GPT 6 Sol
• GPT 6 Luna
• GPT 6.1 Sol

xAI models

• Grok 4
• Grok 4.3
• Grok 4.6

The interface is modeled after the OpenAI Chat Completion API. We will continue to add more models over time; we may also remove models if they are found to not meet our standards. You will be notified if a model is removed.

Model availability varies by tenant. Before sending a request, use the GET /api/v1/models response for your tenant as the authoritative source of supported model IDs.

Each model we draw on has unique capabilities. Models may also respond differently to the same API request. Below is a summary of each endpoint’s functionality:

• Chat Completions: The ability to send prompts and receive a response from the LLM models.
• Embedding: Converts input into numeric vectors representing semantic meaning. The vectors are used for tasks like retrieving information from documents, document analysis, and building Retrieval Augmented Generation (RAG) systems.
• Models: A list of models and IDs used to specify LLM models in requests.

Content types

The models USAi API draws on support multiple types of inputs; text, image, and file content types. However, each model has different input capabilities: for example, Claude Sonnet supports Optical Character Recognition (OCR) and recognizes image-only PDFs, while Llama currently does not. For additional detail about how we handle these content types, reference OpenAI’s API documentation.

Recent models from OpenAI and Anthropic will accept Word (.docx) and Excel (.xlsx) input as well. Note, they do not accept the older .doc and .xls formats. To work with these files provide the following mime types when creating the file_data field in the API call:

.docx:

application/vnd.openxmlformats-officedocument.wordprocessingml.document

.xlsx:

application/vnd.openxmlformats-officedocument.spreadsheetml.sheet

Authentication

All API requests require an API key. Upon authentication, your agency-specific instructions for requesting an API key, and the API endpoint, will be available.

Limitations

Rate limitations

We currently have the following rate limits:

• Models – 3 calls / second / API key
• Chat completions – 3 calls / second / API key
• Embeddings – 100 calls / second / API key

If you hit a rate limit, you should expect a 429 error code.

Cost limitations

If your agency has set a weekly cost limit per person for API usage, and you hit those cost limits, you should expect to hit a 429 error code: “budget limit reached for this billing cycle.”

Weekly budgets run Monday – Sunday. If you have more than one API key, all API calls with any of your keys will count towards your weekly budget.

Feature limitations

Because the underlying platforms have different capabilities and interfaces, not all features of the Chat Completions API are available at this time. Current known limitations include:

• Audio
• Structured output

Guardrail limitations

Unlike USAi Chat, which has built-in safety guardrails and system prompts, building your own application with the USAi API means your team is fully responsible for security and safety.

For OpenAI and xAI models we have Azure’s DefaultV2 content filtering enabled, which provides automated harm detection across four categories (Hate, Self-Harm, Sexual, Violence) plus advanced protections against prompt injection, and copyright violations. When the platform detects harmful content it will raise an HTTP 400 error in non-streaming contexts and return an error chunk when streaming.

For Google, Anthropic, and Meta models we do not have guardrails beyond those provided by our model providers; this means that an API user has full control over what inputs and outputs are allowed when using the API. We expect API users to interact with the API ethically and deliberately, and add their own guardrails and system prompts as needed. We recommend implementing system prompts when using the API in situations where you do not know what inputs the model will receive.

System prompts should address your needs and concerns. Some of the system prompts in USAi Chat include:

• You are a helpful assistant that works for a government agency.
• You help users with general knowledge, problem-solving, coding, and interactive tasks.
• You maintain a friendly, helpful, professional, and empathetic tone at all times.
• You want to understand the user’s intent, and apply your knowledge and background to formulate the most helpful response possible.
• Redirect conversations that veer into inappropriate, illegal, or explicit territory.
• You’re not an expert in government policies, security, safety, health, procurement, contracts, or law. Provide general guidance only and advise users to reference appropriate material.
• Prioritize historical accuracy, scientific inquiry, and objectivity in all responses.
• Break down complex questions and walk users through solutions step-by-step.
• Use real-world analogies to simplify complex concepts.
• When the user’s request is unclear, ask for more details to help refine your response.
• Ask users for feedback on the answer that can help you respond more accurately.
• Never knowingly make false statements or deceive users.
• Avoid generating explicit, hateful, dangerous, or illegal content.
• Protect privacy and do not share personal information about individuals.
• Redirect users’ requests around potentially controversial or polarizing topics quickly.
• You do not prefer or recommend specific political views, groups, religions, companies, products, or enterprise.

Caching Limitations

All USAi models have caching in place.

• Minimum size: Requests must meet the selected model’s minimum token threshold to qualify for caching.
• Cache duration: Cached content lasts 5 minutes. After it expires, the cache must be rebuilt.
• Consistent requests: Changes to tools or request settings (such as web search) can invalidate the cache.
• Disabling cache: Vertex AI and Azure models cannot have cache disabled. However, Bedrock models can have cache disabled if you choose.
• Customizing cache points: Cache points determine which parts of a conversation are cached for reuse, which can reduce processing costs. Only Bedrock models support customizing cache points.

For best results, reuse the same request structure and send follow-up requests within the cache window.

Code demo

To see an example of how to implement features using Python, visit our HTML example notebook. If you would like to run this file in a Jupyter environment, you can download the notebook ipynb file.

Endpoints

API endpoints will be available when you login.

1. Models

GET /api/v1/models

To reach models the documented endpoint is <base url>/api/v1/models, which will allow you to retrieve available model information. The response is the authoritative source of model IDs available to your tenant; the example below is illustrative.

Example Request

curl -X 'GET' \
'<base url>/api/v1/models' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <Your API Key>'

Example Response

{
  "object": "list",
  "data": [
    {
      "id": "claude_4_5_sonnet",
      "created": 1718841600,
      "object": "model",
      "owned_by": "Anthropic"
    },
    {
      "id": "gemini-3.5-flash",
      "created": 1727222400,
      "object": "model",
      "owned_by": "Google"
    },
    {
      "id": "gpt-5.5",
      "created": 1698883200,
      "object": "model",
      "owned_by": "OpenAI"
    },
    {
      "id": "llama_4_maverick",
      "created": 1727222400,
      "object": "model",
      "owned_by": "Meta"
    },
    {
      "id": "cohere_english_v3",
      "created": 1698883200,
      "object": "model",
      "owned_by": "Cohere"
    },
    {
      "id": "text-embedding-005",
      "created": 1698883200,
      "object": "model",
      "owned_by": "Amazon"
    }
  ]
}

2. Chat Completions

POST /api/v1/chat/completions

To reach chat completions the documented endpoint is <base url>/api/v1/chat/completions, which will allow you to retrieve chat completion information.

Request Body

• model: The model ID (e.g: gemini-3.5-flash, claude_4_5_sonnet, gpt-5.5)
• messages: An array of of message items consisting of User message, Image Content, Document Content, Assistant Message
• max_tokens: Maximum response length (optional)
• temperature: Response creativity (0.0-1.0, optional)
• top_p: An alternative to sampling with temperature, called nucleus sampling (optional)
• stream: a boolean indicating whether to send partials responses as available (optional)

Example Request

curl -X 'POST' \
'<base url>/api/v1/chat/completions' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <Your API Key>' \
-H 'Content-Type: application/json' \
-d '{
  "messages": [
    {
      "content": "You speak only pirate",
      "role": "system"
    },
    {
      "content": "Hello!",
      "role": "user"
    }
   ],
  "model": "gemini-3.5-flash"
}'

Example Response

{
   "object": "chat.completion",
   "created": 1748517745,
   "model": "gemini-3.5-flash",
   "choices": [
      {
        "index": 0,
        "message": {
           "role": "assistant",
           "content": "Ahoy there, matey! What brings ye to me waters?\n"
        },
       "finish_reason": "stop"
    }
  ],
  "usage": {
       "prompt_tokens": 14,
       "completion_tokens": 15,
      "total_tokens": 29
  }
}

3. Embeddings

POST /api/v1/embeddings

To reach embeddings the documented endpoint is <base url>/api/v1/embeddings, which will allow you to retrieve embeddings information.

Request Body

• model: The model ID (e.g: cohere_english_v3)
• input: Input text to embed, encoded as a string or array of strings. Each input must not exceed the max input tokens for the model. dimensions: The number of dimensions the resulting output embeddings should have. Only supported in some models (Optional)
• input_type: (Note: this is not part of the OpenAI specification but is useful on some models). Specify the kind of input to allow the model to optimize for specific uses. Options are: “search_document”, “search_query”, “classification”, “clustering”, “semantic_similarity” (Optional)

Example Request

curl -X 'POST' \
'<base url>/api/v1/embeddings' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <Your API Key>' \
-H 'Content-Type: application/json' \
-d '{
    "encodingFormat": "float",
    "input": "A mighty woman with a torch, whose flame / Is the imprisoned lightning",
    "input_type": "search_document",
    "model": "cohere_english_v3"
}

Example Response

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [
        0.06933594,
        -0.030883789,
        0.054351807,
        -0.0018196106,
       …
        0.013938904
      ],
      "index": 0
    }
  ],
  "model": "cohere.embed-english-v3",
  "usage": {
    "promptTokens": 12,
    "totalTokens": 12
  }
}

Best practices

1. Protect API credentials and control access. Keep shared USAi API credentials in secure backend services. Do not embed them in browser code, distributed applications, or source repositories. Authenticate callers and enforce application permissions before making API calls.

2. Treat external content as untrusted. User input, uploaded files, retrieved documents, web pages, and tool results may contain instructions intended to manipulate the model. Validate input formats, enforce appropriate size limits, and process files securely. Separate application instructions from external content, but do not assume text sanitization or prompt formatting will prevent prompt injection.

3. Protect sensitive data and privacy. Only send the exact data fields needed for your tool to work. Avoid sending unnecessary personal data (PII) or sensitive operational details, and make sure raw prompts aren’t sitting in wide-open debug logs.

4. Prevent autonomous SI actions. Do not rely solely on system prompts to stop restricted behaviors; put strict code-level enforcement mechanisms in place. Ensure all production deployments include feature flags or a rapid disable to immediately halt operations if needed.

5. Appropriate human review. Document which outputs or actions require human review, based on the consequences of error and applicable organizational requirements. Require meaningful review before those outputs drive consequential decisions or actions. For lower-risk workflows, use appropriate automated validation, monitoring, sampling, and escalation. Reviewers must have enough context and authority to correct or reject the result.

6. Monitor usage, costs, and failures. Keep an eye on how many tokens you use, how much money it costs, and how fast it responds. Set up budgets, logs, alerts, and test handling or timeouts, service interruptions, and incomplete output. Provide a tested way to disable affected SI features or automated actions quickly.

7. Re-check when things change. Maintain representative tests for task quality, security, and failure handling. Whenever you update your prompts, switch to a new model, or add new source documents, re-run tests and evaluations to make sure safety controls still hold up. 

8. Secure official authorizations for production deployments. Obtaining an API key does not constitute formal deployment authorization. Verify all compliance materials and required ATO clearances are fully secured before launching to end users for any production or operational use. 

Support

If you have any questions, hit rate limits, or otherwise need assistance, please contact us at support@usai.gov.