Models and Voices

Reference for the developer-only model and voice catalog pages, plus the wider authenticated catalog endpoints consumed by staff tooling.

• Routes:
• `/models`
• `/voices`
• Sidebar menu metadata marks both as developer-only (`isDeveloper: true`).
• The pages render `AccessDenied` for non-developer users.
• Frontend `isDeveloper` is populated by mapping backend `is_admin` to frontend `is_developer` in `AuthContext`.
• Backend CRUD endpoints are under `/api/admin/*` and protected by `adminMiddleware`.
• Separate read-only catalog endpoints exist for any authenticated user:
• `GET /api/init/models`
• `GET /api/init/ai-voices`
• Those init endpoints are what AI teammate profile forms and other non-admin flows use.

• UI supports list/search, detail view, and create/edit/delete.
• The detail pane now also shows provider-enriched metadata when available, including context window, pricing, modalities, and supported generation parameters.
• Current dialog fields:
• display name
• model identifier
• provider
• `is_multimodal`
• `is_realtime`
• `is_tts`
• `is_stt`
• `Input Cost`
• `Output Cost`
• `Cached Input Cost`
• Provider options shown in the dialog:
• `OpenAI`
• `OpenRouter`
• `Google`
• `DeepSeek`
• `Anthropic`
• `Codex`
• `Custom`
• Pricing fields are optional numeric USD values stored per 1M tokens.
• If `Cached Input Cost` is left empty, runtime cost calculation falls back to the normal input cost.
• Backend model records also support a `voices` array, but the current models dialog does not expose that field.
• Current uniqueness constraint is `(provider, model)`.
• Current provider-specific runtime notes:
• `Codex` models use an organization-level Codex connection rather than a typed provider API key field.
• Codex supports both streamed agent execution and callers that expect a completed JSON generation result, including structured workflow LLM calls. For completed-result calls, the adapter streams from the upstream Responses endpoint, collects completed output items, and returns the JSON response shape expected by the AI SDK.
• Codex Responses requests disable upstream response storage.
• `Custom` models are treated as OpenAI-compatible models at the runtime layer.
• An active organization Custom connection supplies both the API key and Base URL. If it is unavailable, runtime falls back to the system Custom provider configuration.
• Custom model identifiers are normalized with a `custom/` prefix in the catalog and that prefix is removed before the model ID is sent to the OpenAI-compatible endpoint.

Runtime STT behavior

• Inbound Telegram, Slack, and MS Teams audio or voice attachments are transcribed with the AI teammate `stt_model` before the message is sent to the agent.
• Runtime STT accepts only models marked `is_stt`.
• Current runtime STT providers are:
• `OpenAI`
• `OpenRouter`
• `Custom`
• `Google`
• `OpenAI`, `OpenRouter`, and `Custom` use OpenAI-compatible audio transcription endpoints.
• Custom STT sends requests through the same organization or system Base URL resolved for the Custom provider.
• `Google` uses a multimodal language model with the audio file as input.
• If no STT model is configured, the model is not found, the model is not marked `is_stt`, the provider is unsupported, the API key is unavailable, or transcription fails, the agent receives a system note that the voice message could not be transcribed.

Admin API

• `GET /api/admin/ai-models`
• `GET /api/admin/ai-models/metadata`
• `GET /api/admin/ai-models/{id}`
• `POST /api/admin/ai-models`
• `PUT /api/admin/ai-models/{id}`
• `DELETE /api/admin/ai-models/{id}`
• Metadata lookup requires:
• `provider`
• `model`
• Current backend metadata lookup is implemented for `OpenRouter` models and returns resolved model id, name, pricing, and capability flags when found.

Authenticated catalog API

• `GET /api/init/models`
• Returned fields include:
• `id`
• `model`
• `name`
• `provider`
• `is_connected` for provider availability in the current organization
• `is_multimodal`
• `is_realtime`
• `is_tts`
• `is_stt`
• `voices`
• Frontend consumers use this endpoint to populate chat/TTS/STT selectors outside the admin pages.
• For Codex models, `is_connected` is `false` when the current organization has no active Codex provider connection; other providers currently return `true`.

• UI supports list/search, detail view, and create/edit/delete.
• Current dialog fields:
• provider
• voice identifier
• gender
• Provider options shown in the dialog:
• `OpenAI`
• `OpenRouter`
• `Google`
• Backend/provider model supports more than the current dialog exposes:
• backend validation/model enum accepts `OpenAI`, `OpenRouter`, `Google`, `DeepSeek`, and `Anthropic`
• the current create/edit dialog only offers `OpenAI`, `OpenRouter`, and `Google`
• the list page can still display voices from any stored provider
• Voice uniqueness is `(provider, voice)`.

Admin API

• `GET /api/admin/ai-voices`
• `GET /api/admin/ai-voices/{id}`
• `POST /api/admin/ai-voices`
• `PUT /api/admin/ai-voices/{id}`
• `DELETE /api/admin/ai-voices/{id}`

Authenticated catalog API

• `GET /api/init/ai-voices`
• Optional query filters:
• `provider`
• `gender`
• `search`
• AI teammate voice selectors filter this list by the currently selected TTS model provider.

• Chat `model` must not be a realtime, TTS, or STT model.
• `tts_model` must be TTS-capable or realtime-capable.
• `stt_model` must be STT-capable.
• `voice` cannot be set without a `tts_model`.
• When `tts_model` is set, `voice` is required and must exist for that TTS model's provider.
• Current frontend filtering matches the backend validation:
• chat model dropdown excludes `is_realtime`, `is_tts`, and `is_stt`
• TTS dropdown includes `is_tts` or `is_realtime`
• STT dropdown includes `is_stt`

• Developer-only system settings in `/defaults` can define:
• `AI Model Fallback 1`
• `AI Model Fallback 2`
• Runtime execution retries those fallback models in order before the run is marked failed.
• Primary-model and fallback-model retry diagnostics are written through the shared Mastra logger.
• Provider failure diagnostics are summarized for logs. They keep high-level fields such as provider error name, message, status, retryability, URL, and provider code/message, while raw request and response payload fields are redacted.