Guides

Model Pathways

The two-pathway architecture of Provider + AIService.

Downcity's AI service has two independent pathways sharing a single model registry:

                    Provider.model({ id: "deepseek-v4-flash", ... })
                                    │
                              ai.use(model)
                                    │
              ┌─────────────────────┴─────────────────────┐
              │                                           │
         SDK Pathway                            OpenAI-Compatible Pathway
  For User City                              For downcity agent / curl / OpenAI SDK
              │                                           │
   POST /v1/ai/text               POST /v1/ai/chat/completions
   POST /v1/ai/stream                  ↑
              │                         OpenAI-format body
              ▼                        { model, messages, stream }
   provider.text                                     │
   provider.stream                                   ▼
      (ai-sdk wrapper)                       provider.openai
              │                         (passthrough or format conversion)
              ▼                                    │
         UIMessage                                ▼
      UIMessageStream                          Response
                                           (upstream raw response)

Provider Pattern

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { Provider, type OpenAICompatibleClientConfig, type Context, type AIProviderChargedResponse } from "@downcity/city";

// OpenAI-compatible: override createClient to get default text / stream, no openai → auto-passthrough
class DeepSeekProvider extends Provider {
  constructor() {
    super({
      id: "deepseek",
      env: { DEEPSEEK_API_KEY: "DeepSeek API Key" },
      baseURL: "https://api.deepseek.com",
      envKey: "DEEPSEEK_API_KEY",
    });
  }

  protected createClient({ apiKey, baseURL }: OpenAICompatibleClientConfig) {
    return createOpenAICompatible({ apiKey, baseURL, name: "deepseek" });
  }
}

const deepseek = new DeepSeekProvider();

// Non-OpenAI format: provide a custom openai method
class OpenAICustomProvider extends Provider {
  constructor() {
    super({
      id: "openai-custom",
      env: { OPENAI_API_KEY: "OpenAI API Key" },
      baseURL: "https://api.openai.com/v1",
      envKey: "OPENAI_API_KEY",
    });
  }

  protected createClient({ apiKey, baseURL }: OpenAICompatibleClientConfig) {
    return createOpenAICompatible({ apiKey, baseURL, name: "openai-custom" });
  }

  async openai(ctx: Context): Promise<AIProviderChargedResponse> {
    return openaiCompatibleHandler(ctx);
  }
}

const openaiCustom = new OpenAICustomProvider();

Auto Passthrough

When no openai handler but baseURL + envKey exist, AIService auto-generates a passthrough:

POST /chat/completions → fetch(baseURL + body) → raw upstream Response

Zero adapter code. Fully OpenAI-compatible.

Format Conversion

For OpenAI-compatible providers with custom upstream requirements, the openai handler can do bidirectional conversion:

  • Downstream: OpenAI body → provider-specific request body
  • Upstream (non-streaming): Provider JSON → OpenAI JSON
  • Upstream (streaming): Provider SSE events → OpenAI SSE chunks (event by event)

Provider Billing

Provider actions may return a charge line next to the normal output, but the preferred place for shared pricing logic is bill(ctx, output) on the provider or model. This keeps provider-specific usage parsing inside the provider instead of exposing token/cache fields as a global protocol.

import { Provider, buildAssistantMessage, type Context, type AIProviderChargedOutput } from "@downcity/city";
import type { UIMessage } from "ai";

const ai = new AIService({
  balance,
});

class PricedProvider extends Provider {
  constructor() {
    super({ id: "priced-provider" });
  }

  protected bill(ctx: Context, output: unknown) {
    return {
      amount_microcredits: priceUpstreamUsage(output?.metadata?.usage),
      note: "priced-provider text",
      metadata: {
        provider_id: "priced-provider",
        raw_usage: output?.metadata?.usage,
      },
    };
  }

  async text(ctx: Context): Promise<AIProviderChargedOutput<UIMessage>> {
    const result = await callUpstream(ctx.input);

    return buildAssistantMessage(result.text, ctx, {
      finishReason: "stop",
      usage: result.usage,
    });
  }
}

const provider = new PricedProvider();

For streams, an action may still return charge as a promise when the final amount is only available after the upstream stream finishes:

return {
  response: stream.toUIMessageStreamResponse(),
  charge: stream.totalUsage.then((usage) => ({
    amount_microcredits: priceUpstreamUsage(usage),
    note: "priced-provider stream",
  })),
};

Routes

RoutePathway
POST /v1/ai/textSDK
POST /v1/ai/streamSDK
POST /v1/ai/image/createSDK
POST /v1/ai/image/resultSDK
POST /v1/ai/videoSDK
POST /v1/ai/ttsSDK
POST /v1/ai/asrSDK
POST /v1/ai/chat/completionsOpenAI-Compatible
GET /v1/ai/modelsModel Catalog

Provider Environment Variables

How to configure provider API keys in the Federation environment.

Hooks and Billing Logic

Connect quotas, usage, logs, and billing into the Action lifecycle.

Table of Contents

Provider PatternAuto PassthroughFormat ConversionProvider BillingRoutes