Define a model provider

A provider is data: an id, the wire api its adapter implements, a base URL, how to find a credential and its models. auth names environment variables (or is a function returning a secret handle) and is resolved per request.

Usage

gptr_provider(
  id,
  api,
  base_url = NULL,
  auth = NULL,
  models = NULL,
  compat = list(),
  type = c("chat", "classifier", "cli"),
  headers = list(),
  discover = NULL,
  status = NULL,
  aliases = character(),
  local = FALSE,
  offline = FALSE,
  rate = NULL
)

Arguments

id

Provider id, ^[a-z0-9][a-z0-9-]*$.

api

Wire api (the name of an adapter), for example "openai-completions".

base_url

Endpoint base URL.

auth

Character vector of environment-variable names, a function, or NULL.

models

List of model records (each with at least id).

compat

Named list of compatibility switches for the adapter.

type

"chat", "classifier" (System 1) or "cli".

headers

Named list of non-secret header strings.

discover

function() listing models on request, or NULL.

status

function() reporting cached provider status, or NULL.

aliases

Alternative names of the provider.

local

Endpoint locality hint permitting unknown model ids. Resolved model locality is checked separately: a loopback server may route inference to a remote model.

offline

TRUE when no remote model is called (test providers).

rate

list(requests_per_s, tokens_per_s) static limits, or NULL.

Value

A spec of class c("gptr_provider", "gptr_spec").

Examples

gptr_provider("corp", api = "openai-completions", base_url = "https://llm.corp.example/v1",
              auth = "CORP_LLM_KEY", models = list(list(id = "corp-large", context = 128000)))
#> <gptr_provider corp>
#>   id: corp
#>   api: openai-completions
#>   base_url: https://llm.corp.example/v1
#>   auth: CORP_LLM_KEY
#>   models: <list of 1>
#>   compat: <list of 0>
#>   type: chat
#>   headers: <list of 0>
#>   aliases: 
#>   local: FALSE
#>   offline: FALSE
#>   api_version: 1.0