Skip to content

Feat: Price IBM Bob in Bobcoins in the built-in local config #1201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
huang195 merged 2 commits into from
Sep 30, 2026
Merged

Feat: Price IBM Bob in Bobcoins in the built-in local config #1201

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b378183
Feat: Price IBM Bob in Bobcoins in the built-in local config
huang195 Sep 30, 2026
00a57e3
Docs: Say the built-in config prices Bob, and spell its unit Bobcoins
huang195 Sep 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions cmd/authbridge-proxy/local.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,17 +233,32 @@ tls_bridge:
# Gateways matching a shipped rule already have their discount applied and need
# nothing here. If yours is not one of them and it bills a fraction of list, say
# so once: one scalar tracks upstream repricing, where a copied rate card freezes
# today's numbers and goes stale with nothing to say it has.
# today's numbers and goes stale with nothing to say it has. Add it under
# endpoints below:
#
# pricing:
# endpoints:
# - hosts: ["my-gateway.example.com"]
# multiplier: 0.80 # a FRACTION of list, so 0.80 is a 20% discount

@esnible esnible Sep 30, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion — this example is now stranded above the block it belongs to.

The comment at 236-237 says "Add it under endpoints below:", but the snippet sits above the new live pricing: key (251), separated by the Bob paragraph, and it lost the # pricing: / # endpoints: parents it had on main. On main these four lines were self-contained — uncommenting them produced a valid standalone block. Now, uncommenting just these two lines puts a bare sequence item at document root and the config fails to load:

expected '<document start>', but found '<block mapping start>'
  line 27, column 1:  pricing:

(verified by extracting the template and parsing the result.) Note where that error points: at pricing:, twelve lines below the lines the user actually edited. The diagnostic actively misdirects.

The indent is off by one too — 5 spaces before the dash here vs 4 at line 253 — because the snippet was authored to be read with the # gutter stripped, not uncommented in place.

Suggest moving these two lines to just after 260, inside the live endpoints: list, at 4-space/6-space indent, so uncommenting yields a valid second list item. The prose can stay where it is. A multiplier-only endpoint is legal once correctly placed, so nothing else needs to change.

#
# For a gateway whose prices are genuinely negotiated per model rather than
# derived from list, give rates instead of a multiplier -- see
# docs/pricing.md.
#
# IBM Bob bills in Bobcoins, not dollars, at a flat rate no vendor list carries:
# its models (premium-ide, router, openai/gpt-oss-20b) are in no bundled table,
# so without this entry every Bob request is unpriced. "unit" keeps its figures
# out of the dollar totals -- abctl shows them in Bobcoins and never adds the
# two. Edit the rate here if IBM changes it; nothing else will.
pricing:
endpoints:
- hosts: ["api.us-east.bob.ibm.com"]
unit: Bobcoins
models:
"*": # every Bob model, every tier: 2 per million tokens
input_cost_per_million: 2.00
output_cost_per_million: 2.00
cache_read_cost_per_million: 2.00 # no cache discount on this gateway
cache_write_cost_per_million: 2.00
#
# Cost history on disk. Per-minute totals only -- no prompts, no completions,
# no tool arguments -- under ~/.cortex/cost/YYYY-MM-DD.jsonl, kept 31 days
# (roughly 10 MB). The in-memory counters are a 6-hour ring and this proxy
Expand Down
51 changes: 51 additions & 0 deletions cmd/authbridge-proxy/local_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"encoding/json"
"encoding/pem"
"fmt"
"math"
"math/big"
"os"
"path/filepath"
Expand All @@ -21,6 +22,7 @@ import (
"time"

"github.com/rossoctl/cortex/core/config"
"github.com/rossoctl/cortex/core/cost/pricing"
"github.com/rossoctl/cortex/core/cost/usage"
"github.com/rossoctl/cortex/core/session"
)
Expand Down Expand Up @@ -635,3 +637,52 @@ func TestBuiltinConfig_SeparatesCodingAgentSessions(t *testing.T) {
t.Errorf("built-in config: id_headers = %v, want it to include %s", cfg.Session.SessionIDHeaders(), session.BobSessionHeader)
}
}

// TestBuiltinConfig_PricesBobInBobcoins: the built-in config prices IBM Bob's gateway at its
// flat 2 Bobcoins per million tokens on every tier. Without the entry, every Bob request is
// unpriced (its models — premium-ide, router, openai/gpt-oss-20b — are in no bundled table), and
// a new install shows "—" for Bob until its user finds docs/pricing.md. The rate is resolved
// through pricing.Build rather than read off the struct, so a unit or model key the table would
// not match fails here and not only on a laptop.
func TestBuiltinConfig_PricesBobInBobcoins(t *testing.T) {
cortexDir := t.TempDir()
p, err := writeBuiltinConfig(cortexDir, filepath.Join(cortexDir, "ca"))
if err != nil {
t.Fatal(err)
}
cfg, err := config.Load(p)
if err != nil {
t.Fatal(err)
}
tab, err := pricing.Build(cfg.Pricing)
if err != nil {
t.Fatalf("pricing.Build: %v", err)
}

const bob = "api.us-east.bob.ibm.com"
for _, model := range []string{"premium-ide", "router", "openai/gpt-oss-20b"} {
rates, prov := tab.Resolve(bob, model, 0)
if prov != pricing.ProvConfigured {
t.Errorf("%s on %s: provenance %v, want configured", model, bob, prov)
}
// Rates are stored per token; the config states them per million.
for tier := pricing.TierInput; tier <= pricing.TierOutput; tier++ {

@esnible esnible Sep 30, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit — this loop is correct today and the "every tier" claim in the doc comment holds: TierOutput is 3, the last of the enum (TierInput=0, TierCacheWrite=1, TierCacheRead=2, TierOutput=3), so 0..3 does cover cache read/write.

But it reads as "input through output," and it would silently stop covering a tier appended after TierOutput. tier < pricing.NumTiers states the intent — which is what the doc comment on NumTiers (rates.go:45) says that constant exists for.

if perMillion := rates.Base[tier] * 1e6; !rates.Set[tier] || math.Abs(perMillion-2) > 1e-9 {
t.Errorf("%s on %s: tier %d = %v per Mtok (set %v), want 2", model, bob, tier, perMillion, rates.Set[tier])
}
}
if got := tab.CurrencyFor(bob, model); got != "Bobcoins" {
t.Errorf("%s on %s: unit %q, want Bobcoins", model, bob, got)
}
}

// The entry is scoped to Bob's host: its "*" model must not reprice anyone else's traffic
// in Bobcoins.
const other = "api.anthropic.com"
if _, prov := tab.Resolve(other, "claude-opus-5-5", 0); prov != pricing.ProvBundled {
t.Errorf("claude-opus-5-5 on %s: provenance %v, want bundled", other, prov)
}
if got := tab.CurrencyFor(other, "claude-opus-5-5"); got != pricing.CurrencyUSD {
t.Errorf("claude-opus-5-5 on %s: unit %q, want %s", other, got, pricing.CurrencyUSD)
}
}
16 changes: 10 additions & 6 deletions docs/pricing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,9 +163,12 @@ curl localhost:47601/v1/usage | jq .unpricedBy
`pricing:` entry.

The Bob endpoint above is the worked example for [Billing units](#billing-units): it bills in
credits rather than dollars, so closing its gap means giving that endpoint a `unit:` as well as
rates. Pricing it without one would record credits as dollars — the figure would look right and be
neither.
Bobcoins rather than dollars, so closing its gap means giving that endpoint a `unit:` as well as
rates. Pricing it without one would record Bobcoins as dollars — the figure would look right and be
neither. The built-in local config (`authbridge-proxy --local --write-config`, which both
`install.sh` and `make dev-install` run) ships that entry, so a new install prices Bob already; a
`~/.cortex/config.yaml` written before it is never rewritten, so an older install still shows this
gap until the block below is added.

The TUI annotates the same thing from the other direction: `abctl observe`'s cost total
carries `[bundled]` or `[configured]` when a total is wholly one provenance, and names the
Expand Down Expand Up @@ -314,15 +317,16 @@ host invites the copies to drift. Each host becomes its own table row.

### Billing units

Rates are in **US dollars unless an endpoint says otherwise**:
Rates are in **US dollars unless an endpoint says otherwise**. This is the entry the built-in
local config ships for IBM Bob, a flat rate on every model and tier:

```yaml
pricing:
endpoints:
- hosts: ["api.us-east.bob.ibm.com"]
unit: credits # absent means USD
unit: Bobcoins # absent means USD
models:
"premium-ide":
"*":
input_cost_per_million: 2.00
output_cost_per_million: 2.00
cache_read_cost_per_million: 2.00 # deliberate: this gateway gives no cache discount
Expand Down
Loading