diff --git a/CHANGELOG.md b/CHANGELOG.md index eaeceb3..8961102 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,11 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- Rebuild the Paykit fixtures on the paykit-rs version a Bitkit pull request pins with `scripts/follow-app-paykit` +- Withhold and restore the payment request issuer's endpoints with `POST /endpoints`, so a journey can make the app's request resolution fail and recover +- Route the apps' homeserver traffic through `homeserver-proxy`, whose control port (6298) delays or fails one identity's homeserver requests by path +- Hold LNURL-pay fixture callbacks with a `delay` mode, issue real invoices of the project's LND, and give a wallet a funded channel to it through LNURL-channel +- Issue payment requests payable through an LNURL-pay endpoint, with a separate proposal expiry + ### Changed +- Move the payment request fixture peers to paykit-rs rc65 on Pubky 0.14.0, the SDK current Bitkit builds pin (the `fixture-issuer` and `rc56-peer` service names stay) +- Move the marketplace fixture to the Pubky 0.14.0 homeserver, which grants the `LOCK` write locks current Bitkit builds take, with Paykit Server `0ffd4da` (pubky/paykit-server#46, paykit-rs rc65, locks-core rc8) and the driver on `@synonymdev/pubky` 0.14.0 - Show ready-to-copy settle and cancel commands in `holdinvoice` output - Simplify LND funding step in README to a single command instead of clipboard-based two-step flow ### Fixed +- Fund the payment request issuer's wallet before `/pay`, which failed with insufficient funds on a fresh regtest chain - Clear stale X display locks in `scripts/trezor-emulator` before starting the emulator, fixing `RuntimeError('Emulator process died')` caused by Xvfb refusing to start over a leftover `/tmp/.X-lock` - Validate LNURL-withdraw callback invoices by millisatoshis (`num_msat`) to preserve msat precision for min/max range checks - Preserve LNURL-pay invoice millisatoshi precision by creating invoices with LND `value_msat` instead of truncating callback amounts to sats diff --git a/README.md b/README.md index 6fe8b86..1c26f42 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ A complete Docker-based development environment for Bitcoin and Lightning Networ - **VSS Server**: Versioned Storage Server for app and ldk-node state backups - **Homegate**: Pubky Homeserver signup gatekeeper with local admin API mock - **Pubky marketplace fixture** (opt-in `marketplace` profile): Pubky testnet, Paykit Server and a purchase driver for the marketplace wallet journey -- **Payment Request fixture** (opt-in `payment-requests` profile): rc56 issuer and controlled peer on the marketplace Pubky testnet +- **Payment Request fixture** (opt-in `payment-requests` profile): rc65 issuer and controlled peer on the marketplace Pubky testnet ## Quick Start @@ -95,6 +95,18 @@ docker compose --profile lnurl-pay exec -T lnurl-server-fixture node --test pay- Use the address and forwarded port reachable by the wallet when requesting `/generate/pay` or `/pay/fixture`: the response derives its URLs from the request's host, including any remapped port. Set `LNURL_FIXTURE_DOMAIN` before starting the service if the wallet must use a different origin (for example `http://10.0.2.2:3010` for an Android emulator). +`{"mode":"delay","ms":N}` holds every callback for `N` milliseconds and then answers with an invoice; without `ms` a callback waits until the next `POST /fixture`, which releases it with that request's mode (`healthy` for an invoice, `error` for an error). A callback is held for 15 minutes at most. `GET /fixture` lists the callbacks with how long each was held and what it answered, and `GET /fixture/invoices` lists the issued invoices with `settled` from LND, so a journey can check that a wallet did not pay after its deadline. + +The profile starts the project's LND beside the fixture, and invoices are real invoices of that LND. To make them payable from a wallet, give it a channel: `GET /generate/channel` returns an LNURL-channel; when the wallet accepts it, LND (funded on the project's bitcoind first when it holds too little) opens a 1,000,000 sat static-remote-key channel that pushes 500,000 sat to the wallet, and mines six blocks to confirm it (`CHANNEL_SATS` and `PUSH_SATS` change the amounts). Ask `/generate/pay` and `/generate/channel` with a `Host` header naming the address the wallet dials (`-H 'Host: 127.0.0.1:3010'` for an Android emulator mapped with `adb reverse`) when you reach the fixture on another port: the encoded LNURL takes its origin from that header. The wallet dials LND at `LND_P2P_ADDRESS` (default `127.0.0.1:9735`, which an Android emulator reaches through `adb reverse tcp:9735 tcp:`). `GET /fixture/channels` shows LND's open and pending channels, and `POST /fixture/mine` with `{"blocks":N}` mines more blocks. + +```bash +curl -fsS http://localhost:3010/generate/channel | jq -r .lnurl # paste or scan in the wallet, then accept the connection +curl -fsS http://localhost:3010/fixture/channels | jq '.open[] | {remote_pubkey, capacity, local_balance, remote_balance, active}' +curl -fsS -X POST http://localhost:3010/fixture -H 'Content-Type: application/json' -d '{"mode":"delay"}' # hold the next callbacks +curl -fsS -X POST http://localhost:3010/fixture -H 'Content-Type: application/json' -d '{"mode":"healthy"}' # release them with invoices +curl -fsS http://localhost:3010/fixture/invoices | jq +``` + Healthy invoices use the requested amount in millisatoshis and bind the exact metadata with a SHA-256 description hash. They are signed, freshly generated `lnbcrt` invoices with a one-hour expiry and payment secret. This fixture supports invoice fetching, decoding and callback retry journeys; it has no Lightning node or channels and cannot settle payments. Use the regular LNURL server with LND for actual payments. Its controls are unauthenticated and intended only for disposable local test environments. ### VSS Server @@ -238,9 +250,9 @@ docker compose logs -f bitcoind ### Bitkit Testing -#### Payment Requests and rc56 Deadline History +#### Payment Requests and Deadline History -The `payment-requests` profile starts two disposable Paykit rc56 SDK peers on +The `payment-requests` profile starts two disposable Paykit rc65 SDK peers (paykit-rs `7185ae7`, Pubky 0.14.0) on the marketplace fixture's Pubky testnet. `fixture-issuer` publishes a regtest Paykit endpoint and sends one-time requests. `rc56-peer` can accept, reject, cancel and pay requests through the shared regtest Bitcoin node. Plain @@ -290,6 +302,12 @@ rejected or canceled records, issue another request and call `/reject` or `POST /request` accepts `monthly_starts_at` (UTC RFC3339) and `period_start_deadline_seconds`; `/pay` then needs `billing_period_start` and `billing_period_end`. +`POST /request` also accepts `proposal_expires_at` (UTC RFC3339, the +acceptance deadline, apart from the payment deadline) and `lnurl`, an LNURL-pay +string such as the LNURL fixture's `GET /generate/pay`: the request then +accepts only `btc-lightning-lnurl`, which the private payment list sent with it +offers beside the regtest address. `/pay` funds the issuer's bitcoind wallet by +mining to it when it holds less than the payment and a fee. Example one-time issuance to a linked app after both sides report `Linked`: @@ -304,13 +322,86 @@ Pubky testnet's network namespace, so `./pubky-marketplace down` and `reset` remove them together with the testnet. After `reset`, start them again with the `up -d --no-build fixture-issuer rc56-peer` command above, wait for `/health`, rerun `payment-requests/prepare` and relink the app. `./pubky-marketplace seed` -needs outbound internet for Paykit Server setup; the rc56 peer calls use the +needs outbound internet for Paykit Server setup; the rc65 peer calls use the local testnet. The lane still needs a Bitkit build pointed at the local Pubky testnet and to verify the requested rows on device. The headless preparation command does not populate a separate Bitkit identity's history; accepted and paid app rows require the lane's controlled client to prepare those records with the app's identity or an app build that supports importing fixture state. +##### Withholding the issuer's endpoints + +A journey that needs the app's request resolution to fail (for example +`requested-resolution-failure.xml`) withholds the issuer's payment endpoints +before it sends the request, then restores them: + +```bash +TO_APP=$(jq -nc --arg pubky "$APP_PUBKY" '{peer_pubky:$pubky,peer_path:"bitkit/wallet"}') +curl -fsS -X POST http://127.0.0.1:3012/endpoints -H 'content-type: application/json' \ + -d "$(jq -c '. + {action:"withhold"}' <<<"$TO_APP")" | jq +curl -fsS -X POST http://127.0.0.1:3012/request -H 'content-type: application/json' \ + -d "$(jq -c '. + {amount_sats:15000,reference:"unresolvable"}' <<<"$TO_APP")" | jq +# ... the app retries and shows "The payment request is no longer available." ... +curl -fsS -X POST http://127.0.0.1:3012/endpoints -H 'content-type: application/json' \ + -d "$(jq -c '. + {action:"restore"}' <<<"$TO_APP")" | jq +curl -fsS http://127.0.0.1:3012/endpoints | jq # {"withheld": false, ...} +``` + +`withhold` removes the issuer's public `btc-regtest-p2wpkh` endpoint and sends +the named peer an empty private payment list. While withheld, `/request` sends +its request with that empty list, so the request names an endpoint the app +cannot resolve (`"endpoints_withheld": true` in its answer). `restore` +publishes the endpoint again and sends the peer the full list; the app's next +attempt resolves it. Without `peer_pubky`, only the public endpoint changes. + +#### Following the apps' Paykit pin + +The Paykit fixtures must run the paykit-rs version the app under test pins: the payment request peers +(`payment-request-fixture:rc65-shared`, paykit-rs `7185ae7`, v0.1.0-rc65) and Paykit Server (built from +the head of [pubky/paykit-server#46](https://github.com/pubky/paykit-server/pull/46), `0ffd4da`, until it +merges or is released). Each image records the paykit-rs commit it was built from (label +`tech.masivo.paykit-rs`, or `/usr/local/share/paykit-rs-rev` in the peers' image). + +```bash +scripts/follow-app-paykit synonymdev/bitkit-android 1401 --check # print the pin and what is out of date (exit 3) +scripts/follow-app-paykit synonymdev/bitkit-ios a6846779a71081f262f47883570125bd541b4fd6 +``` + +It reads the pin at the PR head (Android `gradle/libs.versions.toml`, iOS `Package.resolved`), rebuilds the +peers' image as `payment-request-fixture:-shared` when it was built from another commit, and builds +Paykit Server and the driver from the Paykit Server PR the app PR links (its merge commit once merged), +else from master, when that revision locks the same paykit-rs tag (exit 4 when none does). Afterwards every +tag Compose resolves for those images, `COMPOSE_FILE` overrides included, points at the new build. + +#### Homeserver proxy (selective delay) + +The apps reach the testnet homeserver through `homeserver-proxy`, which the +marketplace profile starts with the testnet: the host's 6287 (Pubky TLS) goes +to it, it presents the static testnet's homeserver key (secret `[0; 32]`) and +forwards every request to the homeserver's plain HTTP on 6286. Clients inside +the testnet's namespace (Paykit Server, the payment request peers) still reach +the homeserver on 6287 directly. Without rules the proxy only forwards. + +Its control port, 6298, delays or fails the requests of one identity whose +owner-relative path starts with `path` (empty matches every path): + +```bash +# hold one identity's own-profile reads for 20 s +curl -fsS -X POST http://127.0.0.1:6298/rules -H 'content-type: application/json' \ + -d "$(jq -nc --arg pubky "$APP_PUBKY" '{pubky:$pubky,path:"/pub/pubky.app/profile.json",delay_ms:20000}')" | jq +# fail them instead (after an optional delay): add "status": 503 +curl -fsS http://127.0.0.1:6298/rules | jq +curl -fsS http://127.0.0.1:6298/requests | jq '.requests[-20:]' # owner, path, status, delayed_ms of recent requests +curl -fsS -X DELETE http://127.0.0.1:6298/rules | jq # remove every rule +``` + +A rule with the same `pubky` and `path` replaces the earlier one. `pubky` +takes the z32 key with or without its `pubky` prefix. `/requests` lists the +last 200 requests, which shows the paths an app reads for an identity; +`docker compose logs homeserver-proxy` prints the same lines. Set rules before +the app makes the request: a held request waits on its open connection, and +requests that reach the homeserver before the rule are not held. + #### Trezor Hardware PRs For isolated Linux or Docker-backed simulator projects, use the optional @@ -457,7 +548,7 @@ Then, with the buyer wallet: Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. By default the seller of a purchase is the fixture's headless seller, and `verify` always uses it. -To make a Bitkit wallet the seller, the wallet approves two Pubky requests for the same identity: the Paykit watch-only setup (it gives Paykit Server the wallet's account xpub, so payouts land in that wallet) and a write grant on `/pub/locks.app/` (the role Locks plays: the driver publishes the payment lock with the granted session). One request cannot carry both, because the apps accept the watch-only claim only for exactly the two Paykit paths. +To make a Bitkit wallet the seller, the wallet approves two Pubky requests for the same identity: the Paykit watch-only setup (it gives Paykit Server the wallet's account xpub, so payouts land in that wallet) and a write grant on `/pub/app.locks/` (the role Locks plays: the driver publishes the payment lock with the granted session). One request cannot carry both, because the apps accept the watch-only claim only for exactly the two Paykit paths. ```bash ./pubky-marketplace seed --buyer none # once per fixture; the headless seller stays unused diff --git a/docker-compose.yml b/docker-compose.yml index 6c8841f..7b34c38 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -162,9 +162,19 @@ services: build: context: ./lnurl-server dockerfile: Dockerfile.pay-fixture + # real invoices of the project's LND (payable once `/channel/fixture` gave the wallet a channel), and `/fixture/mine` on its bitcoind + depends_on: + - lnd + - darkhttpd environment: PORT: '3010' LNURL_FIXTURE_DOMAIN: ${LNURL_FIXTURE_DOMAIN:-} + LND_REST_URL: https://lnd:8080 + LND_DIR: /lnd + LND_P2P_ADDRESS: 127.0.0.1:9735 + BITCOIN_RPC_URL: http://polaruser:polarpass@bitcoind:43782 + volumes: + - ./lnd:/lnd:ro ports: - '3010:3010' healthcheck: @@ -412,7 +422,9 @@ services: pubky-testnet: profiles: [marketplace] container_name: marketplace-pubky-testnet - image: bitkit-docker/pubky-testnet:f68014c1 + # The homeserver of 0.14.0 answers the Pubky SDK's LOCK and UNLOCK write locks that current Bitkit builds take before writing Paykit state; the + # earlier Pubky Core pin (f68014c1) answered 405. + image: bitkit-docker/pubky-testnet:0.14.0 build: context: ./marketplace/pubky-testnet restart: "no" # the fixture is disposable: a restarted testnet loses its accounts @@ -428,13 +440,15 @@ services: - "127.0.0.1:15411:15411" # PKARR relay - "127.0.0.1:15412:15412" # HTTP relay - "127.0.0.1:6286:6286" # homeserver ICANN HTTP - - "127.0.0.1:6287:6287" # homeserver Pubky TLS + - "127.0.0.1:6287:6297" # homeserver Pubky TLS, through homeserver-proxy (in-namespace clients reach the homeserver on 6287 directly) + - "127.0.0.1:6298:6298" # homeserver-proxy control: delay or fail one identity's requests - "127.0.0.1:${MARKETPLACE_HOMESERVER_ADMIN_PORT:-16288}:6288" # homeserver admin (6288 is homegate's) - "127.0.0.1:${MARKETPLACE_PAYKIT_PORT:-3001}:3001" # paykit-server, shares this namespace - "127.0.0.1:3012:3012" # opt-in fixture-issuer, shares this namespace - "127.0.0.1:3013:3013" # opt-in rc56-peer, shares this namespace - # Paykit Server 722ef268 (v0.1.0-rc4), built from source with the upstream + # Paykit Server 0ffd4da, the head of pubky/paykit-server#46 (paykit-rs rc65, the version both apps pin; Pubky 0.14.0; move to its merge or + # release once #46 lands), built from source with the upstream # Dockerfile.local. Its setup flow emits the Pubky grant auth URL (cid and cpk) # that the apps' Paykit SDK requires. `./pubky-marketplace build` first checks # out the pinned trees under .marketplace/sources and confirms that the tree's @@ -443,10 +457,14 @@ services: paykit-server: profiles: [marketplace] container_name: marketplace-paykit-server - image: bitkit-docker/paykit-server:722ef268 + image: bitkit-docker/paykit-server:${PAYKIT_SERVER_TAG:-0ffd4da} build: context: ./.marketplace/sources/paykit-server dockerfile: Dockerfile.local + labels: + tech.masivo.paykit-server: ${PAYKIT_SERVER_REV:-0ffd4da2adaab048939e0ea18ff916a27a7cfb0e} + tech.masivo.paykit-rs: ${PAYKIT_RS_REV:-7185ae7da9315028e5331442d71d739c27f1442c} + tech.masivo.paykit-server-patches: ${PAYKIT_SERVER_PATCHES:-paykit-server-reader-accepts-proposal-expiry.patch} additional_contexts: paykit-lib: ./.marketplace/sources/paykit-rs/paykit-lib paykit-sdk: ./.marketplace/sources/paykit-rs/paykit-sdk @@ -486,11 +504,15 @@ services: # paykit-reader-demo helper binaries from the paykit-server image. marketplace-driver: profiles: [marketplace] - image: bitkit-docker/marketplace-driver:local + image: bitkit-docker/marketplace-driver:${PAYKIT_SERVER_TAG:-0ffd4da} build: context: ./marketplace/driver additional_contexts: paykit: service:paykit-server + fixture: docker-image://bitkit-docker/payment-request-fixture:rc65-shared + labels: + tech.masivo.paykit-server: ${PAYKIT_SERVER_REV:-0ffd4da2adaab048939e0ea18ff916a27a7cfb0e} + tech.masivo.paykit-rs: ${PAYKIT_RS_REV:-7185ae7da9315028e5331442d71d739c27f1442c} network_mode: service:pubky-testnet depends_on: - pubky-testnet @@ -504,10 +526,23 @@ services: - ./.marketplace/evidence:/evidence entrypoint: ["node", "/app/driver.mjs"] - # The rc56 SDK peers share the marketplace testnet namespace and regtest chain. + # Pubky TLS proxy in front of the testnet homeserver: the apps reach the homeserver through it (the host's 6287 above). + # It presents the static testnet's homeserver key and forwards to the homeserver's plain HTTP on 6286; its control port + # (6298) delays or fails the requests of one identity and path (README, Homeserver proxy). Without rules it only forwards. + homeserver-proxy: + profiles: [marketplace] + image: bitkit-docker/payment-request-fixture:rc65-shared + build: ./payment-requests + restart: on-failure + network_mode: service:pubky-testnet + depends_on: + - pubky-testnet + entrypoint: ["/usr/local/bin/homeserver-proxy"] + + # The rc65 SDK peers (paykit-rs 7185ae7, the version both apps pin) share the marketplace testnet namespace and regtest chain. The service name `rc56-peer` stays: lanes and journeys address the peer by it. fixture-issuer: profiles: [payment-requests] - image: bitkit-docker/payment-request-fixture:rc56-24162ebb + image: bitkit-docker/payment-request-fixture:rc65-shared build: ./payment-requests restart: "no" network_mode: service:pubky-testnet @@ -521,7 +556,7 @@ services: rc56-peer: profiles: [payment-requests] - image: bitkit-docker/payment-request-fixture:rc56-24162ebb + image: bitkit-docker/payment-request-fixture:rc65-shared build: ./payment-requests restart: "no" network_mode: service:pubky-testnet diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 175cfb2..61e97dd 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -10,39 +10,40 @@ and [bitkit-android#1338](https://github.com/synonymdev/bitkit-android/pull/1338 | Piece | Where | Pin | | --- | --- | --- | | Regtest bitcoind and Electrum on `tcp://127.0.0.1:60001` | the stack's `bitcoind` and `electrs` | as in `docker-compose.yml` | -| Pubky Core static testnet: DHT, PKARR relay, HTTP relay, one homeserver with open signup | `pubky-testnet`, built from `marketplace/pubky-testnet/Dockerfile` | pubky-core `f68014c1` | +| Pubky static testnet: DHT, PKARR relay, HTTP relay, one homeserver with open signup | `pubky-testnet`, built from `marketplace/pubky-testnet/Dockerfile` | `pubky-testnet` crate 0.14.0 | | Homeserver and Paykit databases | `marketplace-postgres` | `postgres:16-alpine` | -| Paykit Server | `paykit-server`, built from source with the upstream `Dockerfile.local` | pubky/paykit-server `722ef268` (v0.1.0-rc4), paykit-rs `9b56a0ea` (v0.1.0-rc48), locks-core `8502ef79` (v0.1.0-rc1) | -| Purchase driver | `marketplace-driver`, run by `./pubky-marketplace` | `marketplace/driver/package-lock.json`, `@synonymdev/pubky` 0.10.0 | +| Paykit Server | `paykit-server`, built from source with the upstream `Dockerfile.local` (classic builder without BuildKit, see below) | pubky/paykit-server `0ffd4da2` (head of [pubky/paykit-server#46](https://github.com/pubky/paykit-server/pull/46), not yet merged or released; image label `tech.masivo.paykit-server`), paykit-rs `7185ae7d` (v0.1.0-rc65, the version both apps pin; label `tech.masivo.paykit-rs`), locks-core `b3dc87c9` (v0.1.0-rc8) | +| Purchase driver | `marketplace-driver`, run by `./pubky-marketplace` | `marketplace/driver/package-lock.json`, `@synonymdev/pubky` 0.14.0, Paykit helpers from the Paykit Server image (tagged with the same revision) | `./pubky-marketplace build` checks the pinned trees out under `.marketplace/sources` (git ignored) and fails if a checkout is not at its pin or if the Paykit Server tree's `Cargo.lock` does not lock paykit-rs and locks-core to those revisions. `Dockerfile.local` then fails closed if a tree differs from the pins in -Paykit Server's Cargo manifests. The pins are at the top of `pubky-marketplace`. - -### Why this Paykit Server revision - -The apps ship Paykit SDK `0.1.0-rc55` (bitkit-ios and bitkit-android at their 2026-09-29 heads). Its setup -approval accepts only the Pubky grant auth URL: `pubkyauth://signin_grant` with `cid` and `cpk`. Paykit Server -`867fc883` (the merge of pubky/paykit-server#2) is built on paykit-rs rc43 and emits the legacy -`pubkyauth://signin?caps&relay&secret&x-bitkit-claim` URL, which both apps reject ("Missing query parameter -cid"). Paykit Server adopted grant URLs with paykit-rs rc48, and `722ef268` (v0.1.0-rc4) is the newest -merged revision. It keeps `/setup` and `x-bitkit-claim=watch-only-account-v1`. Paykit Server pins paykit-rs -rc48, three releases before the apps' rc55, and no setup, auth or companion-claim code changed between them; -the J1 device run on 2026-09-29 already delivered requests from a paykit-rs rc43 server to rc55 apps. The -driver's `setup-url` refuses any auth URL that is not `signin_grant` with `cid` and `cpk`, so a wrong pin -fails before it reaches a wallet. Unmerged Paykit Server branches move to paykit-rs rc56; they are not -pinned here. - -### Why `@synonymdev/pubky` 0.10.0 - -Both the Bitkit seller approval and the headless seller need the grant auth flow, which the driver's earlier -0.9.3 client lacks (it has cookie auth only). The pinned homeserver, Pubky Core `f68014c1` (2026-07-31), sits -between v0.9.3 and v0.10.0 (2026-08-05); the commits between it and v0.10.0 are documentation, callback -parameters and one error-surfacing change. 0.10.0 is therefore the client that matches the homeserver. 0.11.0 and -later upgrade pkarr to v8 and the relay to v2 past that homeserver and are not used until the testnet pin moves. -In 0.10.0 a signin names its client and returns a grant session, so the headless seller signs in as -`marketplace.fixture`. +Paykit Server's Cargo manifests. The pins are at the top of `pubky-marketplace`; move `PAYKIT_SERVER_REV` to #46's merge commit or +release once it lands. Docker without BuildKit cannot build `Dockerfile.local` (named contexts, cache mounts), so `build` then generates a +classic Dockerfile from it (the named contexts become COPYs from `.marketplace/sources`) with the same labels. `build-paykit` builds only +Paykit Server and the driver, and takes the pins from the environment; `scripts/follow-app-paykit` uses it (README, Following the apps' +Paykit pin). + +### Why these versions + +Current Bitkit builds (Paykit SDK rc65) take a Pubky write lock (`LOCK` and `UNLOCK` on the path) before they write Paykit state. The +homeserver of the earlier Pubky Core pin `f68014c1` answers `LOCK` with 405, so creating a profile or publishing Paykit data failed in the +app. The 0.14.0 homeserver grants the locks. + +Paykit Server `0ffd4da` (#46) is on Pubky 0.14.0 and paykit-rs rc65 and keeps `/setup` with `x-bitkit-claim=watch-only-account-v1`; +its setup flow emits the Pubky grant auth URL (`pubkyauth://signin_grant` with `cid` and `cpk`) that the apps accept. The driver's +`setup-url` refuses any auth URL that is not `signin_grant` with `cid` and `cpk`, so a wrong pin fails before it reaches a wallet. The +server's config names its Paykit app with `app_id` (paykit-rs has no receiver folders since rc59), and the driver reads the app registry +(`/pub/paykit/v0/app-registry.json`) where it read `receiver.json`. The driver's client is `@synonymdev/pubky` 0.14.0, the release of the +homeserver, whose signin names its client and returns a grant session, so the headless seller signs in as `marketplace.fixture`. + +### Known limit: the headless seller stand-in + +Paykit Server verifies the seller's app registry (`/pub/paykit/v0/app-registry.json`, written by the Paykit SDK from the seller's Paykit +identity key) before it persists a setup. A Bitkit wallet publishes it itself, so the app paths work: `setup-url`, `setup-wait`, `seller-auth`, +`purchase --seller bitkit`, `receive`, `pay`, `mine` and `peers` against a Bitkit seller, with the headless buyer. The Node driver has no Paykit SDK +to publish that registry for its own headless seller, so `seed` stops at `setup flow ended with HTTP 422 setup_failed`, and `verify` and +`verify-bitkit-seller`, which use the headless seller or a headless stand-in for the wallet, do not run until the driver gets one. ## Ports @@ -93,7 +94,7 @@ Everything lives in the `marketplace_state` volume, and `down` deletes it. - `/state/paykit` (readable by the Paykit Server process): generated config and master key. - `/state/secrets` (root, mode 0700, unreadable by Paykit Server): issuer seed, seller identity seed, - seller wallet seed, buyer identity seed, and `bitkit-seller.session`, the `/pub/locks.app/` grant session a + seller wallet seed, buyer identity seed, and `bitkit-seller.session`, the `/pub/app.locks/` grant session a Bitkit seller approved (bearer-equivalent for that path; the grant lasts two years). - `/state/fixture.json`, `/state/purchases.json`: public facts and the purchase ledger. - `.marketplace/evidence//summary.json`: `verify` output, owned by the user who ran the wrapper (the driver @@ -111,7 +112,7 @@ approves, and payouts land in the wallet. That takes two approvals of the same P | Approval | Fixture command | Requester ID | Permissions | Gives the fixture | | --- | --- | --- | --- | --- | | Paykit setup (`x-bitkit-claim=watch-only-account-v1`) | `setup-url`, then `setup-wait ` | `app.paykit.server` | `/pub/paykit/v0/bitkit/server` and `/pub/paykit/v0/private/bitkit/server`, READ, WRITE | Paykit Server holds the wallet's account xpub and derives the payout addresses | -| Marketplace grant | `seller-auth` | `locks.app` | `/pub/locks.app`, READ, WRITE | a session that writes the payment lock to the seller's homeserver | +| Marketplace grant | `seller-auth` | `locks.app` | `/pub/app.locks`, READ, WRITE | a session that writes the payment lock to the seller's homeserver | One approval cannot carry both. Both apps accept the watch-only claim only when the requested capabilities are exactly the two Paykit paths (a claim with other capabilities, or those two paths without a claim, is @@ -119,7 +120,7 @@ rejected), and Paykit Server fixes those capabilities. An approval without the c grant request: both apps accept any capabilities and requester ID for it and show them for the user to approve, so the marketplace grant needs no app change. -`seller-auth` starts a grant flow (`startGrantAuthFlow` with `/pub/locks.app/:rw`, client id `locks.app`) on the +`seller-auth` starts a grant flow (`startGrantAuthFlow` with `/pub/app.locks/:rw`, client id `locks.app`) on the testnet's HTTP relay, prints the `pubkyauth://signin_grant?caps&relay&secret&cid&cpk` URL, waits up to `--timeout` seconds (default 300; the relay keeps a request about five minutes) and stores the approved session under `/state/secrets`. It records the approving identity as the Bitkit seller and reports Paykit's @@ -130,7 +131,7 @@ setup state for it. `setup-wait` then checks the wallet that approved the setup `seller-auth` prints one compact JSON object per line: `awaiting_approval` (with `auth_url`, `android`, `ios`) at once, then `approved` when the wallet has approved. Read the request from the first line (`... | head -1 | jq -r .auth_url`, or `jq -r 'select(.status == "awaiting_approval") | .auth_url'` over the stream) and collect both -with `jq -s`. `info` shows the result as `bitkit_seller.marketplace_grant` (`locks.app /pub/locks.app/:rw`) +with `jq -s`. `info` shows the result as `bitkit_seller.marketplace_grant` (`locks.app /pub/app.locks/:rw`) and `bitkit_seller.setup_completed_at` (null until `setup-wait` has seen the setup complete), next to `bitkit_seller.pubky` and `kind`; `seller.pubky` is the unused headless seller. @@ -233,7 +234,7 @@ it reads `not_observable`: the fixture holds no key for the app's end of the lin not on the local relay, and offers no config to change it. `seed` and a Bitkit seller's setup approval therefore need outbound internet. Only that one-time handshake leaves the machine: the marketplace grant of a Bitkit seller uses the local relay unless `seller-auth --relay` names another. -- **Locks authority.** A Bitkit seller gives the driver only a write grant on `/pub/locks.app/`, the path Locks +- **Locks authority.** A Bitkit seller gives the driver only a write grant on `/pub/app.locks/`, the path Locks publishes locks under, through the Pubky grant session path. Locks' own connect flow and its other seller APIs are out of scope. - **No Locks server, no guarded content.** The lock has no guarded resource, and the fixture does not @@ -244,5 +245,12 @@ it reads `not_observable`: the fixture holds no key for the app's end of the lin setup with the current pin has been run headlessly (`seed` and `verify`) and not yet against an app. The Bitkit seller path (`seller-auth`, `purchase --seller bitkit`) has a headless self-test, `verify-bitkit-seller`, and has not been run against an app yet. +- **Patched Paykit Server reader helper.** Since pubky/paykit-server `468f12c` (2 Oct, on master and in #46) every invoice's Payment + Request carries an acceptance deadline (`proposal_expires_at`), and the server's own `paykit-reader-demo` still rejects any request + that has one, so the headless buyer's `receive` ends in `protocol_failed`. The fixture applies + `marketplace/patches/paykit-server-reader-accepts-proposal-expiry.patch` to the pinned tree (image label + `tech.masivo.paykit-server-patches`); `fetch_sources` stops when a patch no longer applies, which is the sign upstream fixed it. +- **Paykit Server on an unmerged branch.** The apps pin paykit-rs rc65, and only pubky/paykit-server#46 (`0ffd4da`) builds Paykit + Server on rc65; master (`7ff868b`) is still on rc59. The fixture builds from #46's head until it merges or is released. - **Fixed container names.** The base services keep their fixed container names, so another checkout's stack with the same names must be removed first. diff --git a/lnurl-server/pay-fixture.js b/lnurl-server/pay-fixture.js index 83ea725..2de92bb 100644 --- a/lnurl-server/pay-fixture.js +++ b/lnurl-server/pay-fixture.js @@ -1,39 +1,132 @@ const express = require('express'); +const fs = require('fs'); +const https = require('https'); +const path = require('path'); const { createHash, randomBytes } = require('crypto'); const bolt11 = require('bolt11'); const { encode } = require('lnurl'); const QRCode = require('qrcode'); -// Disposable invoices for fetching and decoding, without an LND wallet or channels. -// The public test key has no funds and is never used by a Lightning node. +// Invoices for LNURL-pay journeys. With `LND_REST_URL` set (the `lnurl-pay` profile starts the project's LND beside this fixture), +// each invoice is a real one of that LND, payable by a wallet that holds a channel to it (`/channel/fixture` opens one). Without it, +// invoices are disposable ones signed with a public test key, for fetching and decoding only. const testKey = '01'.padStart(64, '0'); const metadata = JSON.stringify([['text/plain', 'LNURL-pay regtest fixture']]); const minSendable = 1000; const maxSendable = 1000000000; const reason = 'LNURL fixture invoice callback unavailable'; +const modes = ['error', 'healthy', 'delay']; +// a held callback answers at the latest after this, so a forgotten `delay` does not hold a wallet's request for ever +const maxHoldMs = 15 * 60 * 1000; -function createApp() { +function lndClient(env) { + const base = env.LND_REST_URL && env.LND_REST_URL.replace(/\/$/, ''); + if (!base) return null; + const dir = env.LND_DIR || '/lnd'; + const macaroonPath = env.LND_MACAROON_PATH || path.join(dir, 'data/chain/bitcoin/regtest/admin.macaroon'); + return (method, route, body) => new Promise((resolve, reject) => { + let macaroon; + try { + macaroon = fs.readFileSync(macaroonPath).toString('hex'); + } catch (err) { + return reject(new Error(`LND is not ready: no macaroon at ${macaroonPath}`)); + } + const url = new URL(base + route); + // the regtest LND's self-signed certificate names its container, not the service name this fixture dials + const req = https.request(url, { method, rejectUnauthorized: false, headers: { 'Grpc-Metadata-macaroon': macaroon, 'Content-Type': 'application/json' }, timeout: 30000 }, (res) => { + let data = ''; + res.on('data', (chunk) => { data += chunk; }); + res.on('end', () => { + let parsed; + try { parsed = data ? JSON.parse(data) : {}; } catch (err) { return reject(new Error(`LND ${route}: ${data.slice(0, 200)}`)); } + if (res.statusCode >= 400) return reject(new Error(`LND ${route}: ${parsed.message || parsed.error || res.statusCode}`)); + resolve(parsed); + }); + }); + req.on('timeout', () => req.destroy(new Error(`LND ${route}: timed out`))); + req.on('error', reject); + if (body) req.write(JSON.stringify(body)); + req.end(); + }); +} + +function bitcoinClient(env) { + const url = env.BITCOIN_RPC_URL; + if (!url) return null; + return async (method, params = []) => { + const parsed = new URL(url); + const auth = Buffer.from(`${decodeURIComponent(parsed.username)}:${decodeURIComponent(parsed.password)}`).toString('base64'); + parsed.username = ''; + parsed.password = ''; + const res = await fetch(parsed, { method: 'POST', headers: { Authorization: `Basic ${auth}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ jsonrpc: '1.0', id: 'lnurl-fixture', method, params }) }); + const out = await res.json(); + if (out.error) throw new Error(`bitcoind ${method}: ${out.error.message}`); + return out.result; + }; +} + +function createApp(env = process.env) { const app = express(); + const lnd = lndClient(env); + const bitcoin = bitcoinClient(env); let mode = 'error'; + let delayMs = null; + let released = 0; + const wakers = new Set(); + const callbacks = []; + const invoices = []; + const channels = new Map(); app.use(express.json()); app.use((req, res, next) => { res.set('Cache-Control', 'no-store'); next(); }); - const origin = (req) => (process.env.LNURL_FIXTURE_DOMAIN || `${req.protocol}://${req.get('host')}`).replace(/\/$/, ''); + const origin = (req) => (env.LNURL_FIXTURE_DOMAIN || `${req.protocol}://${req.get('host')}`).replace(/\/$/, ''); const error = (res, message) => res.json({ status: 'ERROR', reason: message }); + const wake = () => { for (const fn of wakers) fn(); }; + const state = () => ({ mode, delay_ms: delayMs, held: callbacks.filter((c) => !c.answered_at).length, invoices: lnd ? 'lnd' : 'synthetic' }); - app.get('/health', (req, res) => res.json({ status: 'OK', network: 'regtest', mode })); - app.get('/fixture', (req, res) => res.json({ mode })); + app.get('/health', (req, res) => res.json({ status: 'OK', network: 'regtest', ...state() })); + app.get('/fixture', (req, res) => res.json({ ...state(), callbacks })); + // `{mode: "delay", ms}` holds every callback for `ms` (without `ms`, until the next POST); any later POST releases the held ones, + // which then answer as the new mode says (`delay` released by its own timer answers as `healthy`) app.post('/fixture', (req, res) => { - if (!['error', 'healthy'].includes(req.body.mode)) { - return res.status(400).json({ status: 'ERROR', reason: 'mode must be error or healthy' }); + if (!modes.includes(req.body.mode)) { + return res.status(400).json({ status: 'ERROR', reason: `mode must be one of ${modes.join(', ')}` }); + } + const { ms } = req.body; + if (req.body.mode === 'delay' && ms !== undefined && (!Number.isSafeInteger(ms) || ms < 0 || ms > maxHoldMs)) { + return res.status(400).json({ status: 'ERROR', reason: `ms must be an integer from 0 to ${maxHoldMs}` }); } mode = req.body.mode; - res.json({ mode }); + delayMs = mode === 'delay' && ms !== undefined ? ms : null; + released += 1; + wake(); + res.json(state()); }); + const invoice = async (amount) => { + const descriptionHash = createHash('sha256').update(metadata).digest(); + if (lnd) { + const created = await lnd('POST', '/v1/invoices', { value_msat: amount, description_hash: descriptionHash.toString('base64'), expiry: 3600 }); + return { pr: created.payment_request, hash: Buffer.from(created.r_hash, 'base64').toString('hex') }; + } + const hash = createHash('sha256').update(randomBytes(32)).digest('hex'); + const encoded = bolt11.encode({ + network: { bech32: 'bcrt', pubKeyHash: 111, scriptHash: 196, validWitnessVersions: [0, 1] }, + millisatoshis: amount, + tags: [ + { tagName: 'payment_hash', data: hash }, + { tagName: 'payment_secret', data: randomBytes(32).toString('hex') }, + { tagName: 'purpose_commit_hash', data: descriptionHash.toString('hex') }, + { tagName: 'expire_time', data: 3600 }, + { tagName: 'min_final_cltv_expiry', data: 18 } + ] + }); + return { pr: bolt11.sign(encoded, testKey).paymentRequest, hash }; + }; + app.get('/pay/fixture', (req, res) => res.json({ tag: 'payRequest', callback: `${origin(req)}/pay/fixture/callback`, @@ -43,37 +136,154 @@ function createApp() { commentAllowed: 0 })); - app.get('/pay/fixture/callback', (req, res) => { - const { amount } = req.query; - if (typeof amount !== 'string' || !/^\d+$/.test(amount) || - !Number.isSafeInteger(Number(amount)) || Number(amount) < minSendable || Number(amount) > maxSendable) { - return error(res, 'amount must be an integer within the advertised millisatoshi range'); + app.get('/pay/fixture/callback', async (req, res, next) => { + try { + const { amount } = req.query; + if (typeof amount !== 'string' || !/^\d+$/.test(amount) || + !Number.isSafeInteger(Number(amount)) || Number(amount) < minSendable || Number(amount) > maxSendable) { + return error(res, 'amount must be an integer within the advertised millisatoshi range'); + } + const entry = { at: new Date().toISOString(), amount_msat: Number(amount), held_ms: 0, answered_at: null, answer: null }; + callbacks.push(entry); + if (mode === 'delay') { + const start = Date.now(); + const generation = released; + const limit = Math.min(delayMs === null ? maxHoldMs : delayMs, maxHoldMs); + await new Promise((resolve) => { + let timer; + const done = () => { clearTimeout(timer); wakers.delete(check); resolve(); }; + const check = () => { if (released !== generation) done(); }; + timer = setTimeout(done, limit); + wakers.add(check); + req.on('close', () => { if (!res.writableEnded) done(); }); + }); + entry.held_ms = Date.now() - start; + } + entry.answered_at = new Date().toISOString(); + if (mode === 'error') { + entry.answer = 'error'; + return error(res, reason); + } + const made = await invoice(amount); + entry.answer = 'invoice'; + entry.payment_hash = made.hash; + invoices.push({ payment_hash: made.hash, amount_msat: Number(amount), issued_at: entry.answered_at }); + res.json({ pr: made.pr, routes: [] }); + } catch (err) { + next(err); } - if (mode === 'error') return error(res, reason); + }); - const invoice = bolt11.encode({ - network: { bech32: 'bcrt', pubKeyHash: 111, scriptHash: 196, validWitnessVersions: [0, 1] }, - millisatoshis: amount, - tags: [ - { tagName: 'payment_hash', data: createHash('sha256').update(randomBytes(32)).digest('hex') }, - { tagName: 'payment_secret', data: randomBytes(32).toString('hex') }, - { tagName: 'purpose_commit_hash', data: createHash('sha256').update(metadata).digest('hex') }, - { tagName: 'expire_time', data: 3600 }, - { tagName: 'min_final_cltv_expiry', data: 18 } - ] - }); - res.json({ pr: bolt11.sign(invoice, testKey).paymentRequest, routes: [] }); + // what the issued invoices became: `settled` comes from LND, so a journey can tell that no payment went out after a deadline + app.get('/fixture/invoices', async (req, res, next) => { + try { + const out = []; + for (const item of invoices) { + let settled = null; + if (lnd) settled = (await lnd('GET', `/v1/invoice/${item.payment_hash}`)).state === 'SETTLED'; + out.push({ ...item, settled }); + } + res.json({ invoices: out }); + } catch (err) { + next(err); + } }); - app.get('/generate/pay', async (req, res, next) => { + const generate = (route, type) => async (req, res, next) => { try { - const url = `${origin(req)}/pay/fixture`; + const url = `${origin(req)}${route}`; const lnurl = encode(url); - res.json({ url, lnurl, qrCode: await QRCode.toDataURL(lnurl), type: 'pay' }); + res.json({ url, lnurl, qrCode: await QRCode.toDataURL(lnurl), type }); + } catch (err) { + next(err); + } + }; + app.get('/generate/pay', generate('/pay/fixture', 'pay')); + app.get('/generate/channel', generate('/channel/fixture', 'channel')); + + const needLnd = (res) => { + if (lnd) return false; + res.status(400).json({ status: 'ERROR', reason: 'this fixture runs without LND (LND_REST_URL unset)' }); + return true; + }; + const mine = async (blocks, address) => { + if (!bitcoin) throw new Error('BITCOIN_RPC_URL unset'); + return bitcoin('generatetoaddress', [blocks, address || await bitcoin('getnewaddress', ['', 'bech32'])]); + }; + + // LNURL-channel: the wallet connects to LND at `LND_P2P_ADDRESS` (the device reaches it on its own loopback), then LND opens a + // channel of `CHANNEL_SATS` and pushes `PUSH_SATS` to the wallet, so the wallet can pay this fixture's invoices; six blocks confirm it + app.get('/channel/fixture', async (req, res, next) => { + try { + if (needLnd(res)) return; + const info = await lnd('GET', '/v1/getinfo'); + const k1 = randomBytes(32).toString('hex'); + channels.set(k1, { created_at: new Date().toISOString() }); + res.json({ tag: 'channelRequest', uri: `${info.identity_pubkey}@${env.LND_P2P_ADDRESS || '127.0.0.1:9735'}`, callback: `${origin(req)}/channel/fixture/callback`, k1 }); + } catch (err) { + next(err); + } + }); + + app.get('/channel/fixture/callback', async (req, res, next) => { + try { + if (needLnd(res)) return; + const { k1, remoteid, private: isPrivate, cancel } = req.query; + const request = channels.get(k1); + if (!request || request.channel_point) return error(res, 'unknown or used k1'); + if (cancel === '1') { + channels.delete(k1); + return res.json({ status: 'OK' }); + } + if (!/^0[23][0-9a-f]{64}$/.test(remoteid || '')) return error(res, 'remoteid must be a compressed public key'); + const capacity = Number(env.CHANNEL_SATS || 1000000); + const push = Number(env.PUSH_SATS || 500000); + const funded = async () => Number((await lnd('GET', '/v1/balance/blockchain')).confirmed_balance || 0) >= capacity * 2; + if (!(await funded())) await mine(101, (await lnd('GET', '/v1/newaddress?type=WITNESS_PUBKEY_HASH')).address); + // LND refuses to open a channel until it has synced the chain; after mining it also has to count the matured coinbase + for (let i = 0; i < 60 && !((await lnd('GET', '/v1/getinfo')).synced_to_chain && await funded()); i++) { + await new Promise((resolve) => setTimeout(resolve, 1000)); + } + const opened = await lnd('POST', '/v1/channels', { + node_pubkey: Buffer.from(remoteid, 'hex').toString('base64'), + local_funding_amount: String(capacity), + push_sat: String(push), + private: isPrivate === '1', + // a static-remote-key channel: an anchor channel would need the wallet to hold an on-chain reserve first + commitment_type: 'STATIC_REMOTE_KEY', + spend_unconfirmed: true + }); + request.channel_point = `${Buffer.from(opened.funding_txid_bytes, 'base64').reverse().toString('hex')}:${opened.output_index}`; + request.remoteid = remoteid; + await mine(6); + res.json({ status: 'OK' }); + } catch (err) { + // LNURL wallets read a JSON ERROR, not an HTTP failure + error(res, err.message); + } + }); + + app.get('/fixture/channels', async (req, res, next) => { + try { + if (needLnd(res)) return; + const [open, pending] = await Promise.all([lnd('GET', '/v1/channels'), lnd('GET', '/v1/channels/pending')]); + res.json({ requests: [...channels.values()], open: open.channels || [], pending: pending.pending_open_channels || [] }); } catch (err) { next(err); } }); + + // mines `blocks` (default 6) on the project's bitcoind, to confirm a channel or a wallet's on-chain payment + app.post('/fixture/mine', async (req, res, next) => { + try { + const blocks = req.body.blocks === undefined ? 6 : req.body.blocks; + if (!Number.isSafeInteger(blocks) || blocks < 1 || blocks > 200) return res.status(400).json({ status: 'ERROR', reason: 'blocks must be 1 to 200' }); + res.json({ mined: (await mine(blocks, req.body.address)).length }); + } catch (err) { + next(err); + } + }); + app.use((err, req, res, next) => res.status(500).json({ status: 'ERROR', reason: err.message })); return app; } diff --git a/lnurl-server/pay-fixture.test.js b/lnurl-server/pay-fixture.test.js index 41c0704..cbf6f72 100644 --- a/lnurl-server/pay-fixture.test.js +++ b/lnurl-server/pay-fixture.test.js @@ -60,3 +60,39 @@ test('LNURL metadata stays available while callbacks fail until explicitly switc await setMode('error'); assert.equal((await get(callbackPath)).status, 'ERROR'); }); + +test('a delayed callback is held until its time passes or the next mode change releases it', async (t) => { + const server = createApp({}).listen(0, '127.0.0.1'); + await new Promise((resolve) => server.once('listening', resolve)); + t.after(() => { + server.closeAllConnections(); + server.close(); + }); + const base = `http://127.0.0.1:${server.address().port}`; + const get = async (path) => (await fetch(`${base}${path}`)).json(); + const setMode = (body) => fetch(`${base}/fixture`, { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) + }); + const callbackPath = '/pay/fixture/callback?amount=21000'; + + assert.equal((await setMode({ mode: 'delay', ms: -1 })).status, 400); + assert.equal((await setMode({ mode: 'delay', ms: 300 })).status, 200); + let start = Date.now(); + assert.ok(bolt11.decode((await get(callbackPath)).pr)); + assert.ok(Date.now() - start >= 280); + + assert.equal((await setMode({ mode: 'delay' })).status, 200); + start = Date.now(); + const held = get(callbackPath); + await new Promise((resolve) => setTimeout(resolve, 200)); + assert.equal((await get('/fixture')).held, 1); + await setMode({ mode: 'error' }); + assert.equal((await held).status, 'ERROR'); + assert.ok(Date.now() - start >= 190); + const state = await get('/fixture'); + assert.equal(state.held, 0); + assert.deepEqual(state.callbacks.map((c) => c.answer), ['invoice', 'error']); + assert.equal((await get('/fixture/invoices')).invoices.length, 1); + assert.equal((await get('/fixture/invoices')).invoices[0].settled, null); + assert.equal((await get('/channel/fixture')).status, 'ERROR'); +}); diff --git a/marketplace/driver/Dockerfile b/marketplace/driver/Dockerfile index 5ba7619..365d99c 100644 --- a/marketplace/driver/Dockerfile +++ b/marketplace/driver/Dockerfile @@ -8,4 +8,7 @@ RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund COPY --from=paykit /usr/local/bin/paykit-companion-auth /usr/local/bin/paykit-reader-demo /usr/local/bin/ # The helpers build their HTTP clients from the system CA store. COPY --from=paykit /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt +# Paykit Server rc65 accepts a setup claim only after the identity published its Paykit noise key authorization; the payment request +# fixture binary publishes it for the headless seller under this name (a Bitkit wallet does it in its own Paykit setup). +COPY --from=fixture /usr/local/bin/fixture /usr/local/bin/paykit-key-authorization COPY driver.mjs ./ diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index 2bd7f1f..1104add 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -8,7 +8,7 @@ // without a wallet app: the seller (watch-only setup) and the buyer (receive // and pay). Bitkit wallets take the buyer and seller roles in the app journey. // A Bitkit seller approves two Pubky grants: the Paykit setup (`setup-url`) and -// a `/pub/locks.app/` write grant for the marketplace (`seller-auth`), and the +// a `/pub/app.locks/` write grant for the marketplace (`seller-auth`), and the // driver publishes the payment lock with that grant session. // // Secrets stay in /state/secrets (root, 0700). Paykit Server only ever sees the @@ -43,7 +43,7 @@ const HEADLESS_CLIENT_ID = 'marketplace.fixture'; // The write grant a Bitkit seller approves for the marketplace (the role Locks plays). The apps show the // client id as "Requester ID" and the path as the requested permission. const LOCKS_CLIENT_ID = 'locks.app'; -const LOCKS_CAPS = '/pub/locks.app/:rw'; +const LOCKS_CAPS = '/pub/app.locks/:rw'; // The testnet's own HTTP relay; wallets reach it on localhost like the homeserver (Android: adb reverse 15412). const LOCKS_RELAY = 'http://localhost:15412/inbox/'; const BITKIT_SESSION_SECRET = 'bitkit-seller.session'; @@ -51,6 +51,7 @@ const BITKIT_SESSION_SECRET = 'bitkit-seller.session'; const STANDIN_ACCOUNT_INDEX = 1; const SERVER_PATH = 'bitkit/server'; const BUYER_PATH = 'bitkit/wallet'; +const BUYER_APP_ID = 'bitkit'; const ACCOUNT_INDEX = 0; const DEFAULT_SATS = 15000; const EXPECTED_ASSET = 'btc'; @@ -191,7 +192,9 @@ async function signedPost(path, body, { signature } = {}) { const text = canonical(body); const issuerSeed = Buffer.from(await readSecret('issuer.seed'), 'base64url'); const headers = { 'content-type': 'application/json' }; - const value = signature ?? b64url(sign(null, Buffer.from(text), keyFromSeed(issuerSeed))); + // Paykit Server signs requests over `paykit-http-signature-v1\0\0\0` (src/http/auth.rs, signature_preimage). + const preimage = Buffer.concat([Buffer.from(`paykit-http-signature-v1\0POST\0${path}\0`), Buffer.from(text)]); + const value = signature ?? b64url(sign(null, preimage, keyFromSeed(issuerSeed))); if (value !== 'none') headers['x-paykit-signature'] = value; const response = await fetch(`${PAYKIT_URL}${path}`, { method: 'POST', headers, body: text }); const raw = await response.text(); @@ -243,7 +246,7 @@ async function signUpIdentity(seed) { return keypair.publicKey.toString(); } -// A write session on the seller's /pub/locks.app/, for publishing the payment lock. The headless seller signs +// A write session on the seller's /pub/app.locks/, for publishing the payment lock. The headless seller signs // in with its own key. A Bitkit seller has no key here: the session is the grant its wallet approved in // `seller-auth`, restored from the state volume (each restore mints a fresh short-lived bearer). async function sellerSession(seller) { @@ -290,9 +293,8 @@ function runHelper(binary, input, env = {}) { const readerEnv = (seller) => ({ PAYKIT_READER_STATE_PATH: `${STATE}/reader/state.bin`, PAYKIT_READER_PUBKY_TESTNET_HOST: 'localhost', - PAYKIT_READER_RECEIVER_PATH: BUYER_PATH, + PAYKIT_READER_APP_ID: BUYER_APP_ID, PAYKIT_READER_SERVER_PUBKY: seller, - PAYKIT_READER_SERVER_PATH: SERVER_PATH, }); // ------------------------------------------------------------------ encodings @@ -345,8 +347,7 @@ allowed_origins = ["${SETUP_ORIGIN}"] [paykit] client_id = "${PAYKIT_CLIENT_ID}" -receiver_path = "${SERVER_PATH}" -receiver_path_priority = ["bitkit"] +app_id = "paykit-server" network = "testnet" [bitcoin] @@ -372,7 +373,7 @@ async function completeSetup(flowId, seconds = 120) { const response = await fetch(`${PAYKIT_URL}/setup/${flowId}/complete`, { method: 'POST' }); if (response.status === 200) return; if (![408, 425, 429, 502, 503, 504].includes(response.status)) { - fail(`setup flow ended with HTTP ${response.status}`); + fail(`setup flow ended with HTTP ${response.status}: ${(await response.text()).slice(0, 300)}`); } if (Date.now() > deadline) fail('setup flow did not complete in time'); await sleep(1500); @@ -409,12 +410,19 @@ async function beginSetup() { // The watch-only setup approval: the wallet's role in Paykit Server's /setup flow. The wallet gives the server // its account xpub with the companion claim, then approves the setup grant with its Pubky identity. async function approveSetupAs(authUrl, identitySeed, xpub, accountIndex) { + // Paykit Server rc65 checks the identity's Paykit noise key authorization before it accepts the claim; a wallet publishes it + // in its own Paykit setup, the headless identity publishes it here. + const authorized = await runHelper('paykit-key-authorization', { version: 1, creator_secret: b64url(identitySeed) }); + if (authorized.code !== 0 || !authorized.stdout.includes('"published":true')) { + fail(`Paykit key authorization failed: ${authorized.stderr || authorized.stdout}`); + } const approval = await runHelper('paykit-companion-auth', { version: 1, auth_url: authUrl, creator_secret: b64url(identitySeed), account_xpub: xpub, account_index: accountIndex, + key_generation: 1, }); if (approval.code !== 0 || !approval.stdout.includes('"approved"')) { fail(`companion approval failed: ${approval.stderr || approval.stdout}`); @@ -478,7 +486,7 @@ async function createBuyer(sellerPubky) { { version: 1, operation: 'prepare', reader_secret: b64url(seed) }, readerEnv(sellerPubky ?? fixture.seller.pubky), ); - if (prepared.code !== 0) fail(`buyer receiver marker failed: ${prepared.stdout || prepared.stderr}`); + if (prepared.code !== 0) fail(`buyer receiver marker failed: ${[prepared.stdout, prepared.stderr].filter(Boolean).join(' ')}`); fixture.buyer = { pubky: buyer, receiver_path: BUYER_PATH, kind: 'headless' }; await writeJson(FIXTURE_FILE, fixture); log(`headless buyer ready: ${buyer}`); @@ -719,14 +727,15 @@ async function purchase(args) { const lock = lockFor({ seller: seller.pubky, sats, issuer: await issuerPubky() }); const lockText = canonical(lock); const lockId = crockford(blake3(Buffer.from(lockText))); - const lockPath = `/pub/locks.app/${lockId}.json`; + const lockPath = `/pub/app.locks/${lockId}.json`; const session = await sellerSession(seller); await session.storage.putText(lockPath, lockText); const bundleId = newBundleId(); const lockResource = `${seller.pubky}${lockPath}`; const response = await signedPost('/invoices', { bundle_id: bundleId, lock_resource: lockResource, reader }); - if (response.status !== 204) { + // Paykit Server answers 200 with invoice_created_at and payment_deadline (204 with no body before the Locks payment window) + if (response.status !== 200 && response.status !== 204) { const code = response.json?.error?.code ? ` ${response.json.error.code}` : ''; const hint = response.status === 503 @@ -748,6 +757,7 @@ async function purchase(args) { derived_address: seller.kind === 'headless' ? await expectedAddress(seller.account_xpub, childIndex) : null, child_index: seller.kind === 'headless' ? childIndex : null, created_at: new Date().toISOString(), + payment_deadline: response.json?.payment_deadline ?? null, state: 'created', }; purchases.push(record); @@ -783,7 +793,7 @@ async function receive(args) { { version: 1, operation: 'receive', reader_secret: seed }, readerEnv(purchase.seller), ); - if (result.code !== 0) fail(`receive failed: ${result.stdout || result.stderr}`); + if (result.code !== 0) fail(`receive failed: ${[result.stdout, result.stderr].filter(Boolean).join(' ')}`); const request = JSON.parse(result.stdout); // The pinned reader rejects any endpoint other than btc-regtest-p2wpkh and any // payload that is not a JSON object with a string value before it projects. @@ -910,7 +920,7 @@ async function waitFor(args) { // It reads what the fixture can see: each side's public Paykit receiver marker, the seller's setup // authority, and Paykit Server's persisted peer state for a purchase's reader binding. async function receiverMarker(pubky, receiverPath) { - const path = `/pub/paykit/v0/${receiverPath}/receiver.json`; + const path = `/pub/paykit/v0/app-registry.json`; const storage = pubkyClient().publicStorage; if (!(await storage.exists(`${pubky}${path}`))) return { path, present: false }; return { path, present: true, marker: await storage.getJson(`${pubky}${path}`) }; @@ -1056,7 +1066,8 @@ async function verify() { evidence.seller = { pubky: fixture.seller.pubky, account_xpub: fixture.seller.account_xpub, account_index: fixture.seller.account_index }; evidence.buyer = { pubky: buyer.pubky }; - const peersBefore = await capture('peers_before', () => peers([])); + // ask about this run's buyer: without --buyer the report follows the latest purchase, which an earlier verify left + const peersBefore = await capture('peers_before', () => peers(['--buyer', buyer.pubky])); if (!peersBefore.ready_for_purchase || peersBefore.linked) fail('before the purchase the peers must be ready for a purchase and not linked yet'); const created = await capture('purchase', () => purchase(['--buyer', 'headless'])); @@ -1098,7 +1109,7 @@ async function verify() { const androidOpen = (authUrl, serial) => `adb${serial ? ` -s ${serial}` : ''} shell "am start -a android.intent.action.VIEW -d '${authUrl}'"`; -// The marketplace's request for a write grant on the seller's /pub/locks.app/, shown to the seller as a Pubky +// The marketplace's request for a write grant on the seller's /pub/app.locks/, shown to the seller as a Pubky // auth request (the role Locks plays). The flow polls the relay as long as this process runs. async function startLocksGrant(relay) { const flow = await pubkyClient().startGrantAuthFlow(LOCKS_CAPS, AuthFlowKind.signin(), { clientId: LOCKS_CLIENT_ID, relay }); @@ -1123,7 +1134,7 @@ async function awaitGrant(flow, seconds) { const writesLocks = (capabilities) => capabilities.some((cap) => { const at = cap.lastIndexOf(':'); - return cap.slice(at + 1).includes('w') && '/pub/locks.app/'.startsWith(cap.slice(0, at)); + return cap.slice(at + 1).includes('w') && '/pub/app.locks/'.startsWith(cap.slice(0, at)); }); // Keep the approved grant as the seller's write session and record the identity that approved it. diff --git a/marketplace/driver/package-lock.json b/marketplace/driver/package-lock.json index f91044f..079a607 100644 --- a/marketplace/driver/package-lock.json +++ b/marketplace/driver/package-lock.json @@ -10,7 +10,7 @@ "dependencies": { "@noble/hashes": "1.8.0", "@scure/bip32": "1.7.0", - "@synonymdev/pubky": "0.10.0" + "@synonymdev/pubky": "0.14.0" } }, "node_modules/@noble/curves": { @@ -64,9 +64,9 @@ } }, "node_modules/@synonymdev/pubky": { - "version": "0.10.0", - "resolved": "https://registry.npmjs.org/@synonymdev/pubky/-/pubky-0.10.0.tgz", - "integrity": "sha512-XlyTQ0yYo/3Jk/M3Xw1x4gMskK2SeAG0n0aO54BZyNQq774bwMBX8W/vYzARKVs1WlNY/RdJbANlZeT2YQePgg==", + "version": "0.14.0", + "resolved": "https://registry.npmjs.org/@synonymdev/pubky/-/pubky-0.14.0.tgz", + "integrity": "sha512-BK+Rn49eah26+a4aTywmMvNcx/e+CE8oqdHbAB2N0onuqTLJ0SoM+JDP9qurYB4i0XuT0haZQyCRlDlMiArfEQ==", "license": "MIT", "dependencies": { "fetch-cookie": "^3.0.1" @@ -89,27 +89,27 @@ "license": "MIT" }, "node_modules/tldts": { - "version": "7.4.4", - "resolved": "https://registry.npmjs.org/tldts/-/tldts-7.4.4.tgz", - "integrity": "sha512-kFXFK7O4WPextIUAOk8qtnw9dxR9UIXP9CjuH1cTBVBZMDeQcUPgr/IazGiw1B0Yiw5L75gHLWeW4iD793r90g==", + "version": "7.4.16", + "resolved": "https://registry.npmjs.org/tldts/-/tldts-7.4.16.tgz", + "integrity": "sha512-QwBER5KMR86IIjpIiO7H/Z3IMJPsZ1A6RKPAqzTTgOyUQUSt9FdnKcqhTaJmkY6HVrgouZHZR0ncK5QxvmnQeg==", "license": "MIT", "dependencies": { - "tldts-core": "^7.4.4" + "tldts-core": "^7.4.16" }, "bin": { "tldts": "bin/cli.js" } }, "node_modules/tldts-core": { - "version": "7.4.4", - "resolved": "https://registry.npmjs.org/tldts-core/-/tldts-core-7.4.4.tgz", - "integrity": "sha512-vwVLJVvvpslm7vqAH7+XNj/neA/Ynq7DT2EEcMuwc5YzN5XaMyRAqxwU+uX3azZ1FQtB2gvrvnLnAEkvYlVdfg==", + "version": "7.4.16", + "resolved": "https://registry.npmjs.org/tldts-core/-/tldts-core-7.4.16.tgz", + "integrity": "sha512-MDolfaSJtlSK5Y0A1xl3277ekubZwobpBjugknDizI9O5Rm60a1m8k4ICK+MRsCDzPygT81mp3BBf5RKDlFRfA==", "license": "MIT" }, "node_modules/tough-cookie": { - "version": "6.0.1", - "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-6.0.1.tgz", - "integrity": "sha512-LktZQb3IeoUWB9lqR5EWTHgW/VTITCXg4D21M+lvybRVdylLrRMnqaIONLVb5mav8vM19m44HIcGq4qASeu2Qw==", + "version": "6.0.2", + "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-6.0.2.tgz", + "integrity": "sha512-exgYmnmL/sJpR3upZfXG5PoatXQii55xAiXGXzY+sROLZ/Y+SLcp9PgJNI9Vz37HpQ74WvDcLT8eqm+kV3FzrA==", "license": "BSD-3-Clause", "dependencies": { "tldts": "^7.0.5" diff --git a/marketplace/driver/package.json b/marketplace/driver/package.json index 587c8ba..f821140 100644 --- a/marketplace/driver/package.json +++ b/marketplace/driver/package.json @@ -7,6 +7,6 @@ "dependencies": { "@noble/hashes": "1.8.0", "@scure/bip32": "1.7.0", - "@synonymdev/pubky": "0.10.0" + "@synonymdev/pubky": "0.14.0" } } diff --git a/marketplace/patches/paykit-server-reader-accepts-proposal-expiry.patch b/marketplace/patches/paykit-server-reader-accepts-proposal-expiry.patch new file mode 100644 index 0000000..35de89e --- /dev/null +++ b/marketplace/patches/paykit-server-reader-accepts-proposal-expiry.patch @@ -0,0 +1,12 @@ +diff --git a/paykit-server/src/bin/paykit-reader-demo/payment_instructions.rs b/paykit-server/src/bin/paykit-reader-demo/payment_instructions.rs +index f862b81..01e22b4 100644 +--- a/paykit-server/src/bin/paykit-reader-demo/payment_instructions.rs ++++ b/paykit-server/src/bin/paykit-reader-demo/payment_instructions.rs +@@ -102,7 +102,6 @@ pub(super) fn payment_instructions( + if terms.amount.asset != "btc" + || terms.required_app_id.as_ref().map(|id| id.as_str()) != Some(PAYKIT_APP_ID) + || terms.recurrence.is_some() +- || terms.proposal_expires_at.is_some() + || terms.accepted_payment_endpoint_identifiers != [BITCOIN_ENDPOINT.to_owned()] + || terms + .metadata diff --git a/marketplace/pubky-testnet/Dockerfile b/marketplace/pubky-testnet/Dockerfile index 63a4ab7..307cd8c 100644 --- a/marketplace/pubky-testnet/Dockerfile +++ b/marketplace/pubky-testnet/Dockerfile @@ -1,22 +1,17 @@ -# Pubky Core static testnet (DHT, PKARR relay, HTTP relay, homeserver) for the -# marketplace fixture. Same build as pubky/locks docker/pubky-testnet.Dockerfile -# at ba49a777, pinned to the Pubky Core revision that Bitkit's local testnet -# client and Paykit Server 722ef268 (Pubky 0.11 grant auth) are exercised against. -FROM rust:1.89.0-bookworm AS builder +# Pubky static testnet (DHT, PKARR relay, HTTP relay, homeserver) for the +# marketplace fixture, built from the published `pubky-testnet` crate with its +# own lockfile. The homeserver of this release answers the WebDAV-style `LOCK` +# and `UNLOCK` requests of the Pubky SDK's write locks (the Paykit SDK takes +# one before it writes shared state); the earlier Pubky Core pin answered 405. +FROM rust:1.98.1-bookworm AS builder -ARG PUBKY_CORE_REV=f68014c111af0458e6a321e2d87a12479bfb3218 -WORKDIR /usr/src/pubky-core -RUN git clone --filter=blob:none https://github.com/pubky/pubky-core.git . \ - && git checkout --detach "${PUBKY_CORE_REV}" -# The pinned Pubky Core revision locks quinn-proto 0.11.14, affected by -# RUSTSEC-2026-0185. Keep this precise override until its lockfile advances. -RUN cargo update -p quinn-proto --precise 0.11.15 \ - && cargo build --release -p pubky-testnet --bin pubky-testnet +ARG PUBKY_TESTNET_VERSION=0.14.0 +RUN cargo install pubky-testnet --version "${PUBKY_TESTNET_VERSION}" --locked --root /opt/pubky FROM debian:bookworm-slim RUN apt-get update \ && apt-get install -y --no-install-recommends ca-certificates \ && rm -rf /var/lib/apt/lists/* -COPY --from=builder /usr/src/pubky-core/target/release/pubky-testnet /usr/local/bin/pubky-testnet +COPY --from=builder /opt/pubky/bin/pubky-testnet /usr/local/bin/pubky-testnet EXPOSE 6881 15411 15412 6286 6287 6288 CMD ["pubky-testnet"] diff --git a/payment-requests/Cargo.lock b/payment-requests/Cargo.lock index 1e35778..c434695 100644 --- a/payment-requests/Cargo.lock +++ b/payment-requests/Cargo.lock @@ -63,13 +63,13 @@ checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" [[package]] name = "argon2" -version = "0.5.3" +version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3c3610892ee6e0cbce8ae2700349fcf8f98adb0dbfbee85aec3c9179d29cc072" +checksum = "134c52ddac6d63c576bef8168db10c83c49c26444ecbc68060fef078925a901c" dependencies = [ "base64ct", - "blake2", - "cpufeatures 0.2.17", + "blake2 0.11.0", + "cpufeatures 0.3.1", "password-hash", ] @@ -266,14 +266,17 @@ dependencies = [ "anyhow", "async-trait", "axum", + "base64 0.22.1", "chrono", "paykit-lib", "paykit-sdk", + "pkarr", "pubky", "reqwest", "serde", "serde_json", "tokio", + "tokio-rustls", ] [[package]] @@ -285,6 +288,15 @@ dependencies = [ "digest 0.10.7", ] +[[package]] +name = "blake2" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5b5d4d889834ee8ecfc0f8426ad30faf7cdcb10f741a8e6d7224d95325479f6f" +dependencies = [ + "digest 0.11.3", +] + [[package]] name = "blake3" version = "1.8.7" @@ -336,9 +348,9 @@ checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" [[package]] name = "cc" -version = "1.5.1" +version = "1.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f360145194ee8e21db5ee7f3fcd4fe52210864c75c985dae33218202c8bbe040" +checksum = "f74872d07caf508b30a21f6836e7d7016a2eaf7d9ff4f48deaa58cd8a0407630" dependencies = [ "find-msvc-tools", "jobserver", @@ -427,6 +439,12 @@ dependencies = [ "cc", ] +[[package]] +name = "cmov" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a" + [[package]] name = "cobs" version = "0.3.0" @@ -611,6 +629,15 @@ dependencies = [ "cipher", ] +[[package]] +name = "ctutils" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e" +dependencies = [ + "cmov", +] + [[package]] name = "curve25519-dalek" version = "4.1.3" @@ -694,6 +721,7 @@ dependencies = [ "block-buffer 0.12.1", "const-oid", "crypto-common 0.2.2", + "ctutils", ] [[package]] @@ -785,17 +813,6 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "eventsource-stream" -version = "0.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "74fef4569247a5f429d9156b9d0a2599914385dd189c539334c625d8099d90ab" -dependencies = [ - "futures-core", - "nom", - "pin-project-lite", -] - [[package]] name = "fastrand" version = "2.5.0" @@ -820,17 +837,6 @@ version = "0.1.14" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "aedcfb3409746eddb02b9e19ebda1c3394f759a152e48ee875a0844d1b955484" -[[package]] -name = "flume" -version = "0.11.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da0e4dd2a88388a1f4ccc7c9ce104604dab68d9f408dc34cd45823d5a9069095" -dependencies = [ - "futures-core", - "futures-sink", - "spin 0.9.9", -] - [[package]] name = "flume" version = "0.12.0" @@ -1441,15 +1447,15 @@ dependencies = [ [[package]] name = "lazy_static" -version = "1.5.0" +version = "1.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" +checksum = "20870f649af7073d53e38067b2a84312175d56ea15217e1b15bc83506ec50afb" [[package]] name = "libc" -version = "0.2.189" +version = "0.2.190" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" +checksum = "ce5d3ddc6d3fa000eb1536d85e147bfe31aacaba692ed6a876f95cb7c855be78" [[package]] name = "litemap" @@ -1502,12 +1508,6 @@ dependencies = [ "tracing-subscriber", ] -[[package]] -name = "lru" -version = "0.16.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f66e8d5d03f609abc3a39e6f08e4164ebf1447a732906d39eb9b99b7919ef39" - [[package]] name = "lru" version = "0.18.5" @@ -1522,18 +1522,18 @@ checksum = "4050469837a6ff301cd14c1f8f24f88549e6d548f24f64e2148eb0f72cebc51f" [[package]] name = "mainline" -version = "8.0.0" +version = "8.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d32eaee3dcba6e0bbbefe8bd896a8bd6039d5e74b199c0fe248e9feb547c2a26" +checksum = "43e632c9dd114af78fdccb1169f01b89ccc10590df11e55f07cc56c857537de6" dependencies = [ "crc", "document-features", "dyn-clone", "ed25519-dalek", - "flume 0.12.0", + "flume", "futures-lite", "getrandom 0.4.3", - "lru 0.16.4", + "lru", "serde", "serde_bencode", "serde_bytes", @@ -1569,33 +1569,17 @@ version = "0.3.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a" -[[package]] -name = "minimal-lexical" -version = "0.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" - [[package]] name = "mio" -version = "1.2.3" +version = "1.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4b18443e9c262bfe8fa82f51666e2642c53393f7e5c27b3e1aeab922cff5b9d8" +checksum = "1788edb87fdc09c7e26304471e2f5be8cdefb1b6930d6e3985fc02ff53bf86ee" dependencies = [ "libc", "wasi", "windows-sys 0.61.2", ] -[[package]] -name = "nom" -version = "7.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a" -dependencies = [ - "memchr", - "minimal-lexical", -] - [[package]] name = "ntimestamp" version = "1.0.0" @@ -1694,30 +1678,32 @@ dependencies = [ [[package]] name = "password-hash" -version = "0.5.0" +version = "0.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "346f04948ba92c43e8469c1ee6736c7563d71012b17d40745260fe106aac2166" +checksum = "aab41826031698d6ffcd9cff78ef56ef998e39dc7e5067cdfebe373842d4723b" dependencies = [ - "base64ct", - "rand_core 0.6.4", - "subtle", + "getrandom 0.4.3", + "phc", ] [[package]] name = "paykit-lib" -version = "0.1.0-rc56" -source = "git+https://github.com/pubky/paykit-rs.git?rev=24162ebbcc703251d8038f2e176d1f4cfb117c6a#24162ebbcc703251d8038f2e176d1f4cfb117c6a" +version = "0.1.0-rc65" +source = "git+https://github.com/pubky/paykit-rs.git?rev=7185ae7da9315028e5331442d71d739c27f1442c#7185ae7da9315028e5331442d71d739c27f1442c" dependencies = [ "anyhow", "base64 0.22.1", + "blake3", "chacha20poly1305", "chrono", + "hex", "pubky", "pubky-noise", "reqwest", "serde", "serde_json", "thiserror", + "tokio", "tracing", "uuid", "zeroize", @@ -1725,24 +1711,30 @@ dependencies = [ [[package]] name = "paykit-sdk" -version = "0.1.0-rc56" -source = "git+https://github.com/pubky/paykit-rs.git?rev=24162ebbcc703251d8038f2e176d1f4cfb117c6a#24162ebbcc703251d8038f2e176d1f4cfb117c6a" +version = "0.1.0-rc65" +source = "git+https://github.com/pubky/paykit-rs.git?rev=7185ae7da9315028e5331442d71d739c27f1442c#7185ae7da9315028e5331442d71d739c27f1442c" dependencies = [ "anyhow", "async-trait", "base64 0.22.1", "bip39", "blake3", + "chacha20poly1305", "chrono", "crypto_secretbox", + "futures-util", "hex", "paykit-lib", + "postcard", "pubky", + "pubky-noise", "reqwest", "serde", "serde_json", "sha2 0.10.9", "thiserror", + "tokio", + "tracing", "url", "zeroize", ] @@ -1753,6 +1745,17 @@ version = "2.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" +[[package]] +name = "phc" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "44dc769b75f93afdddd8c7fa12d685292ddeff1e66f7f0f3a234cf1818afe892" +dependencies = [ + "base64ct", + "ctutils", + "getrandom 0.4.3", +] + [[package]] name = "phf" version = "0.11.3" @@ -1803,9 +1806,9 @@ checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" [[package]] name = "pkarr" -version = "8.0.2" +version = "8.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "26c043ed867b3152b7b0eb3e0121048c522bfe2e0e8368c7a5e91c042c78a2a5" +checksum = "0fa6e085dac5eb00b309bfafd0fa0db088a7d3aaad4b5de75db4732e7c7f7595" dependencies = [ "async-compat", "base32", @@ -1821,7 +1824,7 @@ dependencies = [ "getrandom 0.4.3", "heed", "log", - "lru 0.18.5", + "lru", "mainline", "ntimestamp", "page_size", @@ -1901,9 +1904,9 @@ dependencies = [ [[package]] name = "powerfmt" -version = "0.2.0" +version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" +checksum = "4a6394b9e965e73d0a289ee54f589087e2c676aedf60885baf52c76b771e4958" [[package]] name = "ppv-lite86" @@ -1931,19 +1934,19 @@ checksum = "33cb294fe86a74cbcf50d4445b37da762029549ebeea341421c7c70370f86cac" [[package]] name = "pubky" -version = "0.11.0" +version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "15a7b191157d57d5095f1577c8064aeda85f96606957fcf4c0a6f4b675adb96d" +checksum = "2099d521cacff9a1bf2ad48978d4601608d80860d7b08f7ecb0ced98bb63b628" dependencies = [ "async-trait", - "base64 0.22.1", + "base64 0.23.1", "cookie", - "eventsource-stream", - "flume 0.11.1", + "flume", "futures-lite", "futures-util", "httpdate", "log", + "lru", "percent-encoding", "pkarr", "pubky-common", @@ -1951,6 +1954,7 @@ dependencies = [ "rustls", "serde", "serde_json", + "sse-core", "thiserror", "tokio", "tracing", @@ -1962,12 +1966,12 @@ dependencies = [ [[package]] name = "pubky-common" -version = "0.11.0" +version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6ee26f4cf7d9d11a300180b351f1ea0efebf17ec66f0e615a8867f8aba47dc5f" +checksum = "e9b32c46e101fa9b94fcb3f3baa5751d3f696b3823508697a4f407281cbe9c4c" dependencies = [ "argon2", - "base64 0.22.1", + "base64 0.23.1", "blake3", "crypto_secretbox", "ed25519-dalek", @@ -1984,16 +1988,18 @@ dependencies = [ [[package]] name = "pubky-noise" -version = "0.1.0-rc8" +version = "0.1.0-rc12" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d506939a2d151814267bc94a1e70f6406d98bda0322da33a014a249edd307eea" +checksum = "c034fc7c278e85960a566a032ee26acf9d9c00ee6f64d74fa1f73ee9ed7d4f06" dependencies = [ + "chacha20poly1305", "curve25519-dalek 5.0.0", "ed25519-dalek", "getrandom 0.3.4", "hex", "pubky", "rand 0.9.5", + "reqwest", "sha2 0.11.0", "snow", ] @@ -2045,9 +2051,9 @@ dependencies = [ [[package]] name = "quinn-proto" -version = "0.11.18" +version = "0.11.19" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a9746dbde176634f4f2f1faf2404e30a31b2bc1e9cafb5329c95d8177a18c9fc" +checksum = "0e750cca55fe4f0439a15d0bb529da9651e79993e8e72c61a899a36d462befbe" dependencies = [ "aws-lc-rs", "bytes", @@ -2068,9 +2074,9 @@ dependencies = [ [[package]] name = "quinn-udp" -version = "0.5.15" +version = "0.5.16" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "35a133f956daabe89a61a685c2649f13d82d5aa4bd5d12d1277e1072a21c0694" +checksum = "af66907df18639dcf4db56ca65490cabc4b27a97dbadd96f2926cca73298f016" dependencies = [ "cfg_aliases", "libc", @@ -2639,7 +2645,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "599b506ccc4aff8cf7844bc42cf783009a434c1e26c964432560fb6d6ad02d82" dependencies = [ "aes-gcm", - "blake2", + "blake2 0.10.6", "chacha20poly1305", "curve25519-dalek 4.1.3", "getrandom 0.3.4", @@ -2684,6 +2690,19 @@ dependencies = [ "der", ] +[[package]] +name = "sse-core" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28f4c044039e70ba5724b7e9dacf9fb610394773d15df2bb629170ca30391c88" +dependencies = [ + "bytes", + "futures-core", + "memchr", + "pin-project-lite", + "thiserror", +] + [[package]] name = "stable_deref_trait" version = "1.2.1" @@ -2824,9 +2843,9 @@ checksum = "fd3ca314f692efd6c868f8408f53fe444634a845f96c028b97d35f6a1f79f0ee" [[package]] name = "tokio" -version = "1.53.1" +version = "1.53.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed" +checksum = "e95f91fcc7a621e8b030f6aa23c71fe9838ae2fb4d8118b75602a328f5144044" dependencies = [ "bytes", "libc", @@ -3044,9 +3063,9 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" [[package]] name = "uuid" -version = "1.26.1" +version = "1.27.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2ef6dac1e96601b4fb3acccccff2139741fcb757cb9a36089bf5be91cfb285ce" +checksum = "97277d36b9c3ace13e58fa6e753f8b0bbbf302a18dd193240d70f9e29681059a" dependencies = [ "getrandom 0.4.3", "js-sys", @@ -3403,9 +3422,9 @@ dependencies = [ [[package]] name = "yoke-derive" -version = "0.8.3" +version = "0.8.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78" +checksum = "ec8ebde2db3681e8c9980cc27822030e68752690ddfa9473e739aeb4dbde6d71" dependencies = [ "proc-macro2", "quote", diff --git a/payment-requests/Cargo.toml b/payment-requests/Cargo.toml index 1b5d38e..ba5d31e 100644 --- a/payment-requests/Cargo.toml +++ b/payment-requests/Cargo.toml @@ -7,11 +7,14 @@ edition = "2021" anyhow = "1" async-trait = "0.1" axum = "0.8" +base64 = "0.22" chrono = "0.4" -paykit-lib = { git = "https://github.com/pubky/paykit-rs.git", rev = "24162ebbcc703251d8038f2e176d1f4cfb117c6a" } -paykit-sdk = { git = "https://github.com/pubky/paykit-rs.git", rev = "24162ebbcc703251d8038f2e176d1f4cfb117c6a" } -pubky = "=0.11.0" -reqwest = { version = "0.13", default-features = false, features = ["json", "rustls"] } +paykit-lib = { git = "https://github.com/pubky/paykit-rs.git", rev = "7185ae7da9315028e5331442d71d739c27f1442c" } +paykit-sdk = { git = "https://github.com/pubky/paykit-rs.git", rev = "7185ae7da9315028e5331442d71d739c27f1442c" } +pubky = "=0.14.0" +pkarr = { version = "8.1", features = ["tls"] } +reqwest = { version = "0.13", default-features = false, features = ["json", "rustls", "stream"] } serde = { version = "1", features = ["derive"] } serde_json = "1" tokio = { version = "1", features = ["macros", "rt-multi-thread", "net", "sync", "time"] } +tokio-rustls = { version = "0.26", default-features = false, features = ["ring"] } diff --git a/payment-requests/Dockerfile b/payment-requests/Dockerfile index 925f781..4752cc0 100644 --- a/payment-requests/Dockerfile +++ b/payment-requests/Dockerfile @@ -2,13 +2,23 @@ FROM rust:1.98.1-bookworm@sha256:93ce27a88655056a51dbdd8f5f2d7ddc071c7b0070fb288 WORKDIR /usr/src/fixture COPY Cargo.toml Cargo.lock ./ COPY src ./src +# scripts/follow-app-paykit passes the paykit-rs commit an app pins; empty keeps the committed pin +ARG PAYKIT_RS_REV= +RUN if [ -n "$PAYKIT_RS_REV" ]; then \ + sed -i -E "s|(paykit-rs.git\", rev = \")[0-9a-f]+|\1$PAYKIT_RS_REV|" Cargo.toml \ + && cargo update -p paykit-lib -p paykit-sdk; \ + fi RUN --mount=type=cache,target=/usr/local/cargo/registry \ --mount=type=cache,target=/usr/local/cargo/git \ --mount=type=cache,target=/usr/src/fixture/target \ - cargo build --release --locked && install -Dm755 target/release/bitkit-payment-request-fixture /out/fixture + cargo build --release --locked && install -Dm755 target/release/bitkit-payment-request-fixture /out/fixture \ + && install -Dm755 target/release/homeserver-proxy /out/homeserver-proxy \ + && sed -n 's|^source = "git+https://github.com/pubky/paykit-rs.git?rev=[0-9a-f]*#\([0-9a-f]*\)"|\1|p' Cargo.lock | head -1 > /out/paykit-rs-rev FROM debian:bookworm-slim@sha256:3783cc01769c7b2b1b83a5c5ad96c815348e28ed7da68e2e3687004faa906251 RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \ && rm -rf /var/lib/apt/lists/* COPY --from=builder /out/fixture /usr/local/bin/fixture +COPY --from=builder /out/homeserver-proxy /usr/local/bin/homeserver-proxy +COPY --from=builder /out/paykit-rs-rev /usr/local/share/paykit-rs-rev ENTRYPOINT ["/usr/local/bin/fixture"] diff --git a/payment-requests/prepare b/payment-requests/prepare index b9813e1..90943e2 100755 --- a/payment-requests/prepare +++ b/payment-requests/prepare @@ -93,8 +93,8 @@ sync_issuer canceled=$(issue rc56-canceled 13000) receive -post 3012 /cancel "$(jq -nc --argjson peer "$to_peer" --arg id "$canceled" '$peer + {payment_request_id:$id}')" >/dev/null -receive +post 3013 /cancel "$(record "$canceled")" >/dev/null +sync_issuer completed=$(issue rc56-completed 14000) receive diff --git a/payment-requests/src/bin/homeserver-proxy.rs b/payment-requests/src/bin/homeserver-proxy.rs new file mode 100644 index 0000000..f69ec8b --- /dev/null +++ b/payment-requests/src/bin/homeserver-proxy.rs @@ -0,0 +1,287 @@ +//! Pubky TLS proxy in front of the local testnet homeserver, with per-identity delay and failure rules. +//! The app reaches the homeserver through it (Compose publishes the host's 6287 here); without rules it only forwards. +//! The static testnet's homeserver key comes from the all-zero secret, so this proxy can present that same key. + +use std::{ + collections::VecDeque, + env, + net::SocketAddr, + sync::Arc, + time::{Duration, SystemTime, UNIX_EPOCH}, +}; + +use anyhow::{Context, Result}; +use axum::{ + body::Body, + extract::{Request, State}, + http::{header, HeaderMap, HeaderName, StatusCode}, + response::{IntoResponse, Response}, + routing::get, + serve::Listener, + Json, Router, +}; +use pkarr::Keypair; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; +use tokio::{ + net::{TcpListener, TcpStream}, + sync::{mpsc, Mutex}, +}; +use tokio_rustls::{server::TlsStream, TlsAcceptor}; + +#[derive(Clone, Serialize, Deserialize)] +struct Rule { + // the identity whose requests the rule holds (z32, with or without the `pubky` prefix) + pubky: String, + // owner-relative path prefix, such as `/pub/pubky.app/profile.json`; empty matches every path + #[serde(default)] + path: String, + #[serde(default)] + delay_ms: u64, + // answer this status after the delay instead of forwarding + #[serde(default)] + status: Option, +} + +#[derive(Serialize)] +struct Seen { + at: u64, + method: String, + owner: Option, + path: String, + status: u16, + delayed_ms: u64, +} + +struct Proxy { + upstream: String, + client: reqwest::Client, + rules: Mutex>, + seen: Mutex>, +} + +fn normalize(pubky: &str) -> String { + let pubky = pubky.trim(); + match pubky.strip_prefix("pubky") { + Some(rest) if rest.len() == 52 => rest.to_string(), + _ => pubky.to_string(), + } +} + +/// The owner and owner-relative path of a homeserver request, read the way the homeserver reads them: +/// `/storage//`, else the `pubky-host` header, else a `_pubky.` or `` host. +fn tenant(headers: &HeaderMap, uri: &axum::http::Uri) -> (Option, String) { + let path = uri.path(); + if let Some((owner, rest)) = path.strip_prefix("/storage/").and_then(|rest| rest.split_once('/')) { + return (Some(owner.to_string()), format!("/{rest}")); + } + if let Some(owner) = headers.get("pubky-host").and_then(|value| value.to_str().ok()) { + return (Some(normalize(owner)), path.to_string()); + } + let host = headers + .get(header::HOST) + .and_then(|value| value.to_str().ok()) + .or_else(|| uri.host()) + .unwrap_or_default(); + let host = host.split(':').next().unwrap_or_default(); + let host = host.strip_prefix("_pubky.").unwrap_or(host); + let owner = (host.len() == 52).then(|| host.to_string()); + (owner, path.to_string()) +} + +const HOP_BY_HOP: [&str; 8] = [ + "connection", + "keep-alive", + "proxy-authenticate", + "proxy-authorization", + "te", + "trailer", + "transfer-encoding", + "upgrade", +]; + +fn copy_headers(from: &HeaderMap) -> HeaderMap { + let mut to = HeaderMap::new(); + for (name, value) in from { + if !HOP_BY_HOP.contains(&name.as_str()) { + to.append(HeaderName::from(name), value.clone()); + } + } + to +} + +async fn forward(State(proxy): State>, request: Request) -> Response { + let (owner, path) = tenant(request.headers(), request.uri()); + let rule = match owner.as_ref() { + Some(owner) => proxy + .rules + .lock() + .await + .iter() + .find(|rule| &rule.pubky == owner && path.starts_with(&rule.path)) + .cloned(), + None => None, + }; + let delayed_ms = rule.as_ref().map_or(0, |rule| rule.delay_ms); + if delayed_ms > 0 { + tokio::time::sleep(Duration::from_millis(delayed_ms)).await; + } + let method = request.method().to_string(); + let response = match rule.as_ref().and_then(|rule| rule.status) { + Some(status) => ( + StatusCode::from_u16(status).unwrap_or(StatusCode::SERVICE_UNAVAILABLE), + "homeserver-proxy: injected failure", + ) + .into_response(), + None => relay(&proxy, request).await, + }; + let status = response.status().as_u16(); + println!( + "{method} {} {path} -> {status}{}", + owner.as_deref().unwrap_or("-"), + if delayed_ms > 0 { format!(" (delayed {delayed_ms} ms)") } else { String::new() } + ); + let mut seen = proxy.seen.lock().await; + if seen.len() == 200 { + seen.pop_front(); + } + seen.push_back(Seen { + at: SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |now| now.as_secs()), + method, + owner, + path, + status, + delayed_ms, + }); + response +} + +async fn relay(proxy: &Proxy, request: Request) -> Response { + let target = format!( + "{}{}", + proxy.upstream, + request.uri().path_and_query().map_or("/", |value| value.as_str()) + ); + let method = request.method().clone(); + let headers = copy_headers(request.headers()); + let body = reqwest::Body::wrap_stream(request.into_body().into_data_stream()); + match proxy.client.request(method, target).headers(headers).body(body).send().await { + Ok(upstream) => { + let mut response = Response::builder().status(upstream.status().as_u16()); + if let Some(headers) = response.headers_mut() { + *headers = copy_headers(upstream.headers()); + } + response + .body(Body::from_stream(upstream.bytes_stream())) + .unwrap_or_else(|error| (StatusCode::BAD_GATEWAY, error.to_string()).into_response()) + } + Err(error) => (StatusCode::BAD_GATEWAY, format!("homeserver-proxy: {error}")).into_response(), + } +} + +async fn list_rules(State(proxy): State>) -> Json { + Json(json!({ "rules": *proxy.rules.lock().await })) +} + +/// Adds a rule, replacing one with the same identity and path. +async fn add_rule(State(proxy): State>, Json(mut rule): Json) -> Json { + rule.pubky = normalize(&rule.pubky); + let mut rules = proxy.rules.lock().await; + rules.retain(|known| !(known.pubky == rule.pubky && known.path == rule.path)); + rules.push(rule); + Json(json!({ "rules": *rules })) +} + +async fn clear_rules(State(proxy): State>) -> Json { + proxy.rules.lock().await.clear(); + Json(json!({ "rules": [] })) +} + +async fn requests(State(proxy): State>) -> Json { + Json(json!({ "requests": *proxy.seen.lock().await })) +} + +/// Accepts TCP connections and finishes each TLS handshake on its own task, so one slow client does not hold the others. +struct TlsListener { + local: SocketAddr, + ready: mpsc::Receiver<(TlsStream, SocketAddr)>, +} + +impl TlsListener { + async fn bind(addr: SocketAddr, acceptor: TlsAcceptor) -> Result { + let listener = TcpListener::bind(addr).await?; + let local = listener.local_addr()?; + let (sender, ready) = mpsc::channel(64); + tokio::spawn(async move { + loop { + let Ok((stream, peer)) = listener.accept().await else { continue }; + let (acceptor, sender) = (acceptor.clone(), sender.clone()); + tokio::spawn(async move { + match tokio::time::timeout(Duration::from_secs(10), acceptor.accept(stream)).await { + Ok(Ok(tls)) => { + let _ = sender.send((tls, peer)).await; + } + // the Pubky client's reachability probe connects and closes without a handshake + Ok(Err(error)) if error.kind() == std::io::ErrorKind::UnexpectedEof => {} + Ok(Err(error)) => eprintln!("tls handshake from {peer} failed: {error}"), + Err(_) => eprintln!("tls handshake from {peer} timed out"), + } + }); + } + }); + Ok(Self { local, ready }) + } +} + +impl Listener for TlsListener { + type Io = TlsStream; + type Addr = SocketAddr; + + async fn accept(&mut self) -> (Self::Io, Self::Addr) { + loop { + if let Some(next) = self.ready.recv().await { + return next; + } + } + } + + fn local_addr(&self) -> std::io::Result { + Ok(self.local) + } +} + +fn port(name: &str, default: u16) -> Result { + env::var(name).map_or(Ok(default), |value| value.parse().with_context(|| format!("{name} is not a port"))) +} + +#[tokio::main] +async fn main() -> Result<()> { + let keypair = Keypair::from_secret_key(&[0; 32]); + let tls_port = port("PROXY_TLS_PORT", 6297)?; + let control_port = port("PROXY_CONTROL_PORT", 6298)?; + let upstream = env::var("PROXY_UPSTREAM").unwrap_or_else(|_| "http://127.0.0.1:6286".into()); + let proxy = Arc::new(Proxy { + upstream, + client: reqwest::Client::builder() + .redirect(reqwest::redirect::Policy::none()) + .build()?, + rules: Mutex::new(Vec::new()), + seen: Mutex::new(VecDeque::new()), + }); + let acceptor = TlsAcceptor::from(Arc::new(keypair.to_rpk_rustls_server_config())); + let tls = TlsListener::bind(([0, 0, 0, 0], tls_port).into(), acceptor).await?; + let control = TcpListener::bind(("0.0.0.0", control_port)).await?; + println!( + "homeserver-proxy: Pubky TLS as {} on {tls_port} -> {}, control on {control_port}", + keypair.public_key(), + proxy.upstream + ); + let forwarding = Router::new().fallback(forward).with_state(proxy.clone()); + let controls = Router::new() + .route("/health", get(|| async { "ok" })) + .route("/rules", get(list_rules).post(add_rule).delete(clear_rules)) + .route("/requests", get(requests)) + .with_state(proxy); + tokio::try_join!(async { axum::serve(tls, forwarding).await }, async { axum::serve(control, controls).await })?; + Ok(()) +} diff --git a/payment-requests/src/main.rs b/payment-requests/src/main.rs index 9a87693..7503382 100644 --- a/payment-requests/src/main.rs +++ b/payment-requests/src/main.rs @@ -1,4 +1,4 @@ -//! Disposable rc56 Paykit peer for Bitkit regtest journeys. +//! Disposable rc65 Paykit peer for Bitkit regtest journeys. //! Two Compose services run this binary with separate identities and receiver paths. #![recursion_limit = "512"] @@ -19,10 +19,10 @@ use paykit_lib::{ PaymentRequestId, PaymentRequestTerms, Recurrence, RecurrenceConfig, RecurrenceUnit, }; use paykit_sdk::{ - InMemoryStorage, LinkedPeerState, PaykitReceiverCapabilities, PaykitReceiverPath, PaykitSdk, + PubkySharedStateStorage, StorageAdapter, LinkedPeerState, PaykitAppCapabilities, PaykitAppId, PaykitApp, PaykitSdk, PaykitSdkConfig, PaymentAdapter, PaymentRequestLifecycleState, PubkyLocalSecretKey, PubkyPublicKey, PubkySessionAccess, PubkySessionBootstrap, PubkySessionProvider, - ReceiverNoiseSecretKey, + PAYKIT_AUTHORIZER_SESSION_CAPABILITIES, }; use pubky::{Keypair, Pubky}; use serde::Deserialize; @@ -31,6 +31,8 @@ use tokio::sync::Mutex; const HOMESERVER: &str = "pubky8pinxxgqs41n4aididenw5apqp1urfmzdztr8jt4abrkdn435ewo"; const ENDPOINT: &str = "btc-regtest-p2wpkh"; +// the identifier Bitkit reads an LNURL-pay endpoint from (`MethodId.Lnurl`) +const LNURL_ENDPOINT: &str = "btc-lightning-lnurl"; #[derive(Clone)] struct SessionProvider(Arc>>); @@ -60,16 +62,39 @@ struct FixturePaymentAdapter; #[async_trait] impl PaymentAdapter for FixturePaymentAdapter {} -type FixtureSdk = PaykitSdk; +type FixtureSdk = PaykitSdk; struct App { sdk: FixtureSdk, + // the SDK's own storage, to give a proposal the id a journey names (see `issue`) + storage: PubkySharedStateStorage, pubky: PubkyPublicKey, - receiver_path: PaykitReceiverPath, + receiver_path: PaykitAppId, address: String, role: String, // The SDK stores per-peer operation leases; serialize manual control calls. operation: Mutex<()>, + // `/endpoints` withhold: no public endpoint and an empty private payment list, so an app cannot resolve the request's endpoint + withheld: Mutex, + // the LNURL-pay endpoint the last `/request` named (`lnurl`): offered beside the regtest address from then on + lnurl: std::sync::Mutex>, +} + +impl App { + /// The receiving details the issuer offers: none while its endpoints are withheld. + fn private_details(&self, withheld: bool) -> Vec { + if withheld { + return vec![]; + } + let mut details = vec![paykit_sdk::PrivateReceivingDetail { + identifier: ENDPOINT.into(), + payload: json!({ "value": self.address }).to_string(), + }]; + if let Some(lnurl) = self.lnurl.lock().unwrap().clone() { + details.push(paykit_sdk::PrivateReceivingDetail { identifier: LNURL_ENDPOINT.into(), payload: json!({ "value": lnurl }).to_string() }); + } + details + } } #[derive(Deserialize)] @@ -79,11 +104,8 @@ struct Peer { } impl Peer { - fn parsed(&self) -> Result<(PubkyPublicKey, PaykitReceiverPath)> { - Ok(( - PubkyPublicKey::from_raw_or_app_key(&self.peer_pubky)?, - PaykitReceiverPath::new(&self.peer_path)?, - )) + fn parsed(&self) -> Result { + Ok(PubkyPublicKey::from_raw_or_app_key(&self.peer_pubky)?) } } @@ -91,6 +113,8 @@ impl Peer { struct LinkInput { #[serde(flatten)] peer: Peer, + #[serde(default)] + #[allow(dead_code)] mode: String, } @@ -100,9 +124,18 @@ struct RequestInput { peer: Peer, amount_sats: u64, reference: String, + // the id a journey names for its request (the issuer contract of the Bitkit journeys: `71300000-0000-4000-8000-000000000001`); a random one without it + payment_request_id: Option, + // a request with no deadline + no_deadline: Option, deadline_at: Option, monthly_starts_at: Option, period_start_deadline_seconds: Option, + // an LNURL-pay string (`GET :3010/generate/pay` of the lane's LNURL fixture): the request then accepts only `btc-lightning-lnurl`, + // which the private payment list sent with it offers (bitkit-android#1401 J19, 7 Oct) + lnurl: Option, + // when the proposal itself expires (the acceptance deadline), apart from the payment deadline + proposal_expires_at: Option, } #[derive(Deserialize)] @@ -156,6 +189,21 @@ async fn rpc(method: &str, params: Value) -> Result { Ok(response["result"].clone()) } +/// Mines to this wallet until it can send `sats` and a fee: a seat's chain starts at one block, whose coinbase is immature, so +/// `/pay` failed with bitcoind's "Insufficient funds" (bitkit-android#1401 J14, 7 Oct). Regtest halves the subsidy every 150 blocks, +/// so a long chain may need more than one round of 101 blocks. +async fn fund(sats: u64) -> Result<()> { + let needed = sats as f64 / 100_000_000.0 + 0.001; + for _ in 0..5 { + if rpc("getbalance", json!([])).await?.as_f64().unwrap_or(0.0) >= needed { + return Ok(()); + } + let address = rpc("getnewaddress", json!(["", "bech32"])).await?; + rpc("generatetoaddress", json!([101, address])).await?; + } + bail!("the fixture wallet holds less than {needed} BTC after mining 505 blocks") +} + /// Retries a step that depends on the Pubky testnet or bitcoind, which may still be starting when this /// container starts (Compose only waits for their containers to exist). Bounded by /// `FIXTURE_SETUP_TIMEOUT_SECONDS` (default 120), so a broken dependency still ends in a clear exit. @@ -183,49 +231,53 @@ where async fn setup() -> Result { let role = env::var("FIXTURE_ROLE").context("FIXTURE_ROLE is required")?; - let receiver_path = PaykitReceiverPath::new(match role.as_str() { - "fixture-issuer" => "bitkit/server", - "rc56-peer" => "bitkit/wallet", - _ => bail!("unknown FIXTURE_ROLE"), - })?; + // the issuer contract of the Bitkit journeys: the fixture issuer's App ID is `paykit-server` + let default_app_id = if role == "fixture-issuer" { "paykit-server" } else { "qa-fixture" }; + let receiver_path = PaykitAppId::new(env::var("APP_ID").unwrap_or_else(|_| default_app_id.into()))?; let pubky = Pubky::testnet()?; let bootstrap = PubkySessionBootstrap::with_pubky(pubky, "bitkit-docker.fixture")? .with_auth_relay("http://localhost:15412/inbox")?; - let config = PaykitSdkConfig::new(receiver_path.clone()); + let config = PaykitSdkConfig::new(receiver_path.clone())?; let homeserver = PubkyPublicKey::from_raw_or_app_key(HOMESERVER)?; // A new identity per attempt: a failed sign-up must not leave the retry with a half-created account. let signed_up = retry("sign-up on the local homeserver", || async { - let secret = PubkyLocalSecretKey::new(Keypair::random().secret_key()); + let phrase_file = env::var("MNEMONIC_FILE").ok(); + let secret = if let Some(ref file) = phrase_file { PubkyLocalSecretKey::from_bip39_mnemonic(std::fs::read_to_string(file)?.trim())? } else { PubkyLocalSecretKey::new(Keypair::random().secret_key()) }; + if phrase_file.is_some() { return Ok(bootstrap.sign_in(&secret, PAYKIT_AUTHORIZER_SESSION_CAPABILITIES).await?); } Ok(bootstrap .sign_up( &secret, - ReceiverNoiseSecretKey::random(), &homeserver, None, - &config.required_session_capabilities(), + PAYKIT_AUTHORIZER_SESSION_CAPABILITIES, ) .await?) }) .await?; let provider = SessionProvider(Arc::new(Mutex::new(Some(signed_up.access)))); + let storage = PubkySharedStateStorage::new(provider.clone()); let sdk = PaykitSdk::new( - InMemoryStorage::default(), + storage.clone(), provider, FixturePaymentAdapter, config, - )?; + ); sdk.initialize().await?; + let imported = env::var("MNEMONIC_FILE").is_ok(); + if !imported { + sdk.publish_paykit_noise_key_authorization().await?; retry("receiver marker publication", || async { Ok(sdk - .publish_paykit_receiver_marker(PaykitReceiverCapabilities { + .publish_paykit_app(PaykitApp::new("QA Fixture", PaykitAppCapabilities { private_payments: true, payment_requests: true, receipts: false, outgoing_payments: role == "rc56-peer", - }) + })?) .await?) }) .await?; + } let address = retry("getnewaddress from bitcoind", || async { let address = rpc("getnewaddress", json!(["", "bech32"])) .await? @@ -239,6 +291,7 @@ async fn setup() -> Result { }) .await?; let endpoint_payload = json!({ "value": address }).to_string(); + if !imported { retry("Paykit endpoint publication", || async { let published = sdk .sync_public_endpoints_with_receiving_details(vec![paykit_sdk::PublicReceivingDetail { @@ -252,13 +305,17 @@ async fn setup() -> Result { Ok(()) }) .await?; + } Ok(App { sdk, + storage, pubky: signed_up.public_key, receiver_path, address, role, operation: Mutex::new(()), + withheld: Mutex::new(false), + lnurl: std::sync::Mutex::new(None), }) } @@ -266,31 +323,29 @@ async fn info(State(app): State>) -> ApiResult { Ok(Json(json!({ "status": "ready", "role": app.role, "pubky": app.pubky.to_app_key(), "receiver_path": app.receiver_path.as_str(), "endpoint": ENDPOINT, - "address": app.address, + "address": app.address, "lnurl": app.lnurl.lock().unwrap().clone(), }))) } async fn link(State(app): State>, Json(input): Json) -> ApiResult { let _guard = app.operation.lock().await; - let (peer, path) = input.peer.parsed()?; - let report = match input.mode.as_str() { - "initiate" => app.sdk.initiate_link_with_peer(peer, path).await?, - "accept" => app.sdk.accept_link_with_peer(peer, path).await?, - _ => return Err(anyhow!("mode must be initiate or accept").into()), - }; + let peer = input.peer.parsed()?; + // the canonical link flow of the SDK (`initiate` and `accept` are one call: whichever side starts, it drives the handshake on); `mode` is kept in the + // request for the testers that still send it and is not read + let report = app.sdk.ensure_link_with_peer(peer, 1).await?; Ok(Json(serde_json::to_value(report)?)) } -async fn sync_locked(app: &App, peer: PubkyPublicKey, path: PaykitReceiverPath) -> Result { +async fn sync_locked(app: &App, peer: PubkyPublicKey) -> Result { let current = app .sdk .linked_peers() .await? .into_iter() - .find(|item| item.counterparty == peer && item.counterparty_receiver_path == path); + .find(|item| item.counterparty == peer); let state = current.context("link not started")?.state; if state == LinkedPeerState::Linking { - let report = app.sdk.advance_link_handshake(peer, path).await?; + let report = app.sdk.ensure_link_with_peer(peer, 1).await?; return Ok(json!({ "link": report })); } if state != LinkedPeerState::Linked { @@ -298,28 +353,30 @@ async fn sync_locked(app: &App, peer: PubkyPublicKey, path: PaykitReceiverPath) } let received = app .sdk - .receive_private_messages(peer.clone(), path.clone()) + .receive_private_messages(peer.clone()) .await?; let sent = app .sdk - .process_outbound_private_messages(peer.clone(), path.clone()) + .process_outbound_private_messages(peer.clone()) .await?; - let records = app.sdk.payment_requests_with(&peer, &path).await?; + let records = app.sdk.payment_requests_with(&peer).await?; + let lists = app.sdk.current_private_payment_lists(&peer).await?; Ok( json!({ "link": "linked", "received": received.stream_item_ids.len(), - "sent": sent.sent.len(), "failed": sent.failed.len(), "records": records }), + "sent": sent.sent.len(), "failed": sent.failed.len(), "records": records, + "private_payment_lists": lists }), ) } async fn sync(State(app): State>, Json(input): Json) -> ApiResult { let _guard = app.operation.lock().await; - let (peer, path) = input.parsed()?; - Ok(Json(sync_locked(&app, peer, path).await?)) + let peer = input.parsed()?; + Ok(Json(sync_locked(&app, peer).await?)) } async fn issue(State(app): State>, Json(input): Json) -> ApiResult { let _guard = app.operation.lock().await; - let (peer, path) = input.peer.parsed()?; + let peer = input.peer.parsed()?; if input.amount_sats == 0 || input.reference.is_empty() { return Err(anyhow!("amount_sats and reference are required").into()); } @@ -334,8 +391,16 @@ async fn issue(State(app): State>, Json(input): Json) -> "btc", )?, PaymentReference::new(input.reference)?, - vec![PaymentEndpointIdentifier::new(ENDPOINT)?], - ); + vec![PaymentEndpointIdentifier::new(if input.lnurl.is_some() { LNURL_ENDPOINT } else { ENDPOINT })?], + ) + .proposal_expires_at(input.proposal_expires_at); + if let Some(lnurl) = input.lnurl { + if !lnurl.to_ascii_lowercase().starts_with("lnurl1") { + return Err(anyhow!("lnurl must be a bech32 LNURL (lnurl1...)").into()); + } + *app.lnurl.lock().unwrap() = Some(lnurl); + } + let fixed_id = input.payment_request_id; let terms = if let Some(start) = input.monthly_starts_at { let recurrence = Recurrence::try_from(RecurrenceConfig { every: 1, @@ -349,6 +414,8 @@ async fn issue(State(app): State>, Json(input): Json) -> .payment_deadline(Some(PaymentDeadline::PeriodStart { seconds: input.period_start_deadline_seconds.unwrap_or(86_400), })) + } else if input.no_deadline.unwrap_or(false) { + terms } else { let deadline = input.deadline_at.unwrap_or_else(|| { (now + Duration::days(7)).to_rfc3339_opts(SecondsFormat::Secs, true) @@ -357,17 +424,46 @@ async fn issue(State(app): State>, Json(input): Json) -> timestamp: deadline, })) }; - let record = app + // rc62 apps show "waiting for updated private payment details" until the issuer's Private Payment List reached them: the list goes + // out with the request, with the regtest endpoint of the fixture as its one private receiving detail. + // While `/endpoints` withholds them, the list goes out empty: the request names an endpoint the app cannot resolve. + let withheld = *app.withheld.lock().await; + app.sdk + .enqueue_private_payment_list_with_receiving_details(peer.clone(), app.private_details(withheld)) + .await?; + let mut record = app .sdk - .propose_payment_request(peer.clone(), path.clone(), terms.build()?) + .propose_payment_request(peer.clone(), terms.build()?) .await?; + if let Some(fixed) = fixed_id { + // the SDK names a proposal itself; the queued message is given the id the journey names before it goes out + PaymentRequestId::new(fixed.clone())?; + let generated = record.payment_request_id.clone(); + let counterparty = peer.clone(); + let replacement = fixed.clone(); + app.storage + .transaction(move |tx| { + let mut message = tx + .queued_outbound_private_messages(&counterparty) + .into_iter() + .find(|message| message.raw_json.contains(&generated)) + .ok_or_else(|| paykit_sdk::PaykitSdkError::Protocol { + context: "fixture proposal not queued".into(), + source: None, + })?; + message.raw_json = message.raw_json.replace(&generated, &replacement); + tx.save_outbound_private_message(message) + }) + .await?; + record.payment_request_id = fixed; + } let sent = app .sdk - .process_outbound_private_messages(peer, path) + .process_outbound_private_messages(peer) .await?; - if !sent.failed.is_empty() || sent.sent.len() != 1 { + if !sent.failed.is_empty() || sent.sent.len() < 2 { return Err(anyhow!( - "Payment Request delivery failed: {} failed, {} sent", + "Payment Request delivery failed: {} failed, {} sent (the private payment list and the request)", sent.failed.len(), sent.sent.len() ) @@ -375,47 +471,50 @@ async fn issue(State(app): State>, Json(input): Json) -> } Ok(Json( json!({ "payment_request_id": record.payment_request_id, "state": record.state, - "deadline": record.terms.as_ref().and_then(|terms| terms.payment_deadline.as_ref()) }), + "deadline": record.terms.as_ref().and_then(|terms| terms.payment_deadline.as_ref()), + "endpoints_withheld": withheld }), )) } async fn records(State(app): State>, Json(input): Json) -> ApiResult { let _guard = app.operation.lock().await; - let (peer, path) = input.parsed()?; + let peer = input.parsed()?; Ok(Json( - json!({ "records": app.sdk.payment_requests_with(&peer, &path).await? }), + json!({ "records": app.sdk.payment_requests_with(&peer).await? }), )) } async fn act(State(app): State>, action: &'static str, input: RecordInput) -> ApiResult { let _guard = app.operation.lock().await; - let (peer, path) = input.peer.parsed()?; + let peer = input.peer.parsed()?; let _ = app .sdk - .receive_private_messages(peer.clone(), path.clone()) + .receive_private_messages(peer.clone()) .await?; let id = PaymentRequestId::new(input.payment_request_id)?; let record = match action { "accept" => { + app.sdk.claim_payment_request_for_execution(peer.clone(), &id).await?; app.sdk - .accept_payment_request(peer.clone(), path.clone(), &id) + .accept_payment_request(peer.clone(), &id) .await? } "reject" => { app.sdk - .reject_payment_request(peer.clone(), path.clone(), &id, None) + .reject_payment_request(peer.clone(), &id, None) .await? } "cancel" => { + app.sdk.claim_payment_request_for_execution(peer.clone(), &id).await?; app.sdk - .cancel_payment_request(peer.clone(), path.clone(), &id, None) + .cancel_payment_request(peer.clone(), &id, None) .await? } _ => unreachable!(), }; let sent = app .sdk - .process_outbound_private_messages(peer, path) + .process_outbound_private_messages(peer) .await?; if !sent.failed.is_empty() || sent.sent.len() != 1 { return Err(anyhow!("{action} delivery failed").into()); @@ -437,15 +536,15 @@ async fn cancel(State(app): State>, Json(input): Json) -> async fn pay_impl(app: Arc, input: PayInput, existing_tx: bool) -> ApiResult { let _guard = app.operation.lock().await; - let (peer, path) = input.record.peer.parsed()?; + let peer = input.record.peer.parsed()?; let id = PaymentRequestId::new(input.record.payment_request_id)?; let _ = app .sdk - .receive_private_messages(peer.clone(), path.clone()) + .receive_private_messages(peer.clone()) .await?; let record = app .sdk - .payment_requests_with(&peer, &path) + .payment_requests_with(&peer) .await? .into_iter() .find(|record| record.payment_request_id == id.as_str()) @@ -498,6 +597,7 @@ async fn pay_impl(app: Arc, input: PayInput, existing_tx: bool) -> ApiResul if input.txid.is_some() { return Err(anyhow!("use /proof to retry an existing transaction").into()); } + fund(input.amount_sats).await?; rpc( "sendtoaddress", json!([input.address, input.amount_sats as f64 / 100_000_000.0]), @@ -512,9 +612,9 @@ async fn pay_impl(app: Arc, input: PayInput, existing_tx: bool) -> ApiResul .sdk .submit_payment_proof( peer.clone(), - path.clone(), &id, period, + PaykitAppId::new("bitkit")?, PaymentEndpointIdentifier::new(ENDPOINT)?, proof, ) @@ -528,7 +628,7 @@ async fn pay_impl(app: Arc, input: PayInput, existing_tx: bool) -> ApiResul )) } }; - let sent = app.sdk.process_outbound_private_messages(peer, path).await; + let sent = app.sdk.process_outbound_private_messages(peer).await; let proof_sent = sent .as_ref() .is_ok_and(|report| report.failed.is_empty() && report.sent.len() == 1); @@ -546,8 +646,104 @@ async fn proof(State(app): State>, Json(input): Json) -> ApiR pay_impl(app, input, true).await } +async fn withdraw(State(app): State>, Json(input): Json) -> ApiResult { + let _guard = app.operation.lock().await; + let peer = input.parsed()?; + let report = app.sdk.sync_private_payment_lists_with_reservations_and_process_outbound(vec![paykit_sdk::PrivatePaymentListReservationUpdate {counterparty: peer, reservations: vec![]}], false).await?; + Ok(Json(serde_json::to_value(report)?)) +} + +#[derive(Deserialize)] +struct EndpointsInput { + // `withhold` or `restore` + action: String, + // the peer whose private payment list follows the change; without one only the public endpoint changes + #[serde(flatten)] + peer: Option, +} + +async fn endpoints_state(State(app): State>) -> ApiResult { + Ok(Json(json!({ "withheld": *app.withheld.lock().await, "endpoint": ENDPOINT }))) +} + +/// Withholds or restores the issuer's payment endpoints: its public endpoint and, for the named peer, its private payment list. +async fn endpoints(State(app): State>, Json(input): Json) -> ApiResult { + let _guard = app.operation.lock().await; + let withheld = match input.action.as_str() { + "withhold" => true, + "restore" => false, + other => return Err(anyhow!("action must be withhold or restore, not {other}").into()), + }; + *app.withheld.lock().await = withheld; + let public: Vec = app + .private_details(withheld) + .into_iter() + .map(|detail| paykit_sdk::PublicReceivingDetail { identifier: detail.identifier, payload: detail.payload }) + .collect(); + let published = app.sdk.sync_public_endpoints_with_receiving_details(public).await?; + let mut private = Value::Null; + if let Some(peer) = input.peer { + let peer = peer.parsed()?; + app.sdk + .enqueue_private_payment_list_with_receiving_details(peer.clone(), app.private_details(withheld)) + .await?; + let sent = app.sdk.process_outbound_private_messages(peer).await?; + if !sent.failed.is_empty() || sent.sent.is_empty() { + return Err(anyhow!("private payment list delivery failed: {} failed, {} sent", sent.failed.len(), sent.sent.len()).into()); + } + private = json!({ "sent": sent.sent.len(), "entries": app.private_details(withheld).len() }); + } + Ok(Json(json!({ + "withheld": withheld, + "public": { "published": published.published.len(), "failed": published.failed.len() }, + "private_payment_list": private, + }))) +} + +/// `paykit-key-authorization` (this binary under that name, in the marketplace driver's image): publishes the Paykit noise key +/// authorization of an identity the driver signed up, as a Bitkit wallet does in its own Paykit setup. Paykit Server rc65 checks it +/// before it accepts a setup claim. Reads `{"version":1,"creator_secret":""}` on stdin. +async fn publish_key_authorization() -> Result<()> { + use base64::Engine; + #[derive(Deserialize)] + struct Input { + version: u8, + creator_secret: String, + } + let mut body = String::new(); + std::io::Read::read_to_string(&mut std::io::stdin(), &mut body)?; + let input: Input = serde_json::from_str(&body)?; + if input.version != 1 { + bail!("unsupported input version"); + } + let bytes: [u8; 32] = base64::engine::general_purpose::URL_SAFE_NO_PAD + .decode(input.creator_secret.trim())? + .try_into() + .map_err(|_| anyhow!("creator_secret must be 32 bytes"))?; + let secret = PubkyLocalSecretKey::new(bytes); + let bootstrap = PubkySessionBootstrap::with_pubky(Pubky::testnet()?, "bitkit-docker.fixture")? + .with_auth_relay("http://localhost:15412/inbox")?; + let signed_in = bootstrap.sign_in(&secret, PAYKIT_AUTHORIZER_SESSION_CAPABILITIES).await?; + let pubky = signed_in.public_key.to_app_key(); + let provider = SessionProvider(Arc::new(Mutex::new(Some(signed_in.access)))); + let sdk = PaykitSdk::new( + PubkySharedStateStorage::new(provider.clone()), + provider, + FixturePaymentAdapter, + PaykitSdkConfig::new(PaykitAppId::new("qa-fixture")?)?, + ); + sdk.initialize().await?; + sdk.publish_paykit_noise_key_authorization().await?; + println!("{}", json!({ "published": true, "pubky": pubky })); + Ok(()) +} + #[tokio::main] async fn main() -> Result<()> { + let invoked = env::args().next().unwrap_or_default(); + if invoked.ends_with("paykit-key-authorization") { + return publish_key_authorization().await; + } let app = Arc::new(setup().await?); let port: u16 = env::var("FIXTURE_PORT") .unwrap_or_else(|_| "3002".into()) @@ -564,6 +760,8 @@ async fn main() -> Result<()> { .route("/cancel", post(cancel)) .route("/pay", post(pay)) .route("/proof", post(proof)) + .route("/withdraw", post(withdraw)) + .route("/endpoints", get(endpoints_state).post(endpoints)) .with_state(app); let listener = tokio::net::TcpListener::bind(("0.0.0.0", port)).await?; axum::serve(listener, router).await?; diff --git a/pubky-marketplace b/pubky-marketplace index 150bdfe..5c595f4 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -7,29 +7,37 @@ set -euo pipefail CLI_NAME="$(basename "$0")" cd "$(dirname "$0")" -# Pinned upstream revisions. Paykit Server 722ef268 is v0.1.0-rc4 (pubky/paykit-server master). Its setup -# flow emits the Pubky grant auth URL (pubkyauth://signin_grant with cid and cpk) that the apps' Paykit SDK -# (0.1.0-rc55) requires, and still carries x-bitkit-claim=watch-only-account-v1. Earlier revisions such as -# 867fc883 emit the legacy pubkyauth://signin URL, which the apps reject. -# Pubky Core is pinned in marketplace/pubky-testnet/Dockerfile. -PAYKIT_SERVER_REV=722ef26834d8a4fc849de1d883eb296106dd2006 -PAYKIT_RS_REV=9b56a0eacd6874137370fa79ec0f40b809140809 # v0.1.0-rc48, paykit-server's paykit-lib and paykit-sdk pin -LOCKS_REV=8502ef79c443c640976a2a901b80c5e717319149 # v0.1.0-rc1, paykit-server's locks-core pin +# Pinned upstream revisions. Paykit Server is pubky/paykit-server#46 (head 0ffd4da, paykit-rs v0.1.0-rc65, the version both apps pin; not merged +# or released yet, the latest release is rc8): move PAYKIT_SERVER_REV to its merge or release once #46 lands. Its setup flow emits the Pubky +# grant auth URL (pubkyauth://signin_grant with cid and cpk) that the apps' Paykit SDK requires. +# The Pubky testnet is pinned in marketplace/pubky-testnet/Dockerfile. +PAYKIT_SERVER_REV=${PAYKIT_SERVER_REV:-0ffd4da2adaab048939e0ea18ff916a27a7cfb0e} +PAYKIT_RS_REV=${PAYKIT_RS_REV:-7185ae7da9315028e5331442d71d739c27f1442c} # v0.1.0-rc65, paykit-server's paykit-lib and paykit-sdk pin +PAYKIT_RS_TAG=${PAYKIT_RS_TAG:-v0.1.0-rc65} +LOCKS_REV=${LOCKS_REV:-b3dc87c961f6b907d76cde9c4a28f00670d5231e} # v0.1.0-rc8, paykit-server's locks-core pin +LOCKS_TAG=${LOCKS_TAG:-v0.1.0-rc8} SOURCES=.marketplace/sources +# Compose reads these for the Paykit Server and driver image tags and labels. +export PAYKIT_SERVER_REV PAYKIT_RS_REV PAYKIT_SERVER_TAG="${PAYKIT_SERVER_REV:0:7}" +PAYKIT_SERVER_PATCHES=$(cd marketplace/patches 2>/dev/null && ls paykit-server-*.patch 2>/dev/null | tr '\n' ' ' | sed 's/ $//') +export PAYKIT_SERVER_PATCHES +PAYKIT_SERVER_IMAGE=bitkit-docker/paykit-server:$PAYKIT_SERVER_TAG +DRIVER_IMAGE=bitkit-docker/marketplace-driver:$PAYKIT_SERVER_TAG PAYKIT_PORT="${MARKETPLACE_PAYKIT_PORT:-3001}" COMPOSE=(docker compose --profile marketplace) CHAIN_SERVICES=(bitcoind bitcoinsetup electrs) -FIXTURE_SERVICES=(marketplace-postgres pubky-testnet paykit-server) +FIXTURE_SERVICES=(marketplace-postgres pubky-testnet homeserver-proxy paykit-server) # Opt-in payment request peers (README, Payment Request fixture). They share the testnet's network namespace. PEER_SERVICES=(fixture-issuer rc56-peer) show_help() { cat < [options] Stack: build Fetch the pinned sources and build the images + build-paykit Build only Paykit Server and the driver (pins overridable through the environment) up Build if needed, start the chain and the fixture, wait until ready ps Show the fixture containers and Paykit Server readiness logs [service] Follow the fixture logs @@ -119,6 +127,7 @@ fetch_pinned() { git -C "$dir" remote add origin "$url" fi if [ "$(git -C "$dir" rev-parse -q --verify HEAD 2>/dev/null || true)" != "$rev" ]; then + git -C "$dir" checkout -q -- . 2>/dev/null || true # drop the fixture's patches before moving to another revision git -C "$dir" fetch -q --depth 1 origin "$rev" git -C "$dir" checkout -q --detach FETCH_HEAD fi @@ -133,29 +142,92 @@ fetch_pinned() { # swaps those for the checkouts below. Fail here, before a long build, if a tag does not resolve to our pins. check_lock_pins() { local lock="$SOURCES/paykit-server/Cargo.lock" - if ! grep -Fq "source = \"git+https://github.com/pubky/paykit-rs.git?tag=v0.1.0-rc48#$PAYKIT_RS_REV\"" "$lock" || - ! grep -Fq "source = \"git+https://github.com/pubky/locks.git?tag=v0.1.0-rc1#$LOCKS_REV\"" "$lock"; then + if ! grep -Fq "source = \"git+https://github.com/pubky/paykit-rs.git?tag=$PAYKIT_RS_TAG#$PAYKIT_RS_REV\"" "$lock" || + ! grep -Fq "source = \"git+https://github.com/pubky/locks.git?tag=$LOCKS_TAG#$LOCKS_REV\"" "$lock"; then echo "$lock does not lock paykit-rs $PAYKIT_RS_REV and locks $LOCKS_REV" >&2 exit 1 fi } +# Fixes the fixture carries on top of the pinned Paykit Server tree until upstream has them (marketplace/patches, named in the +# image label tech.masivo.paykit-server-patches). Each applies once; a patch that neither applies nor is applied stops the build. +apply_paykit_server_patches() { + local patch + for patch in marketplace/patches/paykit-server-*.patch; do + [ -e "$patch" ] || continue + if git -C "$SOURCES/paykit-server" apply --check "$PWD/$patch" 2>/dev/null; then + git -C "$SOURCES/paykit-server" apply "$PWD/$patch" + echo "patched $SOURCES/paykit-server with $(basename "$patch")" + elif ! git -C "$SOURCES/paykit-server" apply --check --reverse "$PWD/$patch" 2>/dev/null; then + echo "$patch does not apply to paykit-server ${PAYKIT_SERVER_REV:0:8}; drop it if upstream fixed it" >&2 + exit 1 + fi + done +} + fetch_sources() { fetch_pinned https://github.com/pubky/paykit-server.git "$SOURCES/paykit-server" "$PAYKIT_SERVER_REV" + apply_paykit_server_patches fetch_pinned https://github.com/pubky/paykit-rs.git "$SOURCES/paykit-rs" "$PAYKIT_RS_REV" fetch_pinned https://github.com/pubky/locks.git "$SOURCES/locks" "$LOCKS_REV" check_lock_pins } +# Paykit Server and the driver, which carries its helpers and the payment request fixture's `paykit-key-authorization` (scripts/follow-app-paykit sets the pins through the environment). +build_paykit() { + fetch_sources + if docker buildx version >/dev/null 2>&1; then + compose build paykit-server + compose build marketplace-driver + else + classic_build + fi +} + build() { fetch_sources - compose build pubky-testnet paykit-server - compose build marketplace-driver + # the payment request fixture's image first: the driver copies its `paykit-key-authorization` from it + compose build homeserver-proxy + if docker buildx version >/dev/null 2>&1; then + compose build pubky-testnet paykit-server + compose build marketplace-driver + else + classic_build + fi +} + +# Docker without BuildKit (the QA boxes) cannot build Paykit Server's Dockerfile.local (named contexts, cache mounts) or the +# driver's (a service context). Build both with the classic builder from generated Dockerfiles: the named contexts become +# COPYs from .marketplace/sources, the cache mounts go, and the labels name the pinned revisions as the Compose build does. +classic_build() { + docker image inspect bitkit-docker/pubky-testnet:0.14.0 >/dev/null 2>&1 || compose build pubky-testnet + printf '**/.git\n**/target\n' > "$SOURCES/.dockerignore" + python3 - "$SOURCES/paykit-server/Dockerfile.local" "$PAYKIT_SERVER_REV" "$PAYKIT_RS_REV" "$PAYKIT_SERVER_PATCHES" > "$SOURCES/Dockerfile.classic" <<'PY' +import re, sys +text, server, paykit, patches = open(sys.argv[1]).read(), sys.argv[2], sys.argv[3], sys.argv[4] +text = re.sub(r"^# syntax=.*\n", "", text) +copies = "\n".join([ + "COPY paykit-rs/paykit-lib /build/paykit-rs/paykit-lib", + "COPY paykit-rs/paykit-sdk /build/paykit-rs/paykit-sdk", + "COPY locks/Cargo.toml locks/Cargo.lock /build/locks/", +] + [f"COPY locks/{crate} /build/locks/{crate}" for crate in ("locks-core", "locks-service", "locks-server", "locks-sdk", "locks-e2e")]) +text, named = re.subn(r"RUN --mount=from=paykit-lib.*?/build/locks/\n", copies + "\n", text, flags=re.S) +text, server_copy = re.subn(r"^COPY \. /build/paykit-server$", "COPY paykit-server /build/paykit-server", text, flags=re.M) +text = re.sub(r"RUN --mount=type=cache[^\n]*\\\n(\s+--mount=type=cache[^\n]*\\\n)*", "RUN ", text) +if named != 1 or server_copy != 1 or "--mount" in text: + sys.exit("Dockerfile.local changed shape; update classic_build") +print(text + f'\nLABEL tech.masivo.paykit-server="{server}" tech.masivo.paykit-rs="{paykit}" tech.masivo.paykit-server-patches="{patches}"') +PY + docker build -q -f "$SOURCES/Dockerfile.classic" -t "$PAYKIT_SERVER_IMAGE" "$SOURCES" + { sed -e "s|^COPY --from=paykit |COPY --from=$PAYKIT_SERVER_IMAGE |" -e "s|^COPY --from=fixture |COPY --from=bitkit-docker/payment-request-fixture:rc65-shared |" marketplace/driver/Dockerfile + echo "LABEL tech.masivo.paykit-server=\"$PAYKIT_SERVER_REV\" tech.masivo.paykit-rs=\"$PAYKIT_RS_REV\""; } > marketplace/driver/.Dockerfile.classic + docker build -q -f marketplace/driver/.Dockerfile.classic -t "$DRIVER_IMAGE" marketplace/driver + rm -f marketplace/driver/.Dockerfile.classic } images_present() { - docker image inspect bitkit-docker/pubky-testnet:f68014c1 bitkit-docker/paykit-server:722ef268 \ - bitkit-docker/marketplace-driver:local >/dev/null 2>&1 + docker image inspect bitkit-docker/pubky-testnet:0.14.0 "$PAYKIT_SERVER_IMAGE" \ + "$DRIVER_IMAGE" bitkit-docker/payment-request-fixture:rc65-shared >/dev/null 2>&1 } wait_ready() { @@ -179,7 +251,7 @@ up() { build fi progress "starting the chain, Postgres and the Pubky testnet" - compose up -d "${CHAIN_SERVICES[@]}" marketplace-postgres pubky-testnet + compose up -d "${CHAIN_SERVICES[@]}" marketplace-postgres pubky-testnet homeserver-proxy progress "initializing the driver state" driver init progress "starting Paykit Server" @@ -221,6 +293,7 @@ fi case "$command" in build) build ;; + build-paykit) build_paykit ;; up) up ;; ps) compose ps "${CHAIN_SERVICES[@]}" "${FIXTURE_SERVICES[@]}" || true diff --git a/scripts/follow-app-paykit b/scripts/follow-app-paykit new file mode 100755 index 0000000..4aa1ff0 --- /dev/null +++ b/scripts/follow-app-paykit @@ -0,0 +1,146 @@ +#!/usr/bin/env bash +# Keep the Paykit fixtures on the paykit-rs version a Bitkit pull request pins. +# Reads the pin at the PR head (Android: gradle/libs.versions.toml, iOS: Package.resolved), compares it with the +# paykit-rs commit the local fixture images were built from, and rebuilds and retags the ones that differ. +# Usage: scripts/follow-app-paykit [--check] +# --check only print the pin and what would be rebuilt (exit 3 when something is out of date) +# Exit 4: no Paykit Server revision (the linked pubky/paykit-server PR, else master) locks the app's paykit-rs version. +set -euo pipefail +cd "$(dirname "$0")/.." + +FIXTURE=bitkit-docker/payment-request-fixture +SERVER=bitkit-docker/paykit-server +DRIVER=bitkit-docker/marketplace-driver +PAYKIT_RS=https://github.com/pubky/paykit-rs + +die() { echo "follow-app-paykit: $*" >&2; exit 1; } +[ $# -ge 2 ] || die "usage: $0 [--check]" +repo="$1" ref="$2" check="${3:-}" + +pr="" +if [[ "$ref" =~ ^[0-9]+$ ]]; then + pr="$ref" + ref=$(git ls-remote "https://github.com/$repo" "refs/pull/$ref/head" | cut -f1) + [ -n "$ref" ] || die "no head for $repo#$2" +fi +raw() { curl -fsS "https://raw.githubusercontent.com/$repo/$ref/$1"; } + +case "$repo" in + */bitkit-android) + version=$(raw gradle/libs.versions.toml | sed -n 's|.*com\.synonym:paykit-android", version = "\([^"]*\)".*|\1|p' | head -1) + rev=$(git ls-remote "$PAYKIT_RS" "refs/tags/v$version^{}" | cut -f1) + ;; + */bitkit-ios) + resolved=$(raw Bitkit.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved) + version=$(jq -r '.pins[] | select(.identity=="paykit-rs") | .state.version // empty' <<<"$resolved") + rev=$(jq -r '.pins[] | select(.identity=="paykit-rs") | .state.revision' <<<"$resolved") + ;; + *) die "$repo is not a Bitkit app repository" ;; +esac +[ -n "${version:-}" ] && [ -n "${rev:-}" ] || die "no paykit-rs pin found in $repo at $ref" +short="${version##*-}" # 0.1.0-rc65 -> rc65 +echo "$repo@${ref:0:7} pins paykit-rs $version ($rev)" + +# The tags Compose and the lanes start (COMPOSE_FILE overrides included), kept pointing at the image built for the pin. +compose_tags() { + docker compose --profile marketplace --profile payment-requests config --format json | + jq -r --arg repo "$1" '.services[].image // empty | select(startswith($repo + ":")) | ltrimstr($repo + ":")' | sort -u +} + +# The paykit-rs commit an image was built from: its label, else the file the fixture image carries. +built_rev() { + local label + label=$(docker image inspect --format '{{ index .Config.Labels "tech.masivo.paykit-rs" }}' "$1" 2>/dev/null || true) + [ -n "$label" ] && [ "$label" != "" ] && { echo "$label"; return; } + docker run --rm --entrypoint cat "$1" /usr/local/share/paykit-rs-rev 2>/dev/null || true +} +api() { curl -fsS -H 'Accept: application/vnd.github+json' "https://api.github.com/$1"; } + +no_buildkit_dockerfile() { + # The QA boxes' Docker has no BuildKit: drop the cache mounts, keep everything else. + sed -E -e 's/^RUN --mount=[^ ]+ \\/RUN \\/' -e '/^[[:space:]]+--mount=/d' "$1" > "$2" +} + +stale=0 +fixture_built=0 +current=$(built_rev "$FIXTURE:$short-shared") +if [ "$current" = "$rev" ]; then + echo "fixture: $FIXTURE:$short-shared is built from $rev" +else + stale=1 + echo "fixture: $FIXTURE:$short-shared is built from ${current:-nothing}; needs $rev" + if [ "$check" != --check ]; then + dockerfile=payment-requests/Dockerfile + if ! docker buildx version >/dev/null 2>&1; then + no_buildkit_dockerfile payment-requests/Dockerfile payment-requests/.Dockerfile.nobuildkit + dockerfile=payment-requests/.Dockerfile.nobuildkit + fi + start=$SECONDS + docker build -q -f "$dockerfile" --build-arg "PAYKIT_RS_REV=$rev" -t "$FIXTURE:$short-shared" payment-requests + rm -f payment-requests/.Dockerfile.nobuildkit + echo "fixture: built $FIXTURE:$short-shared in $((SECONDS - start))s" + fixture_built=1 + fi +fi +if [ "$check" != --check ]; then + for tag in $(compose_tags "$FIXTURE"); do + [ "$tag" = "$short-shared" ] || { docker tag "$FIXTURE:$short-shared" "$FIXTURE:$tag"; echo "fixture: $FIXTURE:$tag -> $short-shared"; } + done +fi + +# Paykit Server: built from the Paykit Server pull request the app PR links (its merge commit once merged), else master, +# provided that revision pins the same paykit-rs tag; the driver is rebuilt with it, since it carries the server's helpers. +server_tags=$(compose_tags "$SERVER") +server_image="$SERVER:$(head -1 <<<"$server_tags")" +server_rev=$(built_rev "$server_image") +patches=$(cd marketplace/patches 2>/dev/null && ls paykit-server-*.patch 2>/dev/null | tr '\n' ' ' | sed 's/ $//') +built_patches=$(docker image inspect --format '{{ index .Config.Labels "tech.masivo.paykit-server-patches" }}' "$server_image" 2>/dev/null || true) +[ "$built_patches" = "" ] && built_patches="" +if [ "$server_rev" = "$rev" ] && [ "$built_patches" != "$patches" ]; then + echo "paykit-server: $server_image carries patches '${built_patches}'; the fixture carries '${patches}'" + server_rev="$rev (other patches)" +fi +if [ "$server_rev" = "$rev" ] && [ "$fixture_built" = 0 ]; then + echo "paykit-server: $server_image is built from $rev" +else + if [ "$server_rev" = "$rev" ]; then + # the driver carries the fixture's `paykit-key-authorization`: rebuild it on the server revision already built + server=$(docker image inspect --format '{{ index .Config.Labels "tech.masivo.paykit-server" }}' "$server_image") + from="the revision $server_image was built from" + echo "driver: the fixture was rebuilt; rebuilding the driver on $from (${server:0:7})" + else + stale=1 + echo "paykit-server: $server_image is built from ${server_rev:-an unrecorded paykit-rs}; needs $rev" + [ -n "$pr" ] || pr=$(api "repos/$repo/commits/$ref/pulls" | jq -r --arg ref "$ref" '[.[] | select(.head.sha == $ref)][0].number // empty') + linked=$( [ -n "$pr" ] && api "repos/$repo/pulls/$pr" | jq -r '.body // ""' | + grep -o 'github\.com/pubky/paykit-server/pull/[0-9]*' | head -1 | sed 's|.*/||' || true) + if [ -n "$linked" ]; then + server=$(api "repos/pubky/paykit-server/pulls/$linked" | jq -r 'if .merged then .merge_commit_sha else .head.sha end') + from="pubky/paykit-server#$linked" + else + server=$(api repos/pubky/paykit-server/commits/master | jq -r .sha) + from="pubky/paykit-server master" + fi + fi + lock=$(curl -fsS "https://raw.githubusercontent.com/pubky/paykit-server/$server/Cargo.lock") + if ! grep -Fq "paykit-rs.git?tag=v$version#$rev" <<<"$lock"; then + echo "paykit-server: $from (${server:0:7}) does not lock paykit-rs v$version; no Paykit Server revision to build" >&2 + exit 4 + fi + locks=$(sed -n 's|^source = "git+https://github.com/pubky/locks.git?tag=\([^#]*\)#\([0-9a-f]*\)"|\1 \2|p' <<<"$lock" | head -1) + echo "paykit-server: $from at ${server:0:7} locks paykit-rs v$version and locks ${locks%% *}" + if [ "$check" != --check ]; then + start=$SECONDS + PAYKIT_SERVER_REV=$server PAYKIT_RS_REV=$rev PAYKIT_RS_TAG=v$version LOCKS_TAG=${locks%% *} LOCKS_REV=${locks##* } ./pubky-marketplace build-paykit + echo "paykit-server: built $SERVER:${server:0:7} and $DRIVER:${server:0:7} in $((SECONDS - start))s" + for tag in $server_tags; do + [ "$tag" = "${server:0:7}" ] || { docker tag "$SERVER:${server:0:7}" "$SERVER:$tag"; echo "paykit-server: $SERVER:$tag -> ${server:0:7}"; } + done + for tag in $(compose_tags "$DRIVER"); do + [ "$tag" = "${server:0:7}" ] || { docker tag "$DRIVER:${server:0:7}" "$DRIVER:$tag"; echo "driver: $DRIVER:$tag -> ${server:0:7}"; } + done + fi +fi + +[ "$check" = --check ] && [ "$stale" = 1 ] && exit 3 +exit 0