Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ jobs:
run: sbt -v test
- name: Client conformance tests (2025-11-25)
run: sbt -v "clientConformance/conformance client --requirements 2025-11-25"
- name: Client conformance tests (2026-07-28)
run: sbt -v "clientConformance/conformance client --requirements 2026-07-28"
- name: Server conformance tests (2025-11-25)
run: sbt -v "serverConformance/conformance server --requirements 2025-11-25"
- name: Server conformance tests (2026-07-28)
Expand Down
19 changes: 17 additions & 2 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ lazy val serverConformance = (project in file("server-conformance"))
val baseline = specVersion
.map(v => rootDir / s"conformance-baseline-$v.yml")
.filter(_.exists)
.getOrElse(rootDir / "conformance-baseline.yml")
.getOrElse(rootDir / "conformance-baseline-2025-11-25.yml")
.getAbsolutePath
val log = streams.value.log

Expand Down Expand Up @@ -327,8 +327,23 @@ lazy val clientConformance = (project in file("client-conformance"))
val baseDir = baseDirectory.value
val rootDir = (LocalRootProject / baseDirectory).value
val wrapper = (baseDir / "bin" / "chimp-conformance-client").getAbsolutePath
val specVersion = args
.sliding(2)
.collectFirst {
case Seq("--requirements", v) => v
case Seq("--spec-version", v) => v
}
.orElse(args.collectFirst {
case a if a.startsWith("--requirements=") => a.stripPrefix("--requirements=")
case a if a.startsWith("--spec-version=") => a.stripPrefix("--spec-version=")
})
val baseline = specVersion
.map(v => rootDir / s"conformance-baseline-$v.yml")
.filter(_.exists)
.getOrElse(rootDir / "conformance-baseline-2025-11-25.yml")
.getAbsolutePath
val cmd = List("npx", s"@modelcontextprotocol/conformance@$conformanceHarnessV") ++ args ++
List("--command", wrapper, "--expected-failures", (rootDir / "conformance-baseline.yml").getAbsolutePath)
List("--command", wrapper, "--expected-failures", baseline)
val rc = Process(cmd, rootDir, "CHIMP_CLIENT_CONFORMANCE_JAR" -> jar.getAbsolutePath).!
if (rc != 0) sys.error(s"conformance harness exited with code $rc")
}
Expand Down
13 changes: 8 additions & 5 deletions client-conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,18 @@ The sbt task pins the harness to the version set in `conformanceHarnessV` in [`b

## How to run

Run the full requirement set:
Run a full requirement set (CI runs both revisions):

```bash
sbt 'clientConformance/conformance client --requirements 2025-11-25'
sbt 'clientConformance/conformance client --requirements 2026-07-28'
```

Run one scenario during development:
Run one scenario during development (pass the spec version):

```bash
sbt 'clientConformance/conformance client --scenario initialize --spec-version 2025-11-25'
sbt 'clientConformance/conformance client --scenario tools_call --spec-version 2025-11-25'
sbt 'clientConformance/conformance client --scenario tools_call --spec-version 2026-07-28'
```

The sbt task downloads `@modelcontextprotocol/conformance` with npm; `npx` must be available on the PATH.
Expand All @@ -44,8 +45,10 @@ with `McpClient`, return exit code 0 on success. Once it passes, remove the entr

## The baseline file

[`conformance-baseline.yml`](../conformance-baseline.yml) lists scenarios that are known to fail today. The harness
uses it like this:
Each spec revision keeps its own baseline, selected from the run's `--requirements` (or `--spec-version`):
[`conformance-baseline-2025-11-25.yml`](../conformance-baseline-2025-11-25.yml) and
[`conformance-baseline-2026-07-28.yml`](../conformance-baseline-2026-07-28.yml). Each file lists the scenarios (in its
`server:` and `client:` sections) known to fail today. The harness uses the chosen file like this:

| Scenario result | In baseline? | Exit code | Meaning |
|-----------------|--------------|-----------|---------------------------------------|
Expand Down
File renamed without changes.
44 changes: 42 additions & 2 deletions conformance-baseline-2026-07-28.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Expected server-conformance failures for spec version 2026-07-28.
# Expected conformance failures for spec version 2026-07-28 (server and client).
# These need protocol features that land in later tickets; remove each entry once its
# scenario passes. See server-conformance/README.md for how the baseline is used.
# scenario passes. See server-conformance/README.md and client-conformance/README.md.
server:
# multi-round-trip requests (tasks) — T5, https://github.com/softwaremill/chimp/issues/243
- tasks-capability-negotiation
Expand Down Expand Up @@ -31,3 +31,43 @@ server:
- sep-2164-resource-not-found
- input-required-result-missing-input-response
- input-required-result-ignore-extra-params
client:
# OAuth client flows — chimp's client has no full OAuth support yet
- auth/authorization-server-migration
- auth/basic-cimd
- auth/client-credentials-basic
- auth/client-credentials-jwt
- auth/dpop
- auth/dpop-nonce
- auth/enterprise-managed-authorization
- auth/iss-normalized
- auth/iss-not-advertised
- auth/iss-supported
- auth/iss-supported-missing
- auth/iss-unexpected
- auth/iss-wrong-issuer
- auth/metadata-default
- auth/metadata-issuer-mismatch
- auth/metadata-var1
- auth/metadata-var2
- auth/metadata-var3
- auth/offline-access-not-supported
- auth/offline-access-scope
- auth/pre-registration
- auth/scope-from-scopes-supported
- auth/scope-from-www-authenticate
- auth/scope-omitted-when-undefined
- auth/scope-retry-limit
- auth/scope-step-up
- auth/token-endpoint-auth-basic
- auth/token-endpoint-auth-none
- auth/token-endpoint-auth-post
- auth/wif-jwt-bearer
# modern (2026-07-28) client era: stateless lifecycle, request _meta, HTTP headers — T10-T12, #248/#249/#250
- tools_call
- http-custom-headers
- http-invalid-tool-headers
- json-schema-2020-12-preservation
- json-schema-ref-no-deref
- request-metadata
- sep-2322-client-request-state
34 changes: 34 additions & 0 deletions core/src/main/scala/chimp/protocol/Subscriptions.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package chimp.protocol

import io.circe.{Codec, Json}

/** The notification types a client opts into on a `subscriptions/listen` stream (2026-07-28). Each type is opt-in: the server MUST NOT send
* a type the client has not requested. `resourceSubscriptions` replaces the former `resources/subscribe` RPC.
*/
final case class SubscriptionFilter(
promptsListChanged: Option[Boolean] = None,
resourceSubscriptions: Option[List[String]] = None,
resourcesListChanged: Option[Boolean] = None,
toolsListChanged: Option[Boolean] = None
) derives Codec

/** Params of a `subscriptions/listen` request: the notification filter the client opts into on the stream. */
final case class SubscriptionsListenParams(notifications: SubscriptionFilter, _meta: Option[Map[String, Json]] = None) derives Codec

final case class SubscriptionsListenRequest(method: String = "subscriptions/listen", params: SubscriptionsListenParams) derives Codec

/** Result of `subscriptions/listen`, sent only when the server tears the subscription down gracefully (e.g. on shutdown); the body is
* otherwise empty. The subscription id travels in `_meta` under `io.modelcontextprotocol/subscriptionId`.
*/
final case class SubscriptionsListenResult(resultType: ResultType = ResultType.Complete, _meta: Option[Map[String, Json]] = None)
derives Codec

/** Params of the `notifications/subscriptions/acknowledged` notification: the subset of requested notification types the server will honor.
*/
final case class SubscriptionsAcknowledgedParams(notifications: SubscriptionFilter, _meta: Option[Map[String, Json]] = None) derives Codec

/** The `notifications/subscriptions/acknowledged` notification the server sends first on a listen stream. */
final case class SubscriptionsAcknowledgedNotification(
method: String = "notifications/subscriptions/acknowledged",
params: SubscriptionsAcknowledgedParams
) derives Codec
1 change: 1 addition & 0 deletions core/src/main/scala/chimp/protocol/Versioning.scala
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ object ProtocolMeta:
val ClientInfo: String = "io.modelcontextprotocol/clientInfo"
val ClientCapabilities: String = "io.modelcontextprotocol/clientCapabilities"
val ServerInfo: String = "io.modelcontextprotocol/serverInfo"
val SubscriptionId: String = "io.modelcontextprotocol/subscriptionId"

/** An `UnsupportedProtocolVersion` error (`-32022`) naming the versions the server supports, so the client can retry with one of them. */
def unsupportedVersionError(requested: String, supported: List[String]): JSONRPCErrorObject =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,37 @@ class Schema20260728ConformanceSpec extends SchemaConformance:
val msg: JSONRPCMessage =
JSONRPCMessage.Error(id = RequestId(1), error = ProtocolMeta.unsupportedVersionError("1900-01-01", List("2026-07-28", "2025-11-25")))
validate("UnsupportedProtocolVersionError", msg)

it should "produce a SubscriptionFilter that matches the spec schema" in:
validate(
"SubscriptionFilter",
SubscriptionFilter(resourceSubscriptions = Some(List("file:///x")), resourcesListChanged = Some(true), toolsListChanged = Some(false))
)

it should "produce SubscriptionsListenRequestParams that match the spec schema" in:
validate(
"SubscriptionsListenRequestParams",
SubscriptionsListenParams(
notifications = SubscriptionFilter(resourcesListChanged = Some(true)),
_meta = Some(Map(ProtocolMeta.ProtocolVersionKey -> "2026-07-28".asJson, ProtocolMeta.ClientCapabilities -> io.circe.Json.obj()))
)
)

it should "produce a SubscriptionsListenResult that matches the spec schema" in:
validate(
"SubscriptionsListenResult",
SubscriptionsListenResult(_meta =
Some(
Map(
ProtocolMeta.ServerInfo -> Implementation(name = "chimp", version = "1.0").asJson.deepDropNullValues,
ProtocolMeta.SubscriptionId -> "sub-1".asJson
)
)
)
)

it should "produce SubscriptionsAcknowledgedNotificationParams that match the spec schema" in:
validate(
"SubscriptionsAcknowledgedNotificationParams",
SubscriptionsAcknowledgedParams(notifications = SubscriptionFilter(promptsListChanged = Some(true)))
)
15 changes: 8 additions & 7 deletions server-conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,18 @@ The sbt task pins the harness to the version set in `conformanceHarnessV` in [`b

## How to run

Run the full requirement set:
Run a full requirement set (CI runs both revisions):

```bash
sbt 'serverConformance/conformance server --requirements 2025-11-25'
sbt 'serverConformance/conformance server --requirements 2026-07-28'
```

Run one scenario during development:
Run one scenario during development (pass the spec version):

```bash
sbt 'serverConformance/conformance server --scenario ping --spec-version 2025-11-25'
sbt 'serverConformance/conformance server --scenario server-initialize --spec-version 2025-11-25'
sbt 'serverConformance/conformance server --scenario server-discover --spec-version 2026-07-28'
```

The sbt task downloads `@modelcontextprotocol/conformance` with npm; `npx` must be available on the PATH.
Expand All @@ -41,10 +42,10 @@ and what result shape it wants. Remove the corresponding entry from the baseline

## The baseline file

[`conformance-baseline.yml`](../conformance-baseline.yml) lists scenarios that are known to fail today. When a
`conformance-baseline-<version>.yml` exists for the run's spec version (e.g.
[`conformance-baseline-2026-07-28.yml`](../conformance-baseline-2026-07-28.yml) for `--requirements 2026-07-28`), the sbt
task uses it instead — each spec revision keeps its own baseline. The harness uses the chosen file like this:
Each spec revision keeps its own baseline, selected from the run's `--requirements` (or `--spec-version`):
[`conformance-baseline-2025-11-25.yml`](../conformance-baseline-2025-11-25.yml) and
[`conformance-baseline-2026-07-28.yml`](../conformance-baseline-2026-07-28.yml). Each file lists the scenarios (in its
`server:` and `client:` sections) known to fail today. The harness uses the chosen file like this:

| Scenario result | In baseline? | Exit code | Meaning |
|-----------------|--------------|-----------|---------------------------------------|
Expand Down
Loading