Skip to content

http2: canceled upload leaves session memory accounting over budget #66742

Open

Description

@CoffeeFlux

Version

v26.10.0

Platform

Darwin miki.local 25.6.0 Darwin Kernel Version 25.6.0: Sat Jul 11 15:27:04 PDT 2026; root:xnu-12377.161.13~4/RELEASE_ARM64_T6050 arm64

Subsystem

http2

What steps will reproduce the bug?

Save the following code into a file and run it. The second GET request reports a stream error (ERR_HTTP2_STREAM_ERROR: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM) rather than responding with a 200.

// Cancel a queued upload, then reuse the session: node cancel-queued-body.mjs
import assert from 'node:assert/strict';
import { once } from 'node:events';
import http2 from 'node:http2';

const { NGHTTP2_CANCEL } = http2.constants;
// Keep the upload queued on the client until the server cancels it.
const server = http2.createServer({ settings: { initialWindowSize: 0 } });
const sessions = new Set();
server.on('session', (session) => sessions.add(session));
server.on('stream', (stream, headers) => {
  if (headers[':method'] === 'POST') {
    stream.close(NGHTTP2_CANCEL);
    return;
  }
  stream.respond({ ':status': 200 });
  stream.end();
});

// A timeout reports a hang; it does not control the reproduction's timing.
const timeout = setTimeout(() => {
  console.error('TIMEOUT');
  process.exit(1);
}, 5000);
let client;
try {
  server.listen(0, '127.0.0.1');
  await once(server, 'listening');
  client = http2.connect(`http://127.0.0.1:${server.address().port}`, {
    maxSessionMemory: 1,
  });
  // Wait until the client knows it cannot send any upload bytes.
  await once(client, 'remoteSettings');
  const get = () => new Promise((resolve) => {
    const req = client.request({ ':method': 'GET' }, { endStream: true });
    req.on('response', (headers) => resolve(`status ${headers[':status']}`));
    req.on('error', (err) => resolve(`${err.code}: ${err.message}`));
    req.resume();
  });

  console.log(`node ${process.version}`);
  const before = await get();
  console.log('small request before cancellation:', before);
  assert.equal(before, 'status 200');

  const upload = client.request({ ':method': 'POST' });
  upload.on('error', (err) => {
    assert.equal(err.code, 'ERR_HTTP2_STREAM_ABORTED');
  });
  const closed = new Promise((resolve) => upload.once('close', resolve));
  upload.write(Buffer.alloc(4 * 1024 * 1024));
  await closed;
  assert.equal(upload.rstCode, NGHTTP2_CANCEL);
  console.log('4 MiB queued upload: canceled by server (expected)');

  const after = await get();
  console.log('small request after cancellation: ', after);
  if (after !== 'status 200') process.exitCode = 1;
} finally {
  clearTimeout(timeout);
  client?.destroy();
  for (const session of sessions) session.destroy();
  server.close();
}

How often does it reproduce? Is there a required condition?

It's conditional on a stream being canceled while it has enough unsent body data queued to leave the session’s memory counter over maxSessionMemory after cleanup

What is the expected behavior? Why is that the expected behavior?

I'd expect both GET requests to return a 200

What do you see instead?

Opaque stream error: ERR_HTTP2_STREAM_ERROR: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM

Additional information

No response

Activity

  1. fatihguzeldev commented on Oct 15, 2026

    @fatihguzeldev

    still happens on main (38aa1f38819), even after the deferred cleanup runs.

    the cleanup clears the write queue but leaves the unsent bytes in the session memory count. should RemoveStream() subtract the remaining available_outbound_length_?

  2. emerzon commented on Oct 18, 2026

    @emerzon

    The missing release is still present at f88a1d25e9d. It also shows up on the server when a response is still queued.

    DoWrite charges the session through IncrementAvailableOutboundLength. Provider::OnRead releases that credit as nghttp2 pulls data. CompleteDestroyCleanup then drops whatever is left in the queue and never calls DecrementAvailableOutboundLength. RemoveStream runs before that drain, so subtracting there is the wrong place: OnRead may already have released credit for bytes that are still sitting in the queue. The amount to release is the remaining available_outbound_length_, next to the queue drop. That is what #66784 does. Its test covers a canceled upload. The same counter leak happens for a server response.

    Once current_session_memory_ stays above maxSessionMemory, CanAddStream refuses new streams with RST_STREAM / NGHTTP2_ENHANCE_YOUR_CALM (11). max_rejected_streams is 100. After 101 such refusals, OnBeginHeadersCallback returns NGHTTP2_ERR_CALLBACK_FAILURE and the session ends with GOAWAY INTERNAL_ERROR (2) / ERR_HTTP2_ERROR. The documented default maxSessionMemory is 10.

    Reproduced with a 256 KiB response, client initialWindowSize: 4096, and req.destroy() after 8 KiB:

    {"MSM":2,"firstReject":8,"rejected":101,"death":109,"goaway":2,"serverError":"ERR_HTTP2_ERROR"}
    {"MSM":10,"firstReject":40,"rejected":101,"death":141,"goaway":2,"serverError":"ERR_HTTP2_ERROR"}
    {"MSM":32,"firstReject":126,"rejected":101,"death":227,"goaway":2,"serverError":"ERR_HTTP2_ERROR"}
    

    A client that keeps the default window and reads the body does not hit this. nghttp2 drains the queue, and OnRead has already released the credit by the time the stream is destroyed.

    'use strict';
    const http2 = require('node:http2');
    
    const BODY = Buffer.alloc(256 * 1024, 0x78);
    const MSM = Number(process.env.MSM || 10);
    const server = http2.createServer({
      maxSessionMemory: MSM,
      settings: { initialWindowSize: 65535 },
    });
    let serverError = null;
    server.on('session', (session) => {
      session.on('error', (error) => { serverError = error.code; });
    });
    server.on('stream', (stream) => {
      stream.on('error', () => {});
      stream.respond({ ':status': 200 });
      stream.end(BODY);
    });
    
    server.listen(0, '127.0.0.1', async () => {
      const session = http2.connect(`http://127.0.0.1:${server.address().port}`, {
        settings: { initialWindowSize: 4096 },
      });
      let goaway = null;
      session.on('error', () => {});
      session.on('goaway', (code) => { goaway = code; });
    
      let firstReject = null;
      let rejected = 0;
      for (let i = 0; i < 500; i++) {
        if (session.destroyed || session.closed) break;
        const req = session.request({ ':path': `/${i}` });
        let rst = null;
        let got = 0;
        req.on('error', (error) => { rst = error.code; });
        req.on('data', (chunk) => {
          got += chunk.length;
          if (got > 8000) req.destroy();
        });
        await new Promise((resolve) => {
          req.on('close', resolve);
          setTimeout(resolve, 1500);
        });
        if (rst === 'ERR_HTTP2_STREAM_ERROR') {
          rejected++;
          if (firstReject === null) firstReject = i;
        }
        if (goaway !== null) {
          console.log(JSON.stringify({
            MSM, firstReject, rejected, death: i, goaway, serverError,
          }));
          break;
        }
      }
      session.destroy();
      server.close();
    });
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions