Skip to content

fix(gateway): retire dweb.link before it is switched off on 2026-09-21 #124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ehsan6sha merged 1 commit into from
Sep 12, 2026
Merged

fix(gateway): retire dweb.link before it is switched off on 2026-09-21 #124

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 27 additions & 10 deletions cloudflare/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,27 @@ GET https://fxfiles.top/w/<ipnsName>/page.html -> 302 https://<cid>.ipfs.dweb.l
An optional `?gw=` selects which gateway the redirect lands on:

```
GET /w/<ipnsName>?gw=dweb -> 302 https://<cid>.ipfs.dweb.link/
GET /w/<ipnsName>?gw=filebase -> 302 https://ipfs.filebase.io/ipfs/<cid>
GET /w/<ipnsName>?gw=filebase -> 302 https://ipfs.filebase.io/ipfs/<cid> (default)
GET /w/<ipnsName>?gw=fx -> 302 https://ipfs.cloud.fx.land/gateway/<cid>
```

It exists because **dweb.link returns HTTP 429 once a site sees real traffic**
(measured 2026-09-12: a freshly generated site 429'd on dweb.link and returned
200 from Filebase at the same moment). The app appends this automatically from
the gateway chosen in Settings, so a user who switches gets working links
without re-minting anything.
The app appends this automatically from the gateway chosen in Settings, so a
user who switches gets working links without re-minting anything.

### dweb.link is retired — do not re-add it

The IPFS Foundation **shut dweb.link down permanently on 2026-09-21**
(gatewaychanges.ipfs.io). The HTTP 429s seen beforehand, with a `Retry-After` of
around half an hour, were its announced escalating pauses — not load.

`dweb` is therefore absent from `GATEWAYS`, and that is deliberate in a way
worth spelling out: links minted while dweb was the default carry an **explicit**
`?gw=dweb`, and an explicit key normally beats the default. But that "choice"
was manufactured by the default rather than made by anyone, so honouring it
would send those links to a dead host. Dropping the key makes them fall back to
the default instead. The app makes the matching move — `IpfsGatewayHelper`
lists the dweb template in `retiredTemplates`, which migrates any user still
holding it onto the current default at startup.

`GATEWAYS` in the Worker is a **fixed allowlist**, and that is load-bearing:
this is a link anyone can share, so accepting a caller-supplied destination
Expand Down Expand Up @@ -93,9 +105,14 @@ If you deploy to a different host, set the secure-storage key

- Redirect is **302** (never 301) with `Cache-Control: max-age=30`, so a
regeneration propagates within ~30s while still allowing edge caching.
- The `dweb` gateway assumes CIDv1 (a CIDv0 `Qm…` cannot go in a subdomain —
hostnames are case-insensitive and base58 is not). `filebase` is path-style
and has no such constraint. The app mints CIDv1, so this is not a live issue.
- Both current gateways are **path-style**, so CID encoding is a non-issue. A
subdomain-style gateway would need the guard back: a CIDv0 (`Qm…`, base58 and
case-sensitive) or a CID over the 63-character DNS label limit silently
corrupts when used as a hostname, but is fine in a path.
- The Worker rejects paths whose name isn't a plausible `k51…` IPNS name, and
charset-checks the CID before interpolating it, so it can't be abused as an
open redirector.
- When w3name is unreachable the Worker returns a plain **502**. It used to
redirect to `{name}.ipns.dweb.link`, which never resolved (w3name does not
publish to the DHT) and is now a dead host — redirecting there only turned our
error into a more confusing one.
116 changes: 59 additions & 57 deletions cloudflare/ipns-resolver-worker.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,27 @@
* FxFiles stable-link resolver — a STATELESS Cloudflare Worker that is the
* fast, pretty front door over each website group's IPNS name.
*
* GET https://fxfiles.top/w/<ipnsName>[/<subpath>][?gw=dweb|filebase]
* GET https://fxfiles.top/w/<ipnsName>[/<subpath>][?gw=filebase|fx]
* -> resolve <ipnsName> to its current CID via w3name's plain HTTP API
* -> 302 to that gateway's URL for the CID, e.g.
* https://<cid>.ipfs.dweb.link/<subpath> (gw=dweb, default)
* https://ipfs.filebase.io/ipfs/<cid>/<subpath> (gw=filebase)
* https://ipfs.filebase.io/ipfs/<cid>/<subpath> (gw=filebase, default)
* https://ipfs.cloud.fx.land/gateway/<cid>/<subpath> (gw=fx)
*
* Why this design:
* - The app never talks to Cloudflare and holds NO credential here. The IPNS
* name is the source of truth; this Worker only *reads* the public w3name
* record and redirects. Anyone can redeploy it; losing it loses nothing.
* - Resolving through w3name's HTTP API is fast (no DHT wait) and lands on the
* immutable per-CID URL, which gateways cache aggressively.
* - If w3name is slow/unavailable, we fall back to the raw IPNS gateway URL.
* NOTE (measured 2026-05-30): that fallback only resolves if the record is
* ALSO published to the IPFS DHT — w3name does NOT do that, so today a bare
* {name}.ipns.dweb.link does NOT resolve on a plain gateway (it 500s). So
* name->CID resolution currently depends on this Worker reading w3name (both
* non-fx). The CONTENT (CID) IS fully public-reachable via IPFS gateways
* (verified 200). Net: the link survives fx being down, but not Cloudflare +
* w3name both being down. See README for the optional DHT-publish step that
* makes any gateway work.
* - When w3name is unreachable we return a plain 502 rather than redirecting
* anywhere. There used to be a fallback to {name}.ipns.dweb.link; it never
* actually resolved (w3name does not publish to the DHT, so a bare name 500s
* on a plain gateway — measured 2026-05-30) and that host is switched off for
* good on 2026-09-21 anyway. So name->CID resolution depends on this Worker
* reading w3name, both non-fx. The CONTENT (CID) stays fully public-reachable
* via any IPFS gateway. Net: a link survives fx being down, but not
* Cloudflare + w3name both being down. See README for the optional
* DHT-publish step that would make any gateway resolve the name directly.
*
* Abuse posture (it MUST stay publicly reachable so links + previews work):
* - Not an open redirector: the destination host comes from a FIXED allowlist
Expand All @@ -37,12 +37,11 @@
* (only active if `RW_LIMITER` is configured in wrangler.toml). Pair with a
* dashboard WAF Rate Limiting Rule on `/w/*` for global enforcement.
*
* Cloudflare's own IPFS gateway was decommissioned in Aug 2024 — irrelevant
* here; this Worker `fetch()`es the IPFS Foundation gateways (dweb.link/ipfs.io).
* Gateway churn is the norm, which is why the destination is a one-line change
* here rather than a property of published content: Cloudflare retired its IPFS
* gateway in Aug 2024, and the IPFS Foundation retires dweb.link on 2026-09-21.
*/

const GATEWAY_HOST = 'ipfs.dweb.link';
const IPNS_GATEWAY_HOST = 'ipns.dweb.link';
const W3NAME_ENDPOINT = 'https://name.web3.storage';
const REDIRECT_CACHE_SECONDS = 30; // keep short so regenerations propagate fast
const MAX_NAME_LEN = 80; // base36 `k51…` libp2p-key names are ~62 chars
Expand All @@ -52,20 +51,6 @@ const CID_RE = /^[A-Za-z0-9]{40,120}$/;
// eslint-disable-next-line no-control-regex
const CONTROL_CHARS = /[\u0000-\u001f\u007f]/;

/**
* Can this CID be a DNS label, i.e. is the subdomain gateway shape usable?
*
* Two ways it cannot, both of which silently corrupt the CID rather than
* failing loudly:
* - CIDv0 (`Qm…`) is base58 and CASE-SENSITIVE, but hostnames are not — the
* resolver lowercases the label and the gateway receives a different CID.
* - A DNS label caps at 63 characters (RFC 1035). CIDv1-base32 over SHA-256
* is 59, but a larger hash function would overrun it.
* Either way the answer is the same: use the gateway's path form, which
* preserves case and has no length limit.
*/
const SUBDOMAIN_SAFE_CID = /^[a-z0-9]{1,63}$/;

/**
* Gateways this Worker may redirect to, selected with `?gw=<key>`.
*
Expand All @@ -75,23 +60,28 @@ const SUBDOMAIN_SAFE_CID = /^[a-z0-9]{1,63}$/;
* open redirector, which is exactly the property the checks below exist to
* protect. Unknown or missing `gw` falls back to the default.
*
* `cid` builds the immutable per-CID URL in whichever shape the gateway wants:
* subdomain — https://<cid>.ipfs.dweb.link/<path>
* path — https://ipfs.filebase.io/ipfs/<cid>/<path>
* `dweb` IS DELIBERATELY ABSENT. The IPFS Foundation switched dweb.link off for
* good on 2026-09-21. Links minted while it was the default carry an explicit
* `?gw=dweb`, and an explicit key would normally beat the default — but that
* "choice" was manufactured by the default rather than made by anyone, so
* honouring it would send those links to a dead host. Dropping the key makes
* them fall back here instead, which is the whole point.
*
* Both remaining gateways are PATH-style (`https://host/ipfs/<cid>/<path>`), so
* there is no subdomain-safety problem to handle. A subdomain gateway would need
* that guard back: a case-sensitive CIDv0 (`Qm…`) or a CID over the 63-character
* DNS label limit silently corrupts as a hostname, but is fine in a path.
*/
const GATEWAYS = {
dweb: {
cid: (cid, path) =>
SUBDOMAIN_SAFE_CID.test(cid)
? `https://${cid}.${GATEWAY_HOST}${path}`
: `https://dweb.link/ipfs/${cid}${path}`,
},
filebase: {
// dweb.link starts returning 429 once a site sees real traffic; Filebase
// served the same CID fine at the same moment (measured 2026-09-12).
// Path-style, so it needs no subdomain-safety dance.
// Served the same CID fine at the moment dweb.link was 429ing it
// (measured 2026-09-12).
cid: (cid, path) => `https://ipfs.filebase.io/ipfs/${cid}${path}`,
},
fx: {
// Ours. Verified 2026-09-12 to serve these CIDs with correct content types.
cid: (cid, path) => `https://ipfs.cloud.fx.land/gateway/${cid}${path}`,
},
};

/**
Expand All @@ -111,7 +101,7 @@ function joinPath(inner, subpath) {
}

/** Default when `?gw=` is absent — keeps every already-shared link working. */
const DEFAULT_GATEWAY = 'dweb';
const DEFAULT_GATEWAY = 'filebase';

export default {
async fetch(request, env) {
Expand Down Expand Up @@ -167,26 +157,24 @@ export default {
forwarded.delete('gw');
const query = forwarded.toString() ? `?${forwarded}` : '';

// Unknown keys fall back rather than erroring: a link with a typo should
// still resolve, just on the default gateway.
// Unknown keys fall back rather than erroring: a link with a typo — or a
// retired key like `gw=dweb` — should still resolve, just on the default.
//
// hasOwn, NOT a bare `GATEWAYS[gwKey] ||` — gwKey is caller-controlled and
// a plain object literal inherits from Object.prototype, so `?gw=toString`
// and `?gw=__proto__` would hand back a TRUTHY inherited value whose `.cid`
// is undefined. That throws inside the try below and drops the request on
// the IPNS fallback, which (see the note at the top) does not resolve. The
// typo would break the link instead of quietly using the default.
// is undefined. That throws inside the try below and the request ends as a
// 502, so the typo would break the link instead of using the default.
const gateway = Object.hasOwn(GATEWAYS, gwKey ?? '')
? GATEWAYS[gwKey]
: GATEWAYS[DEFAULT_GATEWAY];

// The happy path below never uses an IPNS gateway: w3name resolves the
// name here and we redirect to the immutable /ipfs/<cid>, which every
// gateway serves. This fallback only runs when w3name is unreachable, and
// per the note at the top it does not resolve anyway (the record is not on
// the DHT). Left on dweb because it is the only host that would even try.
const ipnsFallback =
`https://${name}.${IPNS_GATEWAY_HOST}${subpath}${query}`;
// NOTE: there is no IPNS-gateway fallback any more. It pointed at
// ipns.dweb.link, which (a) never resolved anyway — w3name does not publish
// to the DHT, so a bare name 500s on a plain gateway — and (b) is being
// switched off entirely on 2026-09-21. Redirecting a visitor to a host that
// is guaranteed to fail just turns our error into someone else's confusing
// one, so when w3name is unreachable we now say so plainly instead.

try {
const res = await fetch(`${W3NAME_ENDPOINT}/name/${name}`, {
Expand Down Expand Up @@ -219,10 +207,24 @@ export default {
}
}
} catch (_) {
// fall through to the IPNS gateway fallback
// fall through to the plain error below
}

return redirect(ipnsFallback);
// Reached when w3name is unreachable, returns a non-OK, or hands back a
// value that is not a usable `/ipfs/<cid>`. 502, because the failure is
// upstream of us and the visitor's link is probably fine.
return new Response(
'Could not resolve this FxFiles link right now. The IPNS name service ' +
'is unreachable — please try again shortly.',
{
status: 502,
headers: {
'content-type': 'text/plain; charset=utf-8',
'Cache-Control': 'no-store',
'Retry-After': '30',
},
},
);
},
};

Expand Down
Binary file modified cloudflare/ipns-resolver-worker.test.mjs
View file
Open in desktop
Binary file not shown.
90 changes: 64 additions & 26 deletions lib/core/services/ipfs_gateway_helper.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,25 +12,41 @@ import 'package:fula_files/core/services/secure_storage_service.dart';
class IpfsGatewayHelper {
IpfsGatewayHelper._();

/// Subdomain-style dweb.link template, used as the app-wide default.
static const String defaultTemplate = 'https://{cid}.ipfs.dweb.link/';
/// Subdomain-style dweb.link template. The app-wide default until
/// 2026-09-12, now RETIRED: the IPFS Foundation is shutting this gateway
/// down for good on 2026-09-21 (gatewaychanges.ipfs.io), and the HTTP 429s
/// seen beforehand are its announced escalating pauses, not load. Kept as a
/// constant ONLY so [init] can recognise and migrate anyone still on it.
static const String dwebTemplate = 'https://{cid}.ipfs.dweb.link/';

/// Path-style Filebase gateway, and the app-wide default since dweb's
/// retirement — measured 2026-09-12, a site that returned 429 from dweb.link
/// returned 200 from Filebase for the same CID at the same moment.
static const String filebaseTemplate = 'https://ipfs.filebase.io/ipfs/';

/// Pre-v0.4 default. Anyone still on this exact value is upgraded to
/// [defaultTemplate] on the next [init].
static const String legacyDefault = 'https://ipfs.cloud.fx.land/gateway/';
/// fx's own gateway. This was the pre-v0.4 default, and [init] used to
/// migrate people AWAY from it and onto dweb — that migration is gone,
/// because the destination is now the thing that is dying. It is offered as
/// a first-class preset again: it serves these CIDs with correct content
/// types (verified 2026-09-12) and, unlike any third party, it is ours.
static const String fxTemplate = 'https://ipfs.cloud.fx.land/gateway/';

/// Path-style Filebase gateway. Offered as a preset because dweb.link
/// rate-limits (HTTP 429) once a site gets any real traffic — measured
/// 2026-09-12, a freshly generated site returned 429 from dweb.link and
/// 200 from Filebase at the same moment.
static const String filebaseTemplate = 'https://ipfs.filebase.io/ipfs/';
static const String defaultTemplate = filebaseTemplate;

/// Templates that are dead or dying. A stored value matching one of these is
/// replaced with [defaultTemplate] on the next [init] — deliberately
/// overriding what looks like a user's choice, because for most people the
/// "choice" was just the old default, and leaving it would hand them a
/// broken site. Match-and-replace is idempotent, so no migration flag.
static const Set<String> retiredTemplates = <String>{dwebTemplate};

/// The presets the settings picker offers, in display order. Anything
/// else the user types is "Custom" — [buildUrl] accepts any template in
/// either of the two supported shapes.
/// either of the two supported shapes. dweb is deliberately ABSENT: offering
/// a gateway that [init] would migrate away from on next launch is a trap.
static const Map<String, String> presets = <String, String>{
'dweb.link': defaultTemplate,
'Filebase': filebaseTemplate,
'fx.land': fxTemplate,
};

/// Preset label for [template], or null when it is a custom value.
Expand All @@ -51,8 +67,8 @@ class IpfsGatewayHelper {
/// asset URLs written into the site (client-side, no such risk), while the
/// stable link falls back to the resolver's default.
static const Map<String, String> _frontDoorKeys = <String, String>{
defaultTemplate: 'dweb',
filebaseTemplate: 'filebase',
fxTemplate: 'fx',
};

static String? frontDoorGatewayKey([String? template]) =>
Expand All @@ -67,12 +83,18 @@ class IpfsGatewayHelper {
/// exact staleness that made the setting look inert for asset URLs.
///
/// A preset emits its key even when it matches the resolver's own default,
/// rather than leaving the link bare. Omitting it would read as "no
/// opinion", and the resolver is then free to send the link somewhere else
/// if its default ever moves — but a user who picked dweb.link in Settings
/// HAS an opinion, and it should survive that. Bare links stay reserved for
/// callers that genuinely have none (custom gateways, which the resolver
/// cannot honour anyway).
/// rather than leaving the link bare, so that an explicit choice survives a
/// later change of that default. Bare links stay reserved for callers that
/// genuinely have no opinion (custom gateways, which the resolver cannot
/// honour anyway).
///
/// CAVEAT, learned the hard way when dweb.link was retired: freezing the
/// gateway into a copied link cuts both ways. Every link copied while dweb
/// was the default carries `?gw=dweb`, and that key had to be dropped from
/// the resolver's allowlist so those links would fall back instead of
/// pointing at a dead host. Decorating is right while the set of live
/// gateways is stable; once a per-site preference exists, prefer bare links
/// so they keep following the owner's current choice.
static String decorateFrontDoorUrl(String frontDoorUrl, {String? template}) {
final key = frontDoorGatewayKey(template);
if (key == null || frontDoorUrl.isEmpty) return frontDoorUrl;
Expand All @@ -87,21 +109,37 @@ class IpfsGatewayHelper {
static String get cachedTemplate => _cachedTemplate;

/// Run after [SecureStorageService.init] and before any consumer reads
/// the gateway. Performs the one-time replacement of the legacy default
/// — match-and-replace is naturally idempotent, so no migration flag.
/// the gateway.
///
/// Note this WRITES on first run, which is why retiring a default is not
/// just a matter of changing the constant: every user who has ever launched
/// the app has the then-current default persisted, so a new [defaultTemplate]
/// would reach new installs only. [retiredTemplates] is what actually moves
/// existing users off a dead gateway.
static Future<void> init() async {
final stored = await SecureStorageService.instance
.read(SecureStorageKeys.ipfsGatewayUrl);

if (stored == null || stored.isEmpty || stored == legacyDefault) {
final resolved = resolveStoredTemplate(stored);
if (resolved != stored) {
await SecureStorageService.instance.write(
SecureStorageKeys.ipfsGatewayUrl,
defaultTemplate,
resolved,
);
_cachedTemplate = defaultTemplate;
} else {
_cachedTemplate = stored;
}
_cachedTemplate = resolved;
}

/// The template [init] should end up with, given what is currently stored.
///
/// Split out as a pure function so the migration is testable without a
/// storage backend — it is the part that decides whether a user keeps
/// working after a gateway is retired, which is worth covering directly.
static String resolveStoredTemplate(String? stored) {
if (stored == null || stored.trim().isEmpty) return defaultTemplate;
final trimmed = stored.trim();
if (retiredTemplates.contains(trimmed)) return defaultTemplate;
return trimmed;
}

/// Refresh the in-memory cache after the user saves a new value in
Expand Down
Loading
Loading