Add vortex-server + vortex-demo: live HTTP range demo

demo/README.md

@@ -0,0 +1,144 @@
+# Vortex HTTP range demo
+
+Shows that a filtered, projected `VortexHttpReader` scan over HTTP fetches only the bytes it
+actually touches — not the whole file — using nothing but plain HTTP `Range` requests. No cloud
+account, no query service: just a `.vortex` file sitting behind a minimal object-storage server.
+
+Three standalone tools:
+
+| Module | Artifact | What it does |
+|---|---|---|
+| `fakedata-generator` | `vortex-fakedata-generator.jar` | Generates a synthetic `.vortex` file from a compact column-description grammar |
+| `server` | `vortex-server.jar` | Minimal object-storage HTTP server — `GET`/`HEAD` with byte-range support, `PUT`, listing |
+| `client` | `vortex-demo.jar` | Uploads a file, runs a time-range-filtered/projected scan, reports bytes fetched vs. the file's full size |
+
+## Build once
+
+```bash
+./mvnw package -pl demo/server,demo/fakedata-generator,demo/client -am -DskipTests
+```
+
+Produces:
+
+```
+demo/server/target/vortex-server.jar
+demo/fakedata-generator/target/vortex-fakedata-generator.jar
+demo/client/target/vortex-demo.jar
+```
+
+## Run the live demo (two terminals)
+
+**Terminal 1 — start the server** (serves whatever directory you point it at, on whatever port):
+
+```bash
+java -jar demo/server/target/vortex-server.jar 8080 /tmp/vortex-demo-data
+```
+
+Leave it running — it logs every request it serves, which is the point: you'll watch it print a
+handful of small `Range` fetches instead of one big download.
+
+**Terminal 2 — generate the dataset:**
+
+```bash
+java -jar demo/fakedata-generator/target/vortex-fakedata-generator.jar \
+    --rows 2000000 --out /tmp/trades.vortex \
+    "timestamp:i64:series(1700000000000,1000)" \
+    "symbol:utf8:enum(SYM,30)" \
+    "price:f64:range(50,150)" \
+    "volume:i64:range(100,10000)"
+```
+
+Prints a live progress bar with ETA to stderr while writing (throttled, so it won't flood the
+terminal — only really visible on larger row counts). 2,000,000 rows, real compression
+(`cascading(3)`, the generator's default) — no clustering trick needed. The `timestamp` column is
+a `series(...)`, so it's naturally in row order simply because that's the order it was generated
+in, exactly like a real append-only ingestion stream.
+
+**Terminal 2 — upload it, then query it:**
+
+```bash
+java -jar demo/client/target/vortex-demo.jar --upload /tmp/trades.vortex http://127.0.0.1:8080/
+java -jar demo/client/target/vortex-demo.jar http://127.0.0.1:8080/trades.vortex
+```
+
+The first command just copies the file to the server and exits — no query. The second queries the
+object directly by URL: no local file, no upload, it's already there. Run the second command again
+with different `--range`/`--project` values to try other queries against the same uploaded object
+without re-uploading it.
+
+By default this filters `timestamp` to a narrow window (50,000 of the 2,000,000 rows) and
+projects `price` — a realistic "give me this time range" query, not an equality match on some
+other column. While the scan runs, a live "bytes downloaded so far" counter updates in place on
+stderr — the audience watches it climb a little, then stop well short of the file's full size,
+rather than just seeing a single number appear at the end. It's mostly visible on larger row
+counts, since a small scan can finish before the first redraw.
+
+Expected output (numbers will vary slightly with row count):
+
+```
+Scanning for 1701000000000 <= timestamp <= 1701049999000, projecting 'price' over HTTP...
+
+  Downloaded so far: 401,644 / 24,552,440 bytes (1.6%)
+Matched rows: 131072
+Bytes fetched over HTTP during the scan: 1,363,118 / 24,552,440 (5.55% of the object)
+```
+
+Switch back to **Terminal 1** — you'll see the `PUT` (the upload) followed by a small number of
+`GET ... range=bytes=...` lines, each a few hundred KB, not one 24 MB download.
+
+## One-terminal version (no server to manage)
+
+Pass a local file path instead of a URL and the client embeds its own server, uploads to it, and
+queries it in one shot:
+
+```bash
+java -jar demo/client/target/vortex-demo.jar /tmp/trades.vortex
+```
+
+Good for a quick local check; the two-terminal version is more compelling live, since the
+audience can watch the server's request log update in real time.
+
+## Customizing the story
+
+- **Different filter/projection** — `vortex-demo` accepts `--range COLUMN:MIN:MAX` and `--project`
+  (numeric-range filter, not equality — see below for why). Match these to whatever schema you
+  generate, e.g. `--range price:100:105 --project timestamp`.
+- **Different dataset shape** — `vortex-fakedata-generator`'s column grammar is
+  `name:type:generator(args)`. Run it with no arguments for the full grammar reference
+  (types, generators, an example). The `series(start,step)` generator is a direct nod to SQL's
+  `generate_series`.
+- **Why a range filter on `timestamp`, not equality on `symbol`** — there is deliberately no
+  "sort the rows before writing" option anywhere in this demo: real data never arrives pre-sorted
+  by whatever column a later query happens to filter on, so faking that clustering would
+  misrepresent the workload this is meant to demonstrate. A `series(...)` column is naturally
+  ordered by row position without any sorting, which zone-map pruning can exploit for a
+  range query; an `enum(...)` column's values are scattered uniformly across every chunk by
+  design, so an equality filter on it (e.g. `symbol == "SYM015"`) has nothing to prune — every
+  chunk could contain a match.
+- **Why filtering `price` outside its range now prunes fully, but overlapping it still doesn't** —
+  a `range()`-generated `f64` column has no natural clustering by row position (unlike `series(...)`
+  `timestamp`), so any filter that *overlaps* its data range still touches most chunks — that part
+  is expected, not a bug. A filter *entirely outside* the range (e.g. `[5, 6]` when every value is
+  in `[50, 150)`) is different: with #382 fixed, ALP-RD now computes zone-map `min`/`max`, so that
+  case prunes down to a couple of stray range fetches (well under 1% of the file) instead of the
+  ~54% it used to fetch when ALP-RD reported no stats at all.
+
+## Bugs this demo surfaced
+
+Building this surfaced four real gaps in zone-map pruning in vortex-java's `reader`/`writer`
+modules — all four are now fixed:
+
+- [#378](https://github.com/dfa1/vortex-java/issues/378) — `WriteOptions`'s default
+  `globalDict=true` silently defeated zone-map pruning for `Utf8` columns.
+- [#379](https://github.com/dfa1/vortex-java/issues/379) — a cascade-selected encoder
+  (`cascading(depth) > 0`) dropped zone-map min/max stats, keeping only `sum`.
+- [#380](https://github.com/dfa1/vortex-java/issues/380) — checking whether an HTTP-backed chunk
+  could be pruned fetched that chunk's *entire* segment first, costing as much bandwidth as just
+  reading it.
+- [#382](https://github.com/dfa1/vortex-java/issues/382) — `AlpRdEncodingEncoder` never emitted
+  zone-map min/max stats at all, so any column the cascade routed through ALP-RD (typically `f64`)
+  never pruned, independent of the filter (see above).
+
+This demo's numbers reflect the fixed behavior for #378-#380 and #382. If you're running against
+an older vortex-java build, pruning may not work at all and the byte percentage will be much higher
+than shown above.

demo/client/pom.xml

@@ -0,0 +1,90 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>io.github.dfa1.vortex</groupId>
+        <artifactId>vortex-java</artifactId>
+        <version>0.14.2-SNAPSHOT</version>
+        <relativePath>../../pom.xml</relativePath>
+    </parent>
+
+    <artifactId>vortex-demo</artifactId>
+
+    <name>vortex-demo</name>
+    <description>Standalone demo client: uploads an existing Vortex file (e.g. one produced by
+        vortex-fakedata-generator) to a vortex-server (embedded by default, or an already-running
+        one given on the command line), then runs a filtered/projected VortexHttpReader scan
+        against it -- showing that the scan fetches only the bytes it touches instead of
+        downloading the whole object.</description>
+
+    <properties>
+        <!-- Demo/presentation tooling, not a consumable library: never published to Maven
+             Central. maven.deploy.skip alone is not enough: central-publishing-maven-plugin takes
+             over the deploy phase via <extensions>true</extensions> in the parent's release
+             profile and runs regardless, with its own skip switch (see 74780aa3). -->
+        <maven.deploy.skip>true</maven.deploy.skip>
+        <skipPublishing>true</skipPublishing>
+        <!-- Demo/presentation tooling: main(String[] args) and System.out logging dominate the
+             Sonar findings here with no library-quality signal, same rationale as
+             vortex-performance. -->
+        <sonar.skip>true</sonar.skip>
+    </properties>
+
+    <dependencies>
+        <!-- production -->
+        <dependency>
+            <groupId>io.github.dfa1.vortex</groupId>
+            <artifactId>vortex-reader</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>io.github.dfa1.vortex</groupId>
+            <artifactId>vortex-server</artifactId>
+        </dependency>
+        <!-- testing -->
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.assertj</groupId>
+            <artifactId>assertj-core</artifactId>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-shade-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>shade</goal>
+                        </goals>
+                        <configuration>
+                            <finalName>vortex-demo</finalName>
+                            <transformers>
+                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+                                    <mainClass>io.github.dfa1.vortex.demo.client.HttpRangeDemo</mainClass>
+                                </transformer>
+                            </transformers>
+                            <filters>
+                                <filter>
+                                    <artifact>*:*</artifact>
+                                    <excludes>
+                                        <exclude>META-INF/*.SF</exclude>
+                                        <exclude>META-INF/*.DSA</exclude>
+                                        <exclude>META-INF/*.RSA</exclude>
+                                    </excludes>
+                                </filter>
+                            </filters>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+</project>