brnil6 · Oct 5, 2026
diff --git a/ai/generative-ai-service/tts-model-benchmarking/LICENSE b/ai/generative-ai-service/tts-model-benchmarking/LICENSE
@@ -0,0 +1,32 @@
+Copyright (c) 2026 Oracle and/or its affiliates.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any
+person obtaining a copy of this software, associated documentation and/or data
+(collectively the "Software"), free of charge and under any and all copyright
+rights in the Software, and any and all patent rights owned or freely
+licensable by each licensor hereunder covering either (i) the unmodified
+Software as contributed to or provided by such licensor, or (ii) the Larger
+Works (as defined below), to deal in both
+
+(a) the Software, and
+
+(b) any piece of software and/or hardware listed in the lrg_coverage.txt file,
+if one is included with the Software (each a "Covered Work", collectively
+"Covered Works"), that is available under a license listed in the
+lrg_coverage.txt file, if one is included with the Software (the "Covered
+License"), and
+
+without limitation, the right to copy, create derivative works of, display,
+perform, and distribute the Software and make, use, sell, offer for sale,
+import, export, have made, and have sold the Software and the Covered Works,
+in each case subject to the conditions and limitations set forth herein.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/ai/generative-ai-service/tts-model-benchmarking/README.md b/ai/generative-ai-service/tts-model-benchmarking/README.md
@@ -0,0 +1,75 @@
+# TTS Model Benchmarking on OCI Generative AI
+
+*A benchmark toolkit for text-to-speech models hosted on OCI Generative AI (imported models such as OmniVoice on a dedicated AI cluster, or on-demand models on the same speech route). It measures time to first audio, end-to-end latency, real-time factor, throughput and error rate under concurrency, with and without streaming, in English and Arabic, and scores intelligibility with Whisper to find the input length at which single-pass synthesis degrades.*
+
+Author: Brona Nilsson
+
+Reviewed: 05.10.2026
+
+# When to use this asset?
+
+*Use this asset when you need numbers, not impressions, about a TTS endpoint on OCI: how fast it answers, how much load one hosting unit takes, and whether long inputs stay intelligible.*
+
+### Who
+- Data scientists and ML engineers evaluating a TTS model in an OCI Data Science notebook session
+- Solution architects sizing a dedicated AI cluster (A10, A100, H100) for a voice workload
+- Teams comparing hosting shapes, model versions or serving-stack upgrades with repeatable runs
+
+### When
+- Choosing a hosting shape: run the same suites on each shape and compare the CSVs
+- Designing a voice application: pick streaming vs chunking from measured TTFA, not guesswork
+- Validating a fix: OmniVoice served through vLLM-Omni on OCI garbles speech past ~30 s per request; the `long-text` suite shows the breaking point as a WER curve and confirms when an upgrade removes it
+- Capacity planning: the concurrency sweep finds the knee where latency climbs and throughput flattens
+
+# How to use this asset?
+
+*Point the scripts at an endpoint, run a suite, read the report.*
+
+1. `pip install -r files/requirements.txt`, copy `files/.env.example` to `files/.env` and set the endpoint and compartment OCIDs. In a Data Science notebook session the resource principal is picked up automatically.
+2. `python files/tts_bench.py --suite quick --label <shape>` for a two-minute smoke test, then `--suite latency`, `--suite long-text`, `--suite concurrency` or `--suite full`.
+3. `python files/wer.py results/<run>` to score the saved clips for intelligibility, and `python files/plots.py results/<run>` for charts. Or open `files/notebook/tts_benchmark.ipynb` and run the cells.
+
+See the detailed [README](files/README.md) in the `files/` folder for setup on each platform, the metric definitions, the suites, reference numbers from an H100_X1 cluster and troubleshooting.
+
+### Key Capabilities
+- Industry-standard serving metrics: TTFA, E2E, RTF, p50/p90/p95/p99, error rate, requests per minute and audio seconds per second
+- Three request modes: synchronous, streaming, and client-side sentence chunking (the workaround for long inputs)
+- Closed-loop concurrency sweep (1, 2, 4, 8, 16 parallel requests) to find the capacity of one hosting unit
+- Target durations in seconds (10 to 60 s) with per-language calibration of characters per second, so English and Arabic are compared at equal speech length
+- Whisper-based WER/CER scoring with Arabic normalisation; CPU, CUDA or Apple Silicon backends
+- Constant voice across all requests through a bundled synthetic reference clip; voice design and preset-voice modes for other models
+- Works with API key, resource principal, instance principal, session token or bearer-key auth
+- CSV, Markdown report, sample audio and PNG charts per run; runs are labelled by shape for side-by-side comparison
+
+### File Structure
+```
+.
+├── README.md                     # This file
+├── LICENSE
+└── files/
+    ├── README.md                 # Detailed guide: setup, metrics, suites, reference numbers
+    ├── tts_bench.py              # The benchmark
+    ├── wer.py                    # Intelligibility scoring (Whisper WER / CER)
+    ├── plots.py                  # Charts from a run folder
+    ├── tts_client.py             # Client for the OCI speech route with OCI signing
+    ├── requirements.txt
+    ├── .env.example
+    ├── notebook/tts_benchmark.ipynb
+    ├── texts/                    # en_science, en_report, ar_coffee, ar_city
+    ├── voices/ref_voice.wav      # synthetic reference voice + transcript
+    └── results/                  # one folder per run (git-ignored)
+```
+
+# Useful Links
+
+- [OCI Generative AI documentation](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm)
+- [Importing K2-FSA OmniVoice models](https://docs.oracle.com/en-us/iaas/Content/generative-ai/imported-k2-fsa-models.htm)
+- [OCI OpenAI-compatible API](https://docs.oracle.com/en-us/iaas/Content/generative-ai/openai-compatible-api.htm)
+- [OCI Data Science notebook sessions](https://docs.oracle.com/en-us/iaas/data-science/using/use-notebook-sessions.htm)
+
+# License
+
+Copyright (c) 2026 Oracle and/or its affiliates.
+Licensed under the Universal Permissive License (UPL), Version 1.0.
+
+See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE.txt) for more details.
diff --git a/ai/generative-ai-service/tts-model-benchmarking/files/.env.example b/ai/generative-ai-service/tts-model-benchmarking/files/.env.example
@@ -0,0 +1,25 @@
+# Copy to .env and fill in. Nothing here is a secret unless you choose the bearer-key route.
+
+# The TTS endpoint to benchmark: a dedicated endpoint OCID (imported model on a DAC)
+# or, for on-demand models that answer on the same speech route, the model name.
+TTS_MODEL=ocid1.generativeaiendpoint.oc1.<region>.<endpoint-id>
+
+# Compartment the endpoint lives in (sent as the compartment header).
+GENAI_COMPARTMENT_ID=ocid1.compartment.oc1..<compartment-id>
+
+# How to authenticate. One of:
+#   api_key             ~/.oci/config profile (laptops, default)
+#   resource_principal  OCI Data Science notebook sessions and jobs (auto-detected there)
+#   instance_principal  OCI compute VMs with a dynamic group policy
+#   security_token      `oci session authenticate` profiles
+#   bearer              an OCI Generative AI API key, see GENAI_API_KEY
+OCI_AUTH=api_key
+OCI_CONFIG_PROFILE=DEFAULT
+
+# Only for OCI_AUTH=bearer. Create it under Generative AI > API keys, in the SAME region as the endpoint.
+# GENAI_API_KEY=sk-...
+
+# Optional: force the inference host. Default is derived from the region in TTS_MODEL,
+# or from OCI_REGION when TTS_MODEL is not an OCID.
+# GENAI_SERVICE_ENDPOINT=https://inference.generativeai.<region>.oci.oraclecloud.com
+# OCI_REGION=eu-frankfurt-1