diff --git a/mcp/python-react-agent-simple-ui/README.md b/mcp/python-react-agent-simple-ui/README.md index 770fe59..6bd4373 100644 --- a/mcp/python-react-agent-simple-ui/README.md +++ b/mcp/python-react-agent-simple-ui/README.md @@ -118,7 +118,7 @@ Then put an ` + + + + + + +``` + +For developers, the same thing in Python. `answer_index_for(url)` stands for the 5 steps above: + +```python +import re +from html import unescape as html_unescape # links inside HTML write & as & +from urllib.parse import quote + +IFRAME_SRC = re.compile(r'src="([^"]*tsmcp=true[^"]*)"') +PLACEHOLDER = re.compile(r"\{\{TS_CHART:(\d+)\}\}") + +def to_saved_html(html: str) -> str: + """Before saving: swap each chart link for a placeholder.""" + return IFRAME_SRC.sub( + lambda m: f'src="{{{{TS_CHART:{answer_index_for(html_unescape(m[1]))}}}}}"', html + ) + +def to_display_html(html: str, ts_host: str, conversation_id: str) -> str: + """When reopening: swap each placeholder for a fresh chart link.""" + def fresh_link(m: re.Match) -> str: + return (f"{ts_host}/v2/?tsmcp=true&tsmcpConversationId={quote(conversation_id, safe='')}" + f"&tsmcpAnswerIndex={m[1]}") + return PLACEHOLDER.sub(fresh_link, html) +``` + +Good to know: + +- **Why work out the number when saving, not later?** The saved HTML holds only the charts the user actually saw. Numbering them from ThoughtSpot's list at save time keeps each number right, even if an earlier reply was cut off. +- **Add `srcdoc="Loading chart…"` to the iframe.** Without it, the browser tries to open the link itself before the SDK takes over, and shows a login error for a few seconds. +- **Chart shows "not logged in"?** The iframe must keep `tsmcp=true` in its link, and the SDK must be running on the page. Otherwise the browser opens ThoughtSpot itself, with no login. Also check that your site is on ThoughtSpot's CORS and frame-ancestors allowlists. See [Troubleshooting](#troubleshooting). +- **Only links on your own ThoughtSpot address** should be turned back into charts. Treat anything else in saved HTML as untrusted. + +**Customer impact:** if you need charts to last longer than ThoughtSpot keeps conversations, this approach can't provide it. Once ThoughtSpot removes a conversation, its charts cannot be rebuilt, though the saved messages still show. + +The rest of this section is the technical detail for developers. + `claude_agent_with_spotter3_mcp_server_and_chat_history.py` is the same agent plus a persistent chat history, so conversations survive a restart and the UI can list, reopen and delete them. Run it instead of `claude_agent_with_spotter3_mcp_server`: ```bash @@ -376,7 +484,7 @@ Two tables, because the browser and the model need different things: | `conversations` | `id`, `title`, `created_at`, `updated_at` | The sidebar list. `title` is the first line of the first user message. | | | `analytical_session_id` | The ThoughtSpot session, so a reopened conversation continues in the same one. | | | `claude_messages` | The **raw** Claude message list — `tool_use` / `tool_result` / `thinking` blocks included — replayed into the next request so follow-ups keep full context after a restart. | -| `turns` | `role`, `content`, `answers`, `work` | What the UI renders. `answers` holds each chart's title - **not** its `iframe_url`, which expires with the ThoughtSpot answer (about 8 hours). `work` holds the Analytics Agent's steps for **Show work**; `db_init()` adds the column to databases created before it existed. | +| `turns` | `role`, `content`, `answers`, `work` | What the UI renders. `answers` holds one empty placeholder per chart - **not** its `iframe_url` (expires with the ThoughtSpot answer, about 8 hours) or title. `work` holds the Analytics Agent's steps for **Show work**; `db_init()` adds the column to databases created before it existed. | `turns` cascades on delete (`PRAGMA foreign_keys=ON`), so removing a conversation removes its transcript. @@ -384,7 +492,7 @@ Two tables, because the browser and the model need different things: A reopened chat re-runs its charts against current data. `GET /api/conversations/{id}` gives every stored answer an `answer_index` - its position in the conversation - and the client builds an embed URL from `tsmcpConversationId` (the `analytical_session_id`) plus `tsmcpAnswerIndex`. The SDK resolves that to a live answer. -The index must count the way ThoughtSpot counts, so `reconcile_answers()` asks ThoughtSpot how many answers each turn holds (`/api/rest/2.0/ai/agent/conversations/{id}/messages`) and aligns the stored titles to it. That call is why opening a stored chat takes a few seconds; the UI shows a loading indicator meanwhile. +The index must count the way ThoughtSpot counts, so `reconcile_answers()` asks ThoughtSpot how many answers each turn holds (`/api/rest/2.0/ai/agent/conversations/{id}/messages`) and pads the stored placeholders to match. The `reconcile_answers()` docstring also lists the steps to get an `answer_index` from a live `iframe_url`. That call is why opening a stored chat takes a few seconds; the UI shows a loading indicator meanwhile. `jsonable()` serializes the Claude history with `model_dump(mode="json")` rather than by hand. That matters for thinking blocks: their `signature` must come back **unchanged** on replay, and a hand-rolled `{"type", "text"}` mapping would drop it. @@ -416,6 +524,7 @@ In-memory `conversations` / `analytical_sessions` dicts stay as a hot cache in f ### Limits of this example +- **Charts last only as long as ThoughtSpot keeps the conversation.** If you need charts to last longer, this approach can't provide it. Once ThoughtSpot removes a conversation, its charts cannot be rebuilt, though the saved messages still show. - **No auth and no per-user scoping.** Every conversation in the file is visible to every caller. Add a user id column and filter on the authenticated user before this goes anywhere real. - **No pruning.** `claude_messages` grows with every turn. For long-lived chats, add [context editing or compaction](https://docs.anthropic.com) rather than replaying an unbounded history. - **Single process.** SQLite in WAL mode is fine for one uvicorn worker; use a real database if you run several. @@ -536,6 +645,8 @@ init({ | MCP connection failures | `curl localhost:8001/api/tools` — if that fails, the token or `VITE_TS_HOST` is wrong | | `MCPError: Server returned an error response` | The MCP server's `initialize` returned HTTP 500. It is retried once automatically; if it persists, the MCP service or cluster is having trouble | | Tool calls fail with `Failed to validate connection` | The cluster no longer accepts the token. The server renews its token before expiry; restart it if the error persists | +| Old chat reopens with a blank or login-page chart (new chats are fine) | ThoughtSpot no longer holds that conversation, so the chart cannot be rebuilt. Test: `GET {ThoughtSpot host}/conversation/v2/{analytical_session_id}/public` with your token returns 404 "Conversation ... not found". How long ThoughtSpot keeps conversations is not documented here. In one test, chats from earlier that day loaded and chats from 3 days earlier returned 404. The saved messages still show; only the charts are gone. Show a "chart no longer available" note instead of an empty frame. | +| Chart shows "not logged in" or a login page | Work down this list: (1) **Your site is not allowed by ThoughtSpot.** In ThoughtSpot go to Develop > Customizations > Security Settings and add your site's address (for example `http://localhost:8000`) to both the **CORS** allowlist and the **CSP frame-ancestors** allowlist. A browser console error mentioning `/callosum/v1/session/isactive` and "CORS policy" confirms this. (2) **The SDK is not running, or started too late.** `init()` and `startAutoMCPFrameRenderer()` must run before the chart's iframe is added to the page; the SDK ignores iframes that were already there. (3) **The iframe link has no `tsmcp=true`.** Only links with it are upgraded to a logged-in chart; any other iframe loads ThoughtSpot directly and shows the login page. (4) **No token.** Open `/api/ts-token` in the browser; it must return a `token`. | | Charts blocked / blank iframe | The UI's origin is not in the cluster's CSP `frame-ancestors`. Serve the UI from the allowlisted origin (`http://localhost:8000` here) | | Only legacy tools (`getAnswer`, …) appear | You are on a `/bearer/*` URL; use `/token/mcp?api-version=2026-05-01` | | `list_orgs` / `switch_org` missing | Expected — they are OAuth-only and hidden on `/token/*` | diff --git a/mcp/python-react-agent-simple-ui/server/claude_agent_with_spotter3_mcp_server_and_chat_history.py b/mcp/python-react-agent-simple-ui/server/claude_agent_with_spotter3_mcp_server_and_chat_history.py index ba8987e..00c88a9 100644 --- a/mcp/python-react-agent-simple-ui/server/claude_agent_with_spotter3_mcp_server_and_chat_history.py +++ b/mcp/python-react-agent-simple-ui/server/claude_agent_with_spotter3_mcp_server_and_chat_history.py @@ -1,11 +1,13 @@ """ Python agent: Anthropic Claude + the ThoughtSpot MCP server (Spotter 3 toolset), -with persistent chat history. +with saved chat history. -The same agent as `claude_agent_with_spotter3_mcp_server.py` (the agent lives in -`spotter3_core.py`), plus a SQLite chat history, so past conversations survive a restart -and the UI can list, reopen and delete them. It also records the Analytics Agent's work -(`work` events) for the UI's "Show work" section. +Same agent as `claude_agent_with_spotter3_mcp_server.py` (the agent itself lives in +`spotter3_core.py`). This version also saves every chat to a small SQLite file, so: + + - chats survive a server restart, + - the UI can list, reopen, rename and delete past chats, + - the Analytics Agent's steps are kept for the UI's "Show work" section. MCP server: https://github.com/thoughtspot/mcp-server """ @@ -41,18 +43,23 @@ # ════════════════════════════════════════════════════════════════════════════════ # CHAT HISTORY # -# Stored in two layers, because the browser and the model need different things: +# For the big picture, see "How chat history works" in the README. +# +# For each chat we save three things: # -# turns - what the UI renders: user text, assistant text, the answers -# (titles only, see `storable_answers`) and the Agent's work steps. -# claude_messages - the raw Claude message list, tool_use and tool_result blocks -# included. Replayed into the next request so follow-ups keep full -# context after a restart. +# 1. The messages - every question and reply, so the chat can be shown again +# and the AI remembers the context for follow-up questions. +# 2. The conversation id - ThoughtSpot's id for the chat (`analytical_session_id`). +# ThoughtSpot keeps the chat's charts under this id. +# 3. A chart count - one empty placeholder per chart in each reply. # -# The ThoughtSpot MCP server's own conversation storage is internal plumbing for -# `get_session_updates` delivery, not a client-facing history API - so history belongs to -# the app. SQLite via the stdlib, no extra dependency. Calls are small and run in a worker -# thread (`asyncio.to_thread`) to keep the event loop free. +# The screen's copy (`turns`) holds no chart links or titles. (`claude_messages` is the +# AI's own working copy and can still contain titles from ThoughtSpot's replies.) A chart link stops working after about +# 8 hours. When a chat is reopened, each chart is drawn fresh from the conversation id +# plus the chart's number in the chat (`answer_index`). See `reconcile_answers`. +# +# Where it is stored: a SQLite file next to this script. SQLite comes with Python, so +# there is nothing extra to install. # ════════════════════════════════════════════════════════════════════════════════ DB_PATH = Path(os.getenv("CHAT_HISTORY_DB", Path(__file__).resolve().parent / "chat_history.db")) @@ -131,21 +138,15 @@ def jsonable(value: Any) -> Any: def storable_answers(answers: list[dict]) -> list[dict]: - """Keep the parts of an answer that survive, drop the parts that go stale. + """One empty placeholder per chart - nothing about the chart itself is saved. - An answer object lives about 8 hours on the ThoughtSpot side. After that its - `iframe_url` - and the `answer_id`, really a `{session_id, gen_no}` pair - point at - nothing, and a reopened chat would render a row of error tiles. So only the durable - parts are stored; on replay the client asks the Visual Embed SDK for a live URL from - the conversation id plus the answer's position. `query` is dropped too: nothing reads - it on replay. + A chart's `iframe_url` and `answer_id` stop working after about 8 hours, and its + title is not needed to rebuild it. Only the number of charts matters; their position + (`answer_index`) is worked out on reopen. - Applied on the way out as well, so rows stored before this rule resolve on replay too. + Also applied when reading, so rows saved by older versions lose their stale fields. """ - return [ - {k: v for k, v in answer.items() if k not in ("iframe_url", "answer_id", "query")} - for answer in answers - ] + return [{} for _ in answers] def db_start_conversation(conv_id: str, first_message: str) -> None: @@ -283,10 +284,10 @@ async def load_conversation_state(conv_id: str) -> list: class StreamRecorder: - """Fans every SSE event out to the browser and to the transcript we persist. + """Sends each event to the browser and also keeps a copy to save. - The UI renders a turn from exactly the events it received, so recording them here is - what makes a reopened conversation look like the live one. + Saving what the browser received lets a reopened chat match the live one, except + chart titles, which are not kept. """ def __init__(self, queue: asyncio.Queue) -> None: @@ -300,7 +301,7 @@ def send(self, event: dict) -> None: if kind == "delta": self.text_parts.append(event.get("text") or "") elif kind == "answer": - self.answers.append({k: event.get(k) for k in ("answer_id", "title", "iframe_url")}) + self.answers.append({}) # one placeholder per chart, see `storable_answers` elif kind == "work": append_work(self.work, event) self.queue.put_nowait(event) @@ -340,7 +341,7 @@ async def persist_turn(conv_id: str, recorder: StreamRecorder, history: list | N conv_id, "assistant", recorder.text, - storable_answers(recorder.answers), + recorder.answers, recorder.work, ) if history is not None: @@ -383,8 +384,8 @@ async def turn(queue: asyncio.Queue) -> None: # ── ThoughtSpot conversation lookups ──────────────────────────────────────────── -# A stored conversation is checked against what ThoughtSpot holds, so replayed charts -# line up with the answers the SDK resolves. +# On reopen, the saved chat is checked against the conversation ThoughtSpot holds, so +# each chart gets the right `answer_index`. async def ts_request( @@ -481,17 +482,37 @@ async def ts_load_answer(session_id: str, answer_id: str) -> dict | None: def reconcile_answers(conversation: dict, ts_messages: list[dict] | None) -> dict: - """Align a stored conversation with the answers ThoughtSpot actually holds. - - The stored copy is not a reliable source of truth. A stream can be cut off (the - browser navigates away, the process restarts) after the Agent was already asked; it - finishes anyway, so the answer exists on ThoughtSpot's side while our turn recorded - none of it. And `answer_index` has to count the way ThoughtSpot counts, or a replayed - answer resolves to the wrong chart. - - So ThoughtSpot decides how many answers each turn has and the stored rows only supply - titles. A turn missing answers gets untitled placeholders; the client renders them and - the SDK resolves each by its index. + """Give every saved chart its number in the chat (`answer_index`). + + Think of the chat's charts as numbered pages in a notebook. ThoughtSpot keeps the + notebook (the conversation id); we only need each chart's page number. The first + final chart in the chat is 0, the next is 1, and so on, across the whole chat. The + browser hands both to the Visual Embed SDK (`tsmcpConversationId`, + `tsmcpAnswerIndex`), and ThoughtSpot draws the chart fresh. + + We let ThoughtSpot do the counting, not our saved copy. If the browser closes while a + chart is loading, ThoughtSpot still finishes the chart but we never saw it. Its count + is complete; ours may not be. A chart we missed gets an empty placeholder. + + How to find a chart's number from its link (`iframe_url`) + ---------------------------------------------------------- + A chart link from the MCP server looks like this: + + https:///?tsmcp=true#/embed/conv-assist-answer + ?sessionId=72728061-...&genNo=2&acSessionId=2bfd6f50-...&acGenNo=1 + + 1. From the link, note `sessionId` and `genNo`. Together they name one chart. + 2. Get the chat's messages from ThoughtSpot: + GET /api/rest/2.0/ai/agent/conversations/{analytical_session_id}/messages + 3. Go through the messages oldest first. Count only final charts: items in + `response_items` with `type == "answer"`, `is_thinking == false` and an + `answer_id`. Skip the "thinking" charts (see `answer_items`). + 4. Number those charts 0, 1, 2, ... across the whole chat. + 5. `answer_id` is text holding JSON, like {"session_id": "...", "gen_no": 2}. + Parse it. The chart with the same `session_id` and `gen_no` as step 1 has your + number: that is its `answer_index`. + + This function does steps 2 to 4 for every chart at once, turn by turn. """ turns = conversation.get("turns") or [] if ts_messages is None: @@ -514,7 +535,7 @@ def reconcile_answers(conversation: dict, ts_messages: list[dict] | None) -> dic thinking_answers = answer_items(message, thinking=True) turn["work_answer_ids"] = [item["answer_id"] for item in thinking_answers] answers = turn.get("answers") or [] - # Titles we recorded, padded out to the count ThoughtSpot reports. + # Saved placeholders, padded out to the count ThoughtSpot reports. merged = answers[:expected] + [{} for _ in range(max(0, expected - len(answers)))] for answer in merged: answer["answer_index"] = index