diff --git a/CHANGELOG.md b/CHANGELOG.md index c89fb978..3c5aceec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Host concerns for the engine's models and controllers** (`actionagent`). + `ActionAgent.model_concerns` is included into + `ActionAgent::ApplicationRecord` as it loads, and so into every engine + model; `ActionAgent.controller_concerns` into + `ActionAgent::ApplicationController`, ahead of its own callbacks, and so + into every dashboard controller. (The ingest endpoint, + `Api::TracesController`, inherits `ActionController::API` and keeps its + own bearer-token authentication; it is not touched.) Entries are modules + or their names, resolved + when the class loads. A host that pins the engine's tables to one database + connection, or carries its session helpers onto the dashboard's + controllers, configures that here instead of reopening the classes from a + `to_prepare` block. - **The Evaluations page is rebuilt around runs** (`actionagent`). Evaluations are the top level; every run is kept and listed with its movement against the run before it (`+3 passed vs #2`, `partial run`, `#1 failed`), and a @@ -56,6 +69,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The engine's judge blocks take `ActiveAgent::Evals::Judge`'s `kind:`, so a scenario run's score, recommendation and verdict calls are metered apart. +- `ActionAgent::TelemetryTrace` inherits `ActionAgent::ApplicationRecord` + like every other engine model (`actionagent`), so it carries the model + concerns above, `AdapterAware` and the ownership API (`owner_association`, + `for_owner`) from the same place. Its table name is unchanged. + +### Deprecated + +- Assigning `ActionAgent.base_controller_class`, which has never been + consumed, warns through `ActionAgent.deprecator` and points at + `controller_concerns`. The accessor is removed in 2.0. ### Fixed diff --git a/actionagent/README.md b/actionagent/README.md index 22766838..539e33ad 100644 --- a/actionagent/README.md +++ b/actionagent/README.md @@ -70,6 +70,11 @@ ActionAgent.configure do |config| # Restrict what a given owner can see. config.agent_scope_resolver = ->(owner) { ActionAgent::Agent.where(user: owner) } + + # Put your app's own concerns on the engine's classes: a connection + # switcher on every model, request tagging on every dashboard controller. + config.model_concerns = ["ConnectionSwitching"] + config.controller_concerns = ["RequestTagging"] end ``` diff --git a/actionagent/app/controllers/action_agent/application_controller.rb b/actionagent/app/controllers/action_agent/application_controller.rb index 670421ca..8a08b142 100644 --- a/actionagent/app/controllers/action_agent/application_controller.rb +++ b/actionagent/app/controllers/action_agent/application_controller.rb @@ -5,6 +5,10 @@ module ActionAgent # # Handles authentication and provides helper methods for multi-tenant mode. class ApplicationController < ActionController::Base + # Ahead of the engine's own callbacks, so a concern's before_action or + # around_action runs before the dashboard authenticates. + ActionAgent.controller_concern_modules.each { |concern| include concern } + protect_from_forgery with: :exception before_action :authenticate_dashboard! diff --git a/actionagent/app/models/action_agent/application_record.rb b/actionagent/app/models/action_agent/application_record.rb index 70b6457b..a78747ce 100644 --- a/actionagent/app/models/action_agent/application_record.rb +++ b/actionagent/app/models/action_agent/application_record.rb @@ -17,6 +17,10 @@ class ApplicationRecord < ::ActiveRecord::Base self.abstract_class = true + # After abstract_class, so a concern's included block that reads + # table_name sees nil. + ActionAgent.model_concern_modules.each { |concern| include concern } + # Models that are not themselves owned still answer the ownership # questions, so callers can scope any dashboard relation uniformly. class << self diff --git a/actionagent/app/models/action_agent/telemetry_trace.rb b/actionagent/app/models/action_agent/telemetry_trace.rb index 710d93e8..4ef6324d 100644 --- a/actionagent/app/models/action_agent/telemetry_trace.rb +++ b/actionagent/app/models/action_agent/telemetry_trace.rb @@ -16,9 +16,10 @@ module ActionAgent # @example Creating a trace with account (multi-tenant mode) # ActionAgent::TelemetryTrace.create_from_payload(trace_payload, sdk_info, account: account) # - class TelemetryTrace < ::ActiveRecord::Base - include ActionAgent::AdapterAware - + class TelemetryTrace < ApplicationRecord + # Fixed rather than derived from ActionAgent.table_name_prefix: the + # telemetry migration the install generator writes creates this table + # under its literal name, whatever prefix the host sets. self.table_name = "active_agent_telemetry_traces" # Optional account association for multi-tenant mode diff --git a/actionagent/lib/action_agent.rb b/actionagent/lib/action_agent.rb index ad28e936..8e9086ab 100644 --- a/actionagent/lib/action_agent.rb +++ b/actionagent/lib/action_agent.rb @@ -216,11 +216,52 @@ def deprecator # @return [String, nil] attr_accessor :ingest_api_key + # Concerns included into ActionAgent::ApplicationRecord as it loads, and + # through it into every engine model. An entry is a Module or the name + # of one. A name is resolved when the class + # loads, so an initializer can refer to a constant the host has not + # autoloaded yet, and a name that resolves to nothing raises NameError + # there rather than being skipped. + # + # ActionAgent.configure do |config| + # config.model_concerns = ["MyApp::ConnectionSwitching"] + # end + # + # The class loads after the initializers have run, so set this in an + # initializer; a concern added later is not applied. + # @return [Array] + attr_accessor :model_concerns + + # Concerns included into ActionAgent::ApplicationController as it loads, + # and through it into every dashboard controller: the React dashboard + # and its JSON API, the server-rendered console and the MCP facade. + # They are included ahead of the engine's own callbacks, so a concern's + # before_action or around_action runs before the dashboard + # authenticates. Entries are Modules or names, as for model_concerns. + # + # Not the trace ingest endpoint: Api::TracesController authenticates + # with a bearer token and inherits ActionController::API. + # + # ActionAgent.configure do |config| + # config.controller_concerns = ["MyApp::RequestTagging"] + # end + # @return [Array] + attr_accessor :controller_concerns + # @deprecated Never consumed — dashboard controllers inherit - # ActionController::Base. Retained as a no-op so existing - # initializers that set it keep booting; remove in the next major. + # ActionController::Base, and controller_concerns is how a host puts + # its own behaviour on them. Assigning it warns and stores a value + # nothing reads; removed in the next major. # @return [String] - attr_accessor :base_controller_class + attr_reader :base_controller_class + + def base_controller_class=(value) + deprecator.warn( + "ActionAgent.base_controller_class has never been consumed and is removed in 2.0. " \ + "Set ActionAgent.controller_concerns to extend the dashboard's controllers." + ) + @base_controller_class = value + end # Called before each run/trace-ingest to enforce host-app limits. # Receives (owner, kind) where kind is :execution or :trace_ingest, and @@ -528,6 +569,19 @@ def owner_class name&.safe_constantize end + # The modules model_concerns names. ApplicationRecord reads it as it loads. + # @return [Array] + def model_concern_modules + resolve_concerns(model_concerns) + end + + # The modules controller_concerns names. ApplicationController reads it + # as it loads. + # @return [Array] + def controller_concern_modules + resolve_concerns(controller_concerns) + end + # Configures the dashboard. # # @yield [config] Configuration block @@ -555,6 +609,8 @@ def reset! @storage_service = nil @ingest_api_key = nil @base_controller_class = "ActionController::Base" # deprecated no-op + @model_concerns = [] + @controller_concerns = [] @quota_checker = nil @provider_credentials_resolver = nil @sandbox_backends = {} @@ -655,6 +711,12 @@ def runtime_schema_tools?(klass) def schema_tool_class_for(name) schema_tool_classes.find { |klass| klass.tool?(name) } end + + private + + def resolve_concerns(entries) + Array(entries).map { |entry| entry.is_a?(Module) ? entry : entry.to_s.constantize } + end end # Set defaults diff --git a/actionagent/lib/generators/action_agent/templates/action_agent.rb.erb b/actionagent/lib/generators/action_agent/templates/action_agent.rb.erb index 9e34a86d..ff38173e 100644 --- a/actionagent/lib/generators/action_agent/templates/action_agent.rb.erb +++ b/actionagent/lib/generators/action_agent/templates/action_agent.rb.erb @@ -78,6 +78,20 @@ ActionAgent.configure do |config| # nothing rather than everything. An empty dashboard for a signed-in user # means the resolver above returned nil. + # ========================================================================== + # Host integration + # ========================================================================== + # + # Concerns your own models and controllers carry, applied to the engine's. + # model_concerns are included into ActionAgent::ApplicationRecord, and so + # into every engine model. controller_concerns are included into + # ActionAgent::ApplicationController ahead of its own callbacks, and so + # into every dashboard controller — not the bearer-token ingest endpoint. + # Modules or their names; a name is resolved when the class loads. + # + # config.model_concerns = ["ConnectionSwitching"] + # config.controller_concerns = ["RequestTagging"] + # ========================================================================== # Ask ActiveAgents assistant # ========================================================================== diff --git a/actionagent/test/host_integration_test.rb b/actionagent/test/host_integration_test.rb new file mode 100644 index 00000000..889c8fe0 --- /dev/null +++ b/actionagent/test/host_integration_test.rb @@ -0,0 +1,161 @@ +# frozen_string_literal: true + +require "test_helper" +require "json" + +# The seams a host app extends the engine's classes through — +# ActionAgent.model_concerns and ActionAgent.controller_concerns — and the +# single model hierarchy the first one relies on. +# +# The model tests reopen ActionAgent::ApplicationRecord with `load`, which +# re-runs its class body against the configuration of the moment; the body is +# idempotent, so the process is left as it was apart from the module included. +# Reopening ApplicationController the same way is not harmless: its callback +# macros would re-run and reorder every loaded subclass's chain. What a +# controller concern does on the classes' first load is therefore read from a +# child process that boots the dummy app with both lists configured +# (support/first_load_script.rb). +class HostIntegrationTest < ActiveSupport::TestCase + # The shape of a host's connection-switching concern: class-level behaviour + # every model must carry. + module ConnectionSwitching + extend ActiveSupport::Concern + + class_methods do + def pinned_connection? = true + end + end + + MODEL_FILE = ActionAgent::Engine.root.join("app/models/action_agent/application_record.rb").to_s + FIRST_LOAD_SCRIPT = File.expand_path("support/first_load_script.rb", __dir__) + + setup do + @model_concerns = ActionAgent.model_concerns + @controller_concerns = ActionAgent.controller_concerns + end + + teardown do + ActionAgent.model_concerns = @model_concerns + ActionAgent.controller_concerns = @controller_concerns + end + + test "TelemetryTrace is an engine model like the others" do + assert_operator ActionAgent::TelemetryTrace, :<, ActionAgent::ApplicationRecord + assert_equal "active_agent_telemetry_traces", ActionAgent::TelemetryTrace.table_name + assert_respond_to ActionAgent::TelemetryTrace, :postgres? + assert_nil ActionAgent::TelemetryTrace.owner_association + end + + test "every persisted engine model inherits ApplicationRecord" do + Rails.autoloaders.main.eager_load_dir(ActionAgent::Engine.root.join("app/models").to_s) + engine_models = ActiveRecord::Base.descendants.select { |model| model.name.to_s.start_with?("ActionAgent::") } + strays = engine_models.reject { |model| model <= ActionAgent::ApplicationRecord } + + assert_includes engine_models, ActionAgent::TelemetryTrace + assert_includes engine_models, ActionAgent::Agent + assert_empty strays, "outside the hierarchy model_concerns extends" + end + + test "model concerns are included as ApplicationRecord loads and reach every model" do + ActionAgent.model_concerns = [ "HostIntegrationTest::ConnectionSwitching" ] + reload_class(MODEL_FILE) + + assert ActionAgent::ApplicationRecord.include?(ConnectionSwitching) + assert ActionAgent::Agent.pinned_connection? + assert ActionAgent::TelemetryTrace.pinned_connection? + end + + test "a model concern may be given as a Module" do + concern = Module.new + ActionAgent.model_concerns = [ concern ] + reload_class(MODEL_FILE) + + assert ActionAgent::ApplicationRecord.include?(concern) + end + + test "a model concern that resolves to nothing fails as the class loads, naming it" do + ActionAgent.model_concerns = [ "HostIntegrationTest::NoSuchConcern" ] + + error = assert_raises(NameError) { reload_class(MODEL_FILE) } + assert_includes error.message, "NoSuchConcern" + end + + test "on first load, a model concern's included block sees the abstract base with no table name" do + assert_equal [ [ true, nil ] ], first_load.fetch("included_saw") + end + + test "on first load, a model concern's inherited hook sees every model's table, the trace table among them" do + assert_includes first_load.fetch("registered_tables"), "active_agent_telemetry_traces" + assert_includes first_load.fetch("registered_tables"), "active_agent_agents" + assert_equal [ true, true ], first_load.fetch("models_pinned") + end + + test "on first load, a controller concern's before_action precedes the dashboard's authentication" do + %w[application_controller_filters base_controller_filters].each do |chain| + filters = first_load.fetch(chain) + + assert_operator filters.index("note_host_session"), :<, filters.index("authenticate_dashboard!"), chain + end + end + + # Forgery protection is left to ApiForgeryProtectionTest, which exercises + # it: the dashboard API verifies CSRF tokens (#461), and the callback names + # a skip leaves in the chain differ across Rails versions. + test "on first load, a dashboard controller's own callback skips still hold" do + assert_not_includes first_load.fetch("mcp_controller_filters"), "authenticate_dashboard!" + end + + test "controller concerns do not reach the bearer-token ingest endpoint" do + assert_equal false, first_load.fetch("traces_controller_session_aware") + end + + test "a resolver reaches the concern's session reader while the engine's current_user stays in charge" do + assert_equal "session_user", first_load.fetch("dashboard_current_user") + assert_equal false, first_load.fetch("dashboard_public_current_user") + end + + test "reset! clears both concern lists" do + ActionAgent.model_concerns = [ Module.new ] + ActionAgent.controller_concerns = [ Module.new ] + + ActionAgent.reset! + + assert_equal [], ActionAgent.model_concerns + assert_equal [], ActionAgent.controller_concerns + end + + test "assigning base_controller_class warns and points at controller_concerns" do + previous = ActionAgent.base_controller_class + + assert_deprecated(/controller_concerns/, ActionAgent.deprecator) do + ActionAgent.base_controller_class = "AdminController" + end + assert_equal "AdminController", ActionAgent.base_controller_class + ensure + ActionAgent.instance_variable_set(:@base_controller_class, previous) + end + + private + + # Re-evaluates a class body the way a code reload does, so the includes the + # configuration produces at load time are observable in a booted app. + def reload_class(file) + load file + end + + # What the engine's classes looked like on their first load with both + # concern lists configured, from one boot of the dummy app shared by every + # test that reads it. + def first_load + self.class.first_load + end + + def self.first_load + @first_load ||= begin + output = IO.popen([ RbConfig.ruby, FIRST_LOAD_SCRIPT ], err: %i[child out], &:read) + JSON.parse(output.lines.last.to_s) + rescue JSON::ParserError + raise "the first-load boot did not report: #{output}" + end + end +end diff --git a/actionagent/test/support/first_load_script.rb b/actionagent/test/support/first_load_script.rb new file mode 100644 index 00000000..ef56badb --- /dev/null +++ b/actionagent/test/support/first_load_script.rb @@ -0,0 +1,70 @@ +# frozen_string_literal: true + +# Boots the dummy app with a model concern and a controller concern configured +# the way a host's initializer configures them, then reports what the engine's +# classes looked like on their first load. HostIntegrationTest runs it out of +# process, because the test process has loaded those classes long before any +# test can configure them. +ENV["RAILS_ENV"] = "test" +require File.expand_path("../../../test/dummy/config/environment", __dir__) +require "json" + +included_saw = [] +registered_tables = [] + +# The shape of a host's connection-switching concern: an included block that +# reads the base's table name, and an inherited hook that registers every +# subclass's. +model_concern = Module.new do + extend ActiveSupport::Concern + + included { included_saw << [ abstract_class?, table_name ] } + + class_methods do + define_method(:inherited) do |subclass| + super(subclass) + registered_tables << subclass.table_name + end + + def pinned_connection? = true + end +end + +# The shape of a host's session concern: an included block with class-level +# DSL and a before_action, a public current_user of its own, and the session +# reader the host's resolvers call. +controller_concern = Module.new do + extend ActiveSupport::Concern + + included do + class_attribute :host_session_aware, default: true + before_action :note_host_session + end + + def current_user = :host_user + + def user_from_session = :session_user + + private + + def note_host_session; end +end + +ActionAgent.model_concerns = [ model_concern ] +ActionAgent.controller_concerns = [ controller_concern ] +ActionAgent.current_user_resolver = ->(controller) { controller.send(:user_from_session) } + +models = [ ActionAgent::Agent, ActionAgent::TelemetryTrace ] +filters = ->(controller) { controller._process_action_callbacks.map { |callback| callback.filter.to_s } } + +puts JSON.generate( + included_saw: included_saw, + registered_tables: registered_tables, + models_pinned: models.map(&:pinned_connection?), + application_controller_filters: filters.call(ActionAgent::ApplicationController), + base_controller_filters: filters.call(ActionAgent::Api::BaseController), + mcp_controller_filters: filters.call(ActionAgent::Api::MCPController), + traces_controller_session_aware: ActionAgent::Api::TracesController.respond_to?(:host_session_aware), + dashboard_public_current_user: ActionAgent::DashboardController.public_method_defined?(:current_user), + dashboard_current_user: ActionAgent::DashboardController.new.send(:current_user).to_s +) diff --git a/docs/framework/configuration.md b/docs/framework/configuration.md index 708115cd..fad7deba 100644 --- a/docs/framework/configuration.md +++ b/docs/framework/configuration.md @@ -358,7 +358,9 @@ dashboard's own per-owner keys layer on top of it rather than replacing it. `multi_tenant`, `account_class` and `trace_model_class`; **[Self-Hosted Dashboard](/framework/self-hosted-observability)** adds `ingest_api_key`, `current_account_resolver`, `trace_retention`, -`execution_enabled`, `assistant_enabled`, `sandbox_backends` and `layout`. The host-app +`execution_enabled`, `assistant_enabled`, `sandbox_backends`, `layout`, and +`model_concerns` and `controller_concerns`, which put your app's own concerns +on the engine's models and controllers. The host-app integration seams — `quota_checker`, `usage_recorder`, `agent_scope_resolver`, `tenant_resolver`, `trace_owner_resolver`, `provider_credentials_resolver` and the rest — are documented on diff --git a/docs/framework/dashboard.md b/docs/framework/dashboard.md index 90acb55b..40efc001 100644 --- a/docs/framework/dashboard.md +++ b/docs/framework/dashboard.md @@ -533,6 +533,11 @@ authenticate :user, ->(u) { u.admin? } do end ``` +The engine's controllers are their own base class, so your app's session +helpers are not on them. `config.controller_concerns` puts a concern of +yours there for the lambda to call — see +[Extending engine models and controllers](/framework/self-hosted-observability#extending-engine-models-and-controllers). + The local ingest endpoint accepts unauthenticated posts by default (it receives traces from your own app process on your own machine). If the mount is reachable from other machines, set `config.ingest_api_key` to diff --git a/docs/framework/ruby_llm_dashboard.md b/docs/framework/ruby_llm_dashboard.md index 5befdc05..ecefc5b0 100644 --- a/docs/framework/ruby_llm_dashboard.md +++ b/docs/framework/ruby_llm_dashboard.md @@ -102,7 +102,9 @@ development and test environments (HTTP 403) — a staging or review app is as reachable as production and is treated the same. The lambda receives the engine's controller. The engine's controllers are their own base class — your app's `current_user` helper is -not on them — so read the session or your auth library directly: +not on them — so read the session or your auth library directly, or put +your session concern on them with `config.controller_concerns` (see +[Extending engine models and controllers](/framework/self-hosted-observability#extending-engine-models-and-controllers)): ```ruby # config/initializers/action_agent.rb diff --git a/docs/framework/self-hosted-observability.md b/docs/framework/self-hosted-observability.md index 35040cfb..ab8a47f9 100644 --- a/docs/framework/self-hosted-observability.md +++ b/docs/framework/self-hosted-observability.md @@ -243,6 +243,51 @@ Generations record the same `trace_id` the telemetry pipeline uses (thread it via `prompt_options[:trace_id]`), so conversation rows and dashboard traces correlate. +## Extending engine models and controllers + +A host usually has concerns its own models and controllers already carry — +a connection-switching concern that pins tables to one database, a request +instrumentation or audit concern on every controller. Two settings apply +them to the engine's classes. The engine includes them as those classes +load, so they are in place before any subclass is defined, and again after +every code reload: + +```ruby +# config/initializers/action_agent.rb +ActionAgent.configure do |config| + # Included into ActionAgent::ApplicationRecord, and so into every engine + # model. + config.model_concerns = ["MyApp::ConnectionSwitching"] + + # Included into ActionAgent::ApplicationController, and so into every + # dashboard controller — the React dashboard and its JSON API, the + # server-rendered console and the MCP facade — ahead of the engine's own + # callbacks, so a before_action or around_action runs before the + # dashboard authenticates. + config.controller_concerns = ["MyApp::RequestTagging"] +end +``` + +Entries are modules or their names. A name is resolved when the class +loads, so the initializer can refer to a constant your app has not +autoloaded yet, and a name that resolves to nothing raises `NameError` +there rather than being skipped. Set them in an initializer: the classes +load after the initializers have run, and a concern added later is not +applied. + +The trace ingest endpoint, `ActionAgent::Api::TracesController`, is not a +dashboard controller — it authenticates with a bearer token and inherits +`ActionController::API` — so controller concerns do not reach it. + +`authentication_method` still decides who is admitted. A controller +concern puts your app's session helpers on the engine's controllers, and +the lambda is where you call them. The engine's own `current_user` keeps +precedence over one a concern defines, so a concern's `current_user` is not +reachable on the engine's controllers: `current_user_resolver` remains what +names the signed-in user to the engine, and it and `authentication_method` +call the concern through a method the engine does not define, such as +`->(controller) { controller.send(:user_from_session) }`. + ## Operations - **Retention is yours.** Nothing prunes automatically. Set a window and