observability
nimblebrain_server_sdk.observability · install nimblebrain-server-sdk[observability] · version 0.7.2
from nimblebrain_server_sdk.observability import CapacityRefusal, ErrorCode, ServiceErrorObservability for a server: RED metrics, structured JSON logs and W3C trace propagation.
Needs the [observability] extra (pip install 'nimblebrain-server-sdk[observability]');
imported without it, this package raises an ImportError that names the extra.
A server becomes observable by adopting this package rather than hand-rolling instrumentation. Wiring is three lines:
from nimblebrain_server_sdk.observability import instrument_app, get_logger
app = create_app()instrument_app(app, service="notes")log = get_logger(__name__)After that: GET /metrics serves the registry, every log line is JSON stamped
with the verified tenant_id / workspace_id and the active OTel
trace_id, and traceparent is extracted inbound and exported over OTLP
(when an endpoint is configured). Domain metrics ride the same registry via counter();
cross-boundary calls propagate the trace via inject_trace_headers().
The error contract (ServiceError, ErrorCode, to_tool_error,
rate_limited_tool_error, CapacityRefusal) lives in nimblebrain_server_sdk.errors
and is re-exported here as the same objects.
Functions
Section titled “Functions”counter
Section titled “counter”def counter(name: str, documentation: str, labelnames: tuple[str, ...] = ()) -> CounterDeclare a domain counter on the shared registry (e.g. tasks_created_total).
def gauge(name: str, documentation: str, labelnames: tuple[str, ...] = ()) -> GaugeDeclare a domain gauge on the shared registry (e.g. a current-state level
like downstream_resolvable{service,dep}).
Use for a value that can go up and down — a level to read now, not a
monotonic total (that’s counter). Rides the same /metrics scrape.
get_logger
Section titled “get_logger”def get_logger(name: str | None = None) -> structlog.stdlib.BoundLoggerA structured logger. Identity/correlation are injected automatically.
histogram
Section titled “histogram”def histogram(name: str, documentation: str, labelnames: tuple[str, ...] = (), buckets: tuple[float, ...] = _DURATION_BUCKETS) -> HistogramDeclare a domain histogram on the shared registry (e.g. forward_seconds).
For a distribution a counter cannot express — how long a domain operation
took, not how many there were. Defaults to the same agent/LLM-shaped duration
buckets the RED histogram uses, so a domain latency and its route latency are
readable on one axis; pass buckets for a non-duration distribution.
inject_trace_headers
Section titled “inject_trace_headers”def inject_trace_headers(headers: dict[str, str] | None = None) -> dict[str, str]Inject traceparent (+ baggage) into an outbound header dict.
Use at trust boundaries the auto-instrumentation doesn’t cover so the trace survives the hop:
httpx.post(url, headers=inject_trace_headers({"authorization": token}))instrument_app
Section titled “instrument_app”def instrument_app(app: App, *, service: str, metrics_path: str = '/metrics', log_level: str = 'INFO', caller_resolver: CallerResolver | None = None) -> AppMake a FastAPI/Starlette app observable. Call once after creating the app.
- configures JSON-to-stdout logging stamped with request context
- installs the OTLP tracer provider + FastAPI auto-instrumentation
- adds the RED-metrics + context-binding middleware
- mounts
GET {metrics_path}serving the Prometheus registry - if
METRICS_PORTis set, ALSO serves the registry on that dedicated port
The dedicated METRICS_PORT is for edge-fronted servers whose app
port also serves /mcp: Prometheus scrapes the metrics-only port (admitted
by a network policy), so /mcp is never exposed to the monitoring namespace.
caller_resolver is the infra-plane opt-in: a service with no tenant is attributed
by its caller. Pass a fn(get_header) -> caller and every log line in the
request scope carries caller. Tenant/fleet services omit it — their logs are
unchanged. The resolver must bound cardinality (collapse unknowns to one bucket).
rate_limited_tool_error
Section titled “rate_limited_tool_error”def rate_limited_tool_error(refusal: CapacityRefusal) -> AnyThe rate_limited tool error for a call refused for capacity, to return.
Classified, counted and logged like any ServiceError. The host sees the refusal’s own
message and what to do next; the code is retryable, and the call ran nothing.
service_version
Section titled “service_version”def service_version() -> strThe running service’s own version, for the MCP serverInfo.version field.
Read from SERVICE_VERSION, which the deployment sets to the deployed version
(0.2.0 for a release, a short SHA for a staging build), so a server reports exactly
what’s deployed.
Falls back to "dev" for a local run with nothing injected.
Pass the result as FastMCP(name, version=service_version()). A server that
omits version= reports the FastMCP library version instead, which tells a
connecting client nothing about the service.
to_tool_error
Section titled “to_tool_error”def to_tool_error(err: ServiceError) -> AnyA real fastmcp.ToolResult(is_error=True) to return from a tool.
Returns the FastMCP result object — NOT a dict. Returning a plain dict does
NOT work: FastMCP scores a returned dict as is_error=False (a success with
the flag buried in structured data), which reproduces the very in-band-error
footgun this contract exists to prevent. The sanitized message is the text
block; structured_content.error carries the envelope.
Imports fastmcp when called, so importing this module does not.
Classes
Section titled “Classes”CapacityRefusal
Section titled “CapacityRefusal”class CapacityRefusal(Protocol)A call refused for capacity before it ran anything, carrying host-safe text.
nimblebrain_server_sdk.db.Overloaded is one; this module names the shape rather than
the class, so it takes no database dependency.
ErrorCode
Section titled “ErrorCode”class ErrorCode(StrEnum)The closed set of cross-cutting error codes.
ServiceError
Section titled “ServiceError”class ServiceError(code: ErrorCode, message: str | None = None, *, upstream_status: int | None = None, retryable: bool | None = None, detail: str | None = None)A classified, host-safe failure crossing a service boundary.
message crosses the trust boundary to the host — keep it generic; NEVER
put a vendor name/URL, raw upstream body, secret, or tenant data in it.
detail is operator-only (logs); it never reaches the host envelope.
Domain-specific failures subclass this and override metric_code to roll
up to one of the cross-cutting codes for the metric label (so the
service_errors_total{code=...} label stays the closed set).
ServiceError.from_upstream
Section titled “ServiceError.from_upstream”def from_upstream(status: int, message: str | None = None, *, detail: str | None = None) -> ServiceErrorBuild from a raw upstream HTTP status (the common provider-error path).
The bare status number is stamped as host-safe upstream_status; the
vendor’s prose belongs in detail (logs only), never in message.
Omit message to use the classified code’s host-safe default.
ServiceError.metric_code
Section titled “ServiceError.metric_code”metric_code: strThe closed-set code used as the service_errors_total label.
Cross-cutting errors are their own code; domain subclasses override this to roll up, keeping the metric’s label set bounded.
ServiceError.to_envelope
Section titled “ServiceError.to_envelope”def to_envelope() -> dict[str, object]The host-safe error payload (HTTP body / MCP structuredContent.error).