Deploying and Monitoring AI Systems

Inference is the process of running a trained model to generate an output.

Every time you send a prompt to a model and receive a response, the model is performing inference. For an agent, inference may happen many times during a single task as the model plans, uses tools, observes results, and decides what to do next.

Inference settings can affect quality, speed, and cost. Depending on the model, teams may be able to adjust factors such as reasoning effort, output length, sampling settings, or the amount of computation used for each request.

These settings should be evaluated alongside the model itself because the same model can behave very differently under different inference configurations.

A production AI application usually includes much more than the model.

The full system might contain:

• A model
• Prompts and instructions
• Retrieved documents or data
• Tools and APIs
• Business logic
• Model routing
• Guardrails
• User interfaces
• Human review or escalation
• Logging and monitoring

Performance depends on how all of these components work together.

For that reason, enterprise teams should eventually evaluate the complete system rather than measuring only the underlying model.

An AI agent is a system that uses a model to take actions toward completing a goal.

Instead of generating a single response, an agent may plan a sequence of steps, use tools, inspect the results, make additional decisions, and continue until the task is complete.

For example, an agent might research an account, update a CRM, draft an email, and schedule a follow-up. A coding agent might inspect a repository, edit several files, run tests, and fix errors before finishing.

Agents require different evaluation methods because success often depends on the full trajectory of actions and the final outcome.

Agent evaluations should measure whether the system successfully completes the task under realistic conditions.

That often means evaluating more than the final response. Teams may also care about:

• Whether the correct tools were used
• Whether actions were performed in the right order
• Whether the agent recovered from errors
• Whether it avoided unnecessary actions
• Whether it followed permissions and policies
• How long the task took
• How much the task cost
• Whether the final state was correct

Whenever possible, the environment itself can provide objective signals. For example, a coding task can be evaluated by running tests, while a workflow task can be checked by inspecting the resulting state of the system.

For more subjective or specialized work, human or expert review may still be necessary.

Offline evaluation tests a system on a predefined set of tasks outside of live production traffic.

It is useful for comparing models, testing changes, reproducing known failures, and deciding whether a new version is ready to launch.

Online evaluation measures how the system behaves once it is interacting with real users and real workflows.

Online signals might include task completion, user feedback, escalation rates, corrections, retries, business outcomes, or human review of sampled interactions.

Both are useful. Offline evals provide controlled and repeatable measurements. Online evaluation shows what happens under the complexity of real-world use.

Monitoring should focus on signals that tell you whether the system is still performing as intended.

Depending on the application, that can include:

• Task success rates
• Quality scores
• Critical failure rates
• Human escalation rates
• User corrections
• Tool failures
• Latency
• Cost per task
• Retry rates
• Refusal or fallback rates
• Changes across different user or task segments

Aggregate metrics are useful, but teams should also inspect individual failures. A stable average can hide a serious problem affecting a particular workflow or customer group.

The monitoring plan should reflect the risks and goals of the application.

Human oversight is useful when errors are costly, decisions require professional judgment, or the system is not reliable enough to operate autonomously.

A human might:

• Approve an action before it is executed
• Review outputs from high-risk workflows
• Handle cases where the model is uncertain
• Resolve ambiguous situations
• Take over when tools fail
• Audit a sample of completed work

The right amount of oversight depends on the system's performance and the consequences of failure.

As evaluations show that a capability has become more reliable, teams may be able to reduce the amount of human review. Other workflows may always require approval because of regulatory, financial, or operational requirements.

Escalation is the process of handing a task from the AI system to a person or another system when the model should not complete it on its own.

Good escalation rules define the situations where additional review is required.

These might include:

• Low confidence
• Missing information
• High-risk decisions
• Conflicting instructions
• Requests outside the model's permitted scope
• Tool or system failures
• Specific categories of sensitive work

Escalation should be tested during evaluation. It is important to measure both whether the model escalates when it should and whether it unnecessarily escalates tasks it could have completed safely.

Model updates can change system behavior even when your own application has not changed.

A new version might improve overall performance while introducing regressions on specific workflows. Changes to latency, formatting, tool use, or instruction following can also affect production systems.

Teams should rerun important evaluations when changing model versions or when a provider makes a meaningful update.

Keeping a stable regression suite makes this much easier. It lets you quickly check whether the new model still meets the requirements that mattered before.

In enterprise AI, teams sometimes use "drift" broadly to describe a change in performance over time.

That change can have several causes:

• The model or provider changed
• User behavior changed
• The mix of tasks changed
• Company policies or data changed
• Retrieval sources changed
• Prompts or tools were updated
• New edge cases appeared in production

Monitoring should help determine whether performance is changing and why.

When a meaningful shift appears, teams can add representative examples to the offline eval set and use them to test future versions.

Important production failures should become inputs to the evaluation and improvement process.

A useful workflow is:

1. Capture the failure.
2. Determine what the system should have done.
3. Identify the underlying failure mode.
4. Determine whether similar cases exist.
5. Add representative examples to the eval set.
6. Test potential fixes.
7. Measure whether the fix generalizes.
8. Monitor the behavior after deployment.

This creates a connection between production monitoring and model development.

Over time, the evaluation suite becomes a record of the capabilities and failure modes the business cares about most.

Surge can help you analyze the most common failure modes of your product and augment your evaluation sets to reduce errors in the future.

The right cadence depends on how quickly the system and its environment change.

Reevaluation is particularly useful when:

• You change models
• You change prompts or system architecture
• You add new tools
• You introduce new workflows
• A provider releases a significant model update
• Production monitoring reveals a new failure mode
• Usage patterns change
• Business rules or policies change

Teams can maintain a stable core evaluation suite for regression testing while continuously adding new cases from production.

Reliability is broader than average model quality.

A production system also needs to behave predictably across different users and scenarios, recover gracefully when something goes wrong, and avoid severe failures even when the overall success rate is high.

A useful reliability program looks at:

• Average performance
• Worst-case and high-severity failures
• Performance across different task categories
• Consistency over time
• Tool and infrastructure failures
• Escalation behavior
• Recovery from unexpected situations

The acceptable level of reliability depends on the application. A drafting assistant and an autonomous system moving money should have very different thresholds for deployment.

Surge can help enterprises build evaluation systems that continue working after an AI product launches.

That can include:

• Designing production quality metrics
• Creating regression suites
• Building human and expert review programs
• Developing model graders and validating them against human judgments
• Identifying and categorizing production failure modes
• Evaluating agents and tool-using workflows
• Designing escalation and approval criteria
• Turning production failures into new evaluation and training data
• Comparing system versions as models, prompts, and workflows change

The goal is to give teams an ongoing view of how their AI systems are performing and a clear process for improving them as new problems emerge.