Scaling agentic AI: Seven controls for responsible AI

1. Design for the full operating population

 

Overall accuracy can hide important differences in performance. An agent may work well for common transactions while struggling with less frequent categories, incomplete documents, regional terminology, or unusual exceptions.

 

Testing should reflect the conditions the agent will face in the real world, including:

 

• High-, medium-, and low-volume scenarios

• Rare cases and high-impact exceptions

• Incomplete or conflicting inputs

• Different languages, formats, and communication styles

• Regional terms and abbreviations

• Accessibility needs

   

Teams should assess false positives, false negatives, escalation rates, and differences in outcomes by category. When the results show a material disparity, the organization should investigate it before release.

 

This approach gives leaders a clearer view of where the agent performs reliably, where it needs human support, and who could be affected when it fails.

 

2. Separate reasoning from execution

 

Language models produce probabilistic outputs. Enterprise transactions demand precision.

 

An answer that sounds plausible should not automatically become a payment, contract, customer update, or system-of-record change. The workflow needs a controlled boundary between what the agent recommends and what the business executes.

 

Start by grounding the agent in trusted, current, and approved enterprise sources. Then require structured outputs so the workflow can validate critical fields independently.

 

Deterministic business rules should verify values such as:

 

• Identity

• Amount

• Eligibility

• Pricing

• Dates

• Approval authority

   

Send low-confidence, incomplete, or conflicting cases to a qualified reviewer. Confidence alone should never authorize a high-impact action.

 

The design also needs a safe way to stop. If evidence is missing, a system is unavailable, or validation fails, the agent should pause or escalate instead of inventing an answer. Establish transaction limits, rollback procedures, retry rules, and emergency suspension controls before production.

 

3. Protect information before inference

 

The safest sensitive data is the data that never reaches the model.

 

Before the workflow creates a model request, it should identify personal information, financial data, health information, credentials, and confidential commercial terms. Remove information the agent does not need. Mask or replace information required only for continuity with controlled tokens.

 

A practical sequence is:

 

1. Classify the input

2. Identify sensitive information

3. Remove, mask, or tokenize it

4. Invoke the model

5. Validate the output

6. Restore approved values or delete temporary data

    

For example, the workflow could replace names, account numbers, and email addresses with controlled placeholders. The agent reasons with those placeholders while the original values remain inside the governed workflow boundary.

 

The same discipline applies to agent identity and tool access. Each agent should use an approved identity or governed user context, with least-privilege access, approved model endpoints, secure credential management, connector restrictions, and an allow list of tools.

 

4. Make autonomous activity understandable

 

People need to know what an agent is designed to do, what sources it has used, and when a person must review its work.

 

For a consequential recommendation, an authorized reviewer should be able to see:

 

• The source information

• The values the agent extracted

• The checks the workflow completed

• The basis for the recommendation

• Any exception or uncertainty

• The approval required before action

   

This does not require exposing hidden model reasoning. It requires a clear business explanation supported by evidence.

 

Organizations should also record the prompt, model, workflow, knowledge source, and tool versions associated with an outcome. This allows teams to connect a result to the exact solution configuration in use at the time.

 

5. Match decision rights to business impact

 

Autonomy should be a deliberate business decision. It should not emerge by default as teams connect more tools and workflows.

 

A practical autonomy model can include five levels:

 

• Assist: The agent recommends, and a person acts

• Review before action: The agent prepares the transaction, and an authorized person approves it

• Controlled autonomy: The agent acts within predefined limits

• Exception-based oversight: Routine work proceeds automatically, and deviations escalate

• Prohibited autonomy: The agent cannot perform the specified action

   

Every agent should have a named business owner, an approved risk level, permitted actions, transaction limits, review dates, and retirement criteria.

 

Changes to prompts, models, tools, knowledge sources, or transaction limits should trigger reassessment. Autonomy is not a one-time approval. It is an operating decision that needs active management.

 

6. Build in auditability and traceability

 

Dashboards show what an agent produces. Auditability shows how a consequential outcome was reached and whether the approved controls worked.

 

For each material interaction, capture the following information:

 

• User or service identity

• Input classification and masking status

• Sources retrieved

• Prompt and model versions

• Tools invoked

• Validation results and confidence thresholds

• Human approvals or overrides

• Final system actions

• Exceptions and recovery steps

   

When several agents take part in a process, the trace should preserve handoffs and decisions across the entire workflow. Logging each component separately is not enough if the organization cannot reconstruct the chain of events.

 

Version control matters too. Historical outcomes should link back to the prompts, models, sources, validation rules, and transaction limits in use at that time. This gives internal audit, risk, technology, and operations teams decision-grade evidence for reviews and investigations.

 

7. Clarify who owns governance

 

The controls remain consistent, but ownership changes with the delivery model.

 

For a bespoke client solution, the client's responsible AI framework, risk taxonomy, control standards, and evidence requirements should define the baseline. The delivery team can design, implement, and evidence the controls, while the client's governance authority retains final responsibility for accepting deployment risk.

 

For a Genpact provider-led solution, Genpact's responsible AI baseline and reusable product controls can provide the starting point. The client's governance body should still assess the implementation of its own requirements and retain final deployment acceptance within its environment.

 

In either model, three roles should remain distinct:

 

• Implementation

• Independent assurance

• Final risk acceptance

   

Clear separation makes accountability easier to understand before deployment and easier to demonstrate afterward.