An AI agent that can only prepare a draft has a different risk profile from one that can update a customer record, change code or send an external message. Agentic AI governance starts by making that difference explicit.
For businesses commissioning custom software, the practical questions are: what can the agent read, what can it change, who approves consequential actions and how can its work be investigated or stopped? The answers belong in the application and operating process as well as the prompt.
This checklist focuses on those controls. For choosing an initial workflow, use the AI agent business-process automation guide; for reviewing code changes, use the AI coding-agent delivery guide.
Understand the control layer around the model
The Australian Signals Directorate describes an agentic AI harness as the software connecting a model with organisational data, tools and systems. Its agentic AI harness guidance emphasises restricted access, layered controls and human accountability. A prompt that says “be careful” cannot enforce a database permission or prevent an external API call.
1. Define an owner and an action boundary
Write a short use-case record before choosing tools. Name the business owner, technical owner, users, inputs, outputs and intended benefit. List the actions the agent may perform directly, those it may propose and those outside the scope.
“Can draft a response” should not accidentally become “can send a response.” “Can prepare a patch” should not imply permission to merge it or release it. Approval should identify the actual destination, proposed change and consequence so the reviewer can make a useful decision.
2. Check permissions at the tool boundary
Use narrow tools for the defined task. An enquiry-triage agent may need selected ticket fields, not an unrestricted CRM export. A report assistant may need read-only access to an approved view, not permission to run arbitrary database commands.
Enforce the calling user’s access and the agent’s task scope in software. Test what happens when it requests a record outside that scope, supplies an unexpected identifier or asks a tool to perform an unsupported action. A model’s refusal is useful behaviour, but the server should reject an unauthorised request even when the model does not.
3. Treat documents and retrieved text as untrusted input
A web page, email, attachment or code comment can contain instructions aimed at an agent. Australia’s software development guidance specifically addresses task-scoped agent authorisation and untrusted retrieved content. Sanitising a document does not make its instructions authoritative.
Include hostile and misleading inputs in testing: a document asking the agent to disclose secrets, a ticket requesting a change outside its scope, an outdated policy presented as current. Verify the denied action and escalation path, not just the wording of the reply.
4. Know where business data goes
Record the information sent to the provider, its sensitivity, storage and retention settings, who can access it and how it is removed. Keep credentials outside prompts and minimise the data included in each task.
The OAIC’s guidance on commercially available AI products covers product due diligence, oversight and personal information in both inputs and outputs. Identify the privacy obligations that apply to the organisation and use case before including personal information. A private account or an Australian user does not by itself establish the provider’s data handling arrangements.
5. Design for uncertain and interrupted work
Set limits on work, time and expenditure. Define what happens when a source is unavailable, information conflicts, an integration times out or the process stops after making a request. Retrying a draft is different from retrying a payment or public message.
Record enough context to establish what was attempted and what the destination system confirmed, with access and retention appropriate to the data. Where an external action has an ambiguous outcome, reconciliation may be needed before a retry. Someone should own that queue and be able to disable the integration.
First-hand observation
Kitek’s editorial worker creates review-only WordPress drafts; its importer has no publish operation. A person checks the sources, claims, copy and artwork before publication. A separate social worker records a publishing attempt before sending it. If the outcome is uncertain, it stops for reconciliation rather than automatically risking a duplicate post. These are explicit controls around an internal content workflow, not a claim that generated copy is always correct.
6. Make the pilot acceptance record reviewable
Before enabling a live workflow, assemble a small record that a business owner and engineer can both understand:
- The approved task and permission boundaries, including prohibited actions.
- Representative normal, incomplete, conflicting and hostile inputs.
- The observed result for each case and any corrections required.
- The evidence shown to an approver before a write or external action.
- Failure, retry, stop and recovery behaviour.
- The support owner and the conditions that trigger reassessment.
Revisit that record when the model, prompt, tools, data sources or permissions change. Adding a new write capability changes the decision even if the chat interface looks the same.
Start with an accountable integration
Kitek can help define the workflow and build the surrounding controls through workflow automation and AI integration and custom software development. For an existing application, include its ongoing support and maintenance arrangements in the design: an agent integration needs an owner after the pilot ends.