Improving Copilot: skills, feedback and iteration

The principle: evolving Copilot during use

A Copilot agent does not improve on its own. It is continuous iteration — via refined prompts, documented skills and structured feedback — that drives its improvement. The question raised in this Microsoft 365 webinar deserves a concrete answer: which mechanisms should be activated so that the agent truly progresses over time, without model retraining or developer intervention.

Three levers stand out: real-time conversational refinement, building reusable skills, and implementing an evaluation loop (thumbs up/down, confidence meter, feedback monitoring tenant).

Refining the response in the conversation

The first level of improvement is the simplest: continue dialoguing with the agent in the same Copilot thread to correct course.

Typical example: an agent generates a deployment plan for three offices, but forgets that one of the sites is international and does not have access to the same models. Rather than launching a new prompt, you add directly: "We have three offices, but one is international and does not have access to all models". The agent reintegrates this constraint and refines its output.

This conversational iteration remains one-off. It does not persist from one session to the next — that is where skills come in.

Skills: capitalize improvement in a document

A skill is a Markdown file that encapsulates reusable instructions for an agent. It does not modify the underlying API calls — the agent continues to query the same services (Microsoft 365 Copilot, Azure AI Foundry, or the cognitive engines used in the background) — but it controls how these calls are orchestrated and interpreted.

Practical points to remember:

• You can ask Copilot directly to generate a skill template: structure in approximately eight sections to complete.
• A skill can integrate quality rules, capitalized feedback from experience, and correction instructions learned over time through uses.
• The file is stored in a SharePoint folder, which allows it to be governed with the usual SharePoint permissions and retention mechanisms (access control, versioning, content lifecycle).
• Skills apply across the board: Azure, Copilot in Microsoft 365 applications, and custom agentic builds.

Closing the loop with evaluation: thumbs up/down and confidence meter

Continuous improvement also goes through measurement. Two native mechanisms deserve to be exploited systematically:

• Thumbs up / thumbs down in the Copilot interface: this signal feeds satisfaction metrics and, if feedback monitoring is enabled at the tenant level, it becomes visible to adoption specialists and administrators.
• Net Promoter Score (NPS): still used as a complement to assess overall perception of productivity services, but less granular than feedback per interaction.

The business objective is what speakers call "hill climbing": reducing the rate of unsatisfactory responses while increasing the satisfaction rate, in a progressive and measurable way.

A simple personal practice to generalize in Copilot instructions:

• Systematically ask the agent for the "best next prompt" to progress on the subject being discussed.
• Request a confidence level on a scale of 1 to 10 for each response, to decide whether to dig deeper or verify sources.

These two requests can be entered once and for all in Copilot's custom instructions, so you don't have to repeat them at each session.

Knowledge sources: keeping the agent up to date

An agent connected to static content (a single reference document, for example) quickly plateaus in relevance. For an agent with broad scope, the recommendation is to connect it to a living SharePoint site, regularly updated, rather than to a static corpus.

This distinction has a direct design impact:

• Agent with restricted and stable scope → a single well-structured document is sufficient.
• Generalist agent intended to answer on an evolving domain → connection to a continuously refreshed data source (SharePoint site, living document library).

The choice of underlying model is also part of iterative improvement: asking the agent itself whether there is a more efficient way to handle a task, or if another model would be better suited to the scenario, is an integral part of the optimization process.

Creating a culture of continuous improvement

The technical dimension is not enough. Improvement of Copilot agents progresses faster in organizations that structure the sharing of experience between users.

Observed best practices:

• Organize monthly roundtables with some prepared content, but especially free time for questions and prompt sharing.
• The champion or facilitator must lead by example by sharing their own obstacles: "here is what caused me problems this week".
• Create a prompt-sharing channel (Teams, distribution list, or any other medium) where everyone posts their discoveries, including prompts unrelated to work — these playful examples lower the barrier to entry for hesitant users.

Each user starts from a different starting point in terms of data, experience and perspective. An initial prompt will vary widely from person to person — this is not a problem, it is the basis for personalized learning comparable to individual coaching available continuously.

Key takeaways

• Copilot's self-improvement is based on three combinable mechanisms: immediate conversational iteration, persistent skills in Markdown governed via SharePoint, and feedback loop (thumbs up/down, confidence meter).
• Skills do not change underlying API calls: they orchestrate the use of intelligence already available via Microsoft 365 Copilot or Azure AI Foundry.
• Connecting an agent to a living SharePoint source rather than a static document improves its relevance over time.
• Tenant feedback monitoring makes user evaluations visible to administrators: activate with full awareness.
• Adoption culture (roundtables, prompt sharing, tolerance for error) accelerates collective skill development as much as technical adjustments.

For more information on agent governance and their deployment at the tenant scale, consult the official documentation on Microsoft Copilot Studio and on governance of Microsoft 365 Copilot agents.