Before training an AI agent, decide what feedback it can learn from

A course map is useful when it changes your first project

Hugging Face published a curriculum for post-training agents on October 20. It gathers material on supervised fine-tuning, distillation, reinforcement learning, environments, and agent harnesses. Its central distinction is useful even if you never launch a training run: different approaches depend on different kinds of feedback.

An agent that calls tools and works across several turns leaves more to evaluate than its final sentence. Did it use the right input? Did it notice an error? Did it change something it was not supposed to change? The curriculum is an invitation to examine those sequences, rather than treat an agent as a chatbot with a longer answer.

Our practical interpretation is that the best first training question is often an inventory question. What reliable signals do you already possess? Clean demonstrations, access to a suitable teacher, and a checkable outcome are different assets. Calling all three data can hide the work needed to make them usable.

The exercise in this article is a proposed learning project, not a tested recipe or a claim about training performance. It uses an imaginary coding task to make the choices concrete. You can do the planning on paper, with no paid model, GPU rental, or new software purchase.

Source: Hugging Face: A curriculum for post-training agents ↗ · Hugging Face TRL: SFT Trainer documentation ↗ · Hugging Face TRL: GRPO Trainer documentation ↗ · Python: unittest documentation ↗ · DK: Beginner's Step-by-Step Coding Course, 2020 edition publisher description ↗

Start with a task small enough to explain

Imagine an agent helping a student club clean up event descriptions. Each input includes a title, a date field, and a short description. The desired output is a consistent record. The agent may inspect examples and run a validator, but it may not publish events or contact attendees.

Write a task contract before thinking about models. Require that the original meaning survive, that missing dates remain missing, and that unexpected fields be reported rather than silently discarded. Include a few deliberately awkward inputs: an empty description, a date written ambiguously, and two events with similar titles.

These choices determine what success means. If you reward only whether the output parses, an agent could produce an empty but well-formed record. If you reward only the presence of a date, it could invent one. If you reward a polished tone, it could rewrite away an important limitation.

A small task makes those tradeoffs visible. That is why a learning project should have a result you can inspect by hand. You are learning how to design feedback, not trying to automate an entire organization on the first afternoon. A precise miniature can teach more than an ambitious demo whose failures are impossible to locate.

TRL's SFT Trainer documentation describes supported training-data formats, including conversational and prompt-completion examples. That provides a concrete place to inspect what your examples would look like in a real training workflow. Formatting matters because a useful conversation, a useful target completion, and an accidental transcript dump are different things.

For the student-club exercise, propose ten demonstrations. Each should show the input, the permitted steps, the result of any check, and the final record. Then review them for contradictions. Does one example guess a date while another preserves uncertainty? Does one delete an unfamiliar field while another asks for review?

Do not hide those conflicts by producing a larger dataset. Resolve the policy first. A hundred examples containing two incompatible definitions of correct behavior are not a more precise teacher than ten examples with one clear definition. Quantity can wait until you know what behavior you want to repeat.

Also include a reviewer note outside the training target explaining why each demonstration is acceptable. That note can help your team audit the dataset without requiring the model to reproduce a long explanation. For this exercise, the point is to make your own standards visible before turning them into examples that another system will imitate.

A teacher helps only when you know what to check

The curriculum distinguishes imitation of saved traces from learning with a teacher's outputs or probabilities. It also highlights that training on successful demonstrations alone does not give a model experience recovering from its own mistakes. Those are reasons to inspect the kind of supervision, rather than assume the word distillation guarantees a smaller model with all the teacher's strengths.

In the club example, imagine a teacher producing a proposed record and a reason for each correction. Your job is still to verify both. If the source description says registration opens later, the teacher should not convert that into an open registration status. A convincing explanation is another output to review.

Try a paper audit before generating more examples. Give a reviewer five fictional teacher outputs, including one with a subtle invented detail. Ask which evidence supports every added field. If the reviewer cannot reliably identify the unsupported detail, making the teacher faster will not solve the supervision problem.

A reasonable stopping point is to decide that a deterministic validator and human review are enough for the task. Training is a possible intervention, not the required destination. The feedback inventory may reveal that your problem is unclear instructions, inconsistent source records, or missing checks. Those are worthwhile findings even when they lead to no model update.

Make a score earn your trust

TRL's GRPO documentation supports custom reward functions. In practical terms, this lets a project define what gets counted as a better generated result. The existence of that interface does not make your scoring rule a good representation of the task.

Draft a scoring proposal for the imaginary event cleaner. Award credit for retaining the supplied title, preserving a known date, and flagging an ambiguous date. Keep separate checks for output structure and factual preservation. Instead of immediately merging everything into one number, show the components to a reviewer.

Now attack your own score with three fake outputs. One should be empty, one beautifully written but wrong, and one correct except for a missing uncertainty flag. Ask whether the score puts them in an order you would defend. If it does not, change the evaluator before discussing optimization.

This is a useful exercise because optimizing a score is meaningful only relative to what the score represents. A validator that checks punctuation can be excellent at punctuation while missing the substance you care about. Naming the narrow job of each check helps prevent a small technical success from being mistaken for a broad improvement in judgment.

Python's unittest documentation offers assertions for checking equality, truth conditions, and expected exceptions, along with ways to organize repeatable test cases. A beginner does not need an agent-training system to start learning those ideas. A tiny program with explicit expected outputs is already a laboratory for feedback design.

For example, write down how a date-preservation function should behave for a known date, a missing value, and ambiguous text. First predict the outputs yourself. Then ask an AI coding assistant to explain a proposed implementation line by line, identifying assumptions instead of merely supplying a completed file.

If you prefer a printed introduction, DK’s Beginner’s Step-by-Step Coding Course (affiliate link) is an optional starting point; the publisher describes project-based programming tutorials and exercises. Free language documentation and a notebook are sufficient for the planning exercise here, and the book is not an agent-training manual.

As an Amazon Associate I earn from qualifying purchases. This optional recommendation is not a claim of hands-on testing.

Keep the goal modest: understand one function well enough to create a case it should pass and a case it should reject. If you cannot explain why the expected result is correct, adding an AI assistant to the loop will not make the expectation more trustworthy. It may simply make the misunderstanding harder to spot.

The environment is part of the assignment

Suppose the event agent can read a folder, edit a draft, and invoke a checker. List those actions and the state they change. Put a disposable copy of the input records in the exercise design, and define what resets between attempts. Make it possible to tell whether one run benefited from changes left by an earlier run.

That matters for interpreting progress. An agent that sees a correction left in a workspace is facing a different problem from an agent starting fresh. A score that ignores those conditions can make an exercise appear easier without revealing why. Your proposed environment should make the starting point legible.

Write an end condition as well. The task ends when the output is valid and evidence-preserving, when the allowed attempt budget is reached, or when the agent encounters information the contract says it cannot resolve. Stopping with a clear uncertainty report should count differently from claiming completion with a guessed answer.

This environment design is our suggested workshop activity. It does not require running a particular framework, and it is not a replacement for one framework's setup instructions. It prepares you to read those instructions with a clearer understanding of which parts represent the world the agent will experience.

Create a small second set of fictional event records that the person writing examples does not use while refining the task. Include the same kinds of difficulty without copying the exact surface details. A differently worded missing-date case is more informative than the same case with a new title.

When you review a proposed solution, record outcomes by case type. Perhaps ordinary records are handled well while ambiguity is missed. That finding suggests a different next step from a uniform failure to produce valid structure. A single combined score would make it harder to choose where to work.

Do not present a tiny classroom set as a general benchmark. It is a diagnostic aid for your own exercise. Note who wrote the cases, which were seen during revisions, and whether any expected answers changed. If you correct a mistaken expected answer, retain a record of the correction rather than pretending the exercise was flawless.

You can also ask a second person to invent one challenge the first person did not anticipate. This adds an independent perspective without requiring a large dataset. The point is to practice distinguishing improvement on familiar examples from a solution that survives a slightly different but still legitimate demand.

Choose the next step from the failure you observed

After the paper workshop, describe one failure precisely. Avoid the sentence the agent is bad at dates. A better statement is that the proposed process turns ambiguous day-month strings into definite dates without asking for context. That statement identifies the behavior and the missing information.

Then choose the smallest intervention that could address it. Clarifying the output contract may help if your instructions were vague. Adding examples may help if the intended behavior was absent. A better check may help if the wrong output was being rewarded. A different tool may help if the task required information the system could not access.

These are hypotheses to test, not guaranteed fixes. Write down what result would convince you an intervention worked and what new failure would make you reconsider. That turns a training discussion into a sequence of understandable experiments rather than a search for the most fashionable method.

The curriculum is useful because it gives names and resources for several learning paths. Your job is to connect a path to the feedback your project can actually supply. A well-designed first project may end with clearer demonstrations, a better evaluator, or a decision to avoid training for now. Each can be progress if it makes the eventual system easier to understand and judge.