Skill transfer depends on task-distribution design, not more demonstrations

I think skill transfer is decided before the first demonstration is shown—by the task distribution chosen for training. More examples drawn from the same narrow pattern mostly make that pattern easier to recognise. They do not teach an agent how to infer what changed, preserve what matters, or adapt with little new data. If I want transfer, I have to design a family of tasks whose variation exposes the reusable structure.

Meta-learning makes the distinction concrete. The learner adapts inside a task; the meta-learner improves the starting point, metric, memory, or update rule across tasks. Episodic training therefore alternates between a support set for adaptation and a query set for judging whether that adaptation worked. The outer loop is not rewarded for memorising one answer. It is rewarded for producing an inner loop that learns a new task quickly. That is the technical version of measuring skill transfer rather than answer throughput.

The task distribution is the curriculum and the specification at once. If every episode shares the same surface cues, the system can win by encoding those cues. If tasks vary without preserving any common structure, there is nothing stable to transfer. Useful design sits between those failures: enough shared structure for an inductive bias to form, enough variation that the bias must survive unfamiliar instances. Task sampling is therefore not neutral plumbing. Like training-data mixtures that express product priorities as weights, it decides which adaptations deserve finite learning capacity.

The three meta-learning families change the mechanism, not this dependency. Metric-based methods learn an embedding geometry so a new example can be classified by distance or prototypes. Model-based methods adapt through hidden state, fast weights, or memory reads. Optimisation-based methods such as MAML, FOMAML, and Reptile learn parameters from which a few gradient steps work well. Each can reduce the data needed for a new task, but each learns only from the relationships represented in its episodes. A clever adaptation rule cannot recover task variation the curriculum never contained.

For agents, I would design variation along operational dimensions: goals, observations, rewards, environment dynamics, available context, and the evidence needed to infer which task is active. Domain randomisation broadens conditions deliberately. POET co-evolves environments and solutions rather than freezing a benchmark. DIAYN discovers distinguishable skills without externally supplied task labels. Meta-RL methods go further by learning exploration and task inference—sometimes through recurrent state, sometimes through an inferred context variable. The shared move is to train adaptation against a distribution, not rehearse a canonical trace.

This also changes assessment. Hold out task combinations, not merely examples. Measure environment interactions or adaptation steps before competence, not just final accuracy. Test whether the agent can identify latent task context, explore usefully, and reuse prior episodes without confusing resemblance for identity. A benchmark should reveal whether the system found a transferable manifold or memorised the training neighbourhood. That extends grading the seams of skill composition: the critical seam here is between recognising a new task and selecting how to adapt.

One precise concession: when deployment truly repeats a fixed, narrow task and its input distribution is stable, more representative demonstrations may be the cheapest route to better performance. The transfer claim begins when the system must face new tasks, sparse feedback, changed dynamics, or unfamiliar combinations. At that boundary, demonstration count without distribution design buys confidence on yesterday’s neighbourhood, not readiness for tomorrow’s task.

I would therefore review a transfer curriculum the way I review an interface contract. What remains invariant across tasks? Which dimensions vary? How is task context revealed or inferred? What counts as successful adaptation, and how many samples may it consume? Which held-out tasks would falsify the claim? Progressive complexity in training design matters here because each episode should make the next adaptation demand legible. Skill transfer is not a volume effect. It is an outcome engineered by the structure of the tasks.