元学习应用，在神经网络方面的应用

1

Meta-Learning in Neural Networks: A Survey

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, Amos Storkey

Abstract—The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent years. Contrary to

conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta-learning aims to improve the

learning algorithm itself, given the experience of multiple learning episodes. This paradigm provides an opportunity to tackle many

conventional challenges of deep learning, including data and computation bottlenecks, as well as generalization. This survey describes

the contemporary meta-learning landscape. We first discuss definitions of meta-learning and position it with respect to related fields,

such as transfer learning and hyperparameter optimization. We then propose a new taxonomy that provides a more comprehensive

trained from scratch for a specific task using a fixed learn-

ing algorithm designed by hand. Deep learning-based ap-

proaches specifically have seen great successes in a variety

of fields [1]–[3]. However there are clear limitations [4]. For

example, successes have largely been in areas where vast

quantities of data can be collected or simulated, and where

huge compute resources are available. This excludes many

applications where data is intrinsically rare or expensive [5],

or compute resources are unavailable [6].

Meta-learning provides an alternative paradigm where

where learning strategies improve both on a lifetime and

evolutionary timescales [8]–[10].

Historically, the success of machine learning was driven

by the choice of hand-engineered features [11], [12]. Deep

learning realised the promise of joint feature and model

learning [13], providing a huge improvement in perfor-

mance for many tasks [1], [3]. Meta-learning in neural

networks can be seen as aiming to provide the next step

of integrating joint feature, model, and algorithm learning.

Neural network meta-learning has a long history [7],

supervised learning. Meta-learning has proven useful both

in multi-task scenarios where task-agnostic knowledge is

T. Hospedales is with Samsung AI Centre, Cambridge and University of Edin-

burgh. A. Antoniou, P. Micaelli and Storkey are with University of Edinburgh.

ing of new tasks from that family [7], [16]; and single-task

improved over multiple episodes [17]–[19]. Successful appli-

cations have been demonstrated in areas spanning few-shot

image recognition [16], [20], unsupervised learning [21],

learn as occurring when a learner’s performance at solving

tasks drawn from a given task family improves with respect

to the number of tasks seen. (cf., conventional machine

learning performance improves as more data from a single

task is seen). This perspective [27]–[29] views meta-learning

as a tool to manage the ‘no free lunch’ theorem [30] and im-

prove generalization by searching for the algorithm (induc-

tive bias) that is best suited to a given problem, or problem

family. However, this definition can include transfer, multi-

task, feature-selection, and model-ensemble learning, which

per [27], [28], but focus specifically on where this is achieved

by end-to-end learning of an explicitly defined objective func-

tion (such as cross-entropy loss). Additionally we consider

single-task meta-learning, and discuss a wider variety of

(meta) objectives such as robustness and compute efficiency.

This paper thus provides a unique, timely, and up-to-

date survey of the rapidly growing area of neural network

meta-learning. In contrast, previous surveys are rather out

of date and/or focus on algorithm selection for data mining

[27], [31], [34], [35], AutoML [32], [33], or particular appli-

neural architecture search [37].

2

in terms of meta-representation, meta-objective and meta-

optimizer. This framework provides a design-space for de-

veloping new meta learning methods and customizing them

for different applications. We survey several popular and

emerging application areas including few-shot, reinforce-

ment learning, and architecture search; and position meta-

learning with respect to related topics such as transfer and

multi-task learning. We conclude by discussing outstanding

challenges and areas for future research.

2 B ACKGROUND

ing to learn, which refers to the process of improving a

learning algorithm over multiple learning episodes. In con-

trast, conventional ML improves model predictions over

multiple data instances. During base learning, an inner

(or lower/base) learning algorithm solves a task such as

image classification [13], defined by a dataset and objective.

During meta-learning, an outer (or upper/meta) algorithm

updates the inner learning algorithm such that the model

it learns improves an outer objective. For instance this

objective could be generalization performance or learning

could fall within the definition of meta-learning. The

salient characteristic of contemporary neural-network meta-

learning is an explicitly defined meta-level objective, and end-

to-end optimization of the inner algorithm with respect to

this objective. Often, meta-learning is conducted on learning

episodes sampled from a task family, leading to a base

learning algorithm that performs well on new tasks sampled

from this family. However, in a limiting case all training

episodes can be sampled from a single task. In the following

section, we introduce these notions more formally.

(1)

The conventional assumption is that this optimization is

This setup leads to analogies of conventional underfit-

ting and overfitting: meta-underfitting and meta-overfitting. In

particular, meta-overfitting is an issue whereby the meta-

knowledge learned on the source tasks does not generalize

to the target tasks. It is relatively common, especially in

the case where only a small number of source tasks are

solutions to the source tasks.

3

by casting the meta-training step as a bilevel optimization

problem. While this picture is arguably only accurate for

the optimizer-based methods (see section 3.1), it is helpful

to visualize the mechanics of meta-learning more generally.

Bilevel optimization [40] refers to a hierarchical optimiza-

tion problem, where one optimization contains another

optimization as a constraint [17], [41]. Using this notation,

meta-training can be formalised as follows:

we can use D

are usually used during meta-training.

Meta-Learning: Feed-Forward Model View As we will

see, there are a number of meta-learning approaches that

synthesize models in a feed-forward manner, rather than via

X

i

Here we meta-train by optimizing over a distribution of

tasks. For each task a train and validation set is drawn. The

the linear regression weights to predict examples x from the

vary in the complexity of the predictive model g used, and

how the support set is embedded [44] (e.g., by pooling,

CNN or RNN). These models are also known as amortized

[45] because the cost of learning a new task is reduced

to a feed-forward operation through g

Meta-learning and learning-to-learn first appear in the lit-

erature in 1987 [14]. J. Schmidhuber introduced a family of

methods that can learn how to learn, using self-referential

learning. Self-referential learning involves training neural

networks that can receive as inputs their own weights and

predict updates for said weights. Schmidhuber proposed to

learn the model itself using evolutionary algorithms.

Meta-learning was subsequently extended to multiple

areas. Bengio et al. [46], [47] proposed to meta-learn biolog-

ically plausible learning rules. Schmidhuber et al.continued

more extensions in 2001 [51], [52], with [27] giving an

overview of the literature at that time. Meta-learning was

used in the context of reinforcement learning in 1995 [53],

followed by various extensions [54], [55].

Here we position meta-learning against related areas whose

relation to meta-learning is often a source of confusion.

Transfer Learning (TL) TL [34], [56] uses past experi-

ence from a source task to improve learning (speed, data

efficiency, accuracy) on a target task. TL refers both to

without the use of a meta-objective. In meta-learning, the

corresponding prior would be defined by an outer opti-

mization that evaluates the benefit of the prior when learn

a new task, as illustrated by MAML [16]. More generally,

meta-learning deals with a much wider range of meta-

representations than solely model parameters (Section 4.1).

Domain Adaptation (DA) and Domain Generalization

(DG) Domain-shift refers to the situation where source

and target problems share the same objective, but the input

distribution of the target task is shifted with respect to the

source task [34], [58], reducing model performance. DA is

a variant of transfer learning that attempts to alleviate this

unlabeled data from the target. DG refers to methods to train

4

objective to optimize ‘how to learn’ across domains. Mean-

while, meta-learning methods can be used to perform both

DA [59] and DG [42] (see Sec. 5.8).

Continual learning (CL) Continual or lifelong learning

[60]–[62] refers to the ability to learn on a sequence of tasks

drawn from a potentially non-stationary distribution, and

in particular seek to do so while accelerating learning new

framework to advance continual learning, and a few recent

studies have begun to do so by developing meta-objectives

that encode continual learning performance [63]–[65].

Multi-Task Learning (MTL) aims to jointly learn sev-

eral related tasks, to benefit from regularization due to

parameter sharing and the diversity of the resulting shared

representation [66]–[68], as well as compute/memory sav-

ings. Like TL, DA, and CL, conventional MTL is a single-

level optimization without a meta-objective. Furthermore,

the goal of MTL is to solve a fixed number of known tasks,

or regularization strength describe ‘how to learn’. Here we

include HO tasks that define a meta objective that is trained

end-to-end with neural networks, such as gradient-based

hyperparameter learning [69], [71] and neural architecture

search [18]. But we exclude other approaches like random

search [72] and Bayesian Hyperparameter Optimization

[73], which are rarely considered to be meta-learning.

Hierarchical Bayesian Models (HBM) involve Bayesian

is written as a conditional density on some other variable

h

i

Learning is usually full-pipeline, but using some form of

Bayesian marginalisation to compute the posterior over

Q

exact due to the choice of conjugate exponential models,

in others (see e.g. [75]), a stochastic variational approach is

used to calculate an approximate posterior, from which a

lower bound to the marginal likelihood is computed.

Bayesian hierarchical models provide a valuable view-

most meta-learning work considers complex inner-loop

learning processes, involving many iterations. Nonetheless,

some meta-learning methods like MAML [16] can be under-

stood through the lens of HBMs [76].

AutoML: AutoML [31]–[33] is a rather broad umbrella

for approaches aiming to automate parts of the machine

learning process that are typically manual, such as data

preparation, algorithm selection, hyper-parameter tuning,

and architecture search. AutoML often makes use of numer-

ous heuristics outside the scope of meta-learning as defined

Previous [77], [78] categorizations of meta-learning meth-

ods have tended to produce a three-way taxonomy across

optimization-based methods, model-based (or black box)

methods, and metric-based (or non-parametric) methods.

Optimization Optimization-based methods include those

where the inner-level task (Eq. 6) is literally solved as

an optimization problem, and focuses on extracting meta-

mance. A famous example is MAML [16], which aims to

, such that a small number

activation state, with predictions for test data being made

based on this state. Typical architectures include recurrent

networks [39], [51], convolutional networks [38] or hyper-

networks [83], [84] that embed training instances and labels

of a given task to define a predictor for test samples. In this

case all the inner-level learning is contained in the activation

states of the model and is entirely feed-forward. Outer-

RNN or hypernetwork parameters. The outer and inner-

explicit storage buffer and can be seen as a model-based