Section: Overall Objectives
SequeL means “Sequential Learning”. As such, SequeL focuses on the task of learning in artificial systems (either hardware, or software) that gather information along time. Such systems are named (learning) agents (or learning machines) in the following. These data may be used to estimate some parameters of a model, which in turn, may be used for selecting actions in order to perform some long-term optimization task.
For the purpose of model building, the agent needs to represent information collected so far in some compact form and use it to process newly available data.
The acquired data may result from an observation process of an agent in interaction with its environment (the data thus represent a perception). This is the case when the agent makes decisions (in order to attain a certain objective) that impact the environment, and thus the observation process itself.
Hence, in SequeL , the term sequential refers to two aspects:
Examples of sequential learning problems include:
- Supervised learning
- Unsupervised learning
- Reinforcement learning
tasks deal with the control (a policy) of some system which has to be optimized (see ). We do not assume the availability of a model of the system to be controlled.
In all these cases, we mostly assume that the process can be considered stationary for at least a certain amount of time, and slowly evolving.
We wish to have any-time algorithms, that is, at any moment, a prediction may be required/an action may be selected making full use, and hopefully, the best use, of the experience already gathered by the learning agent.
The perception of the environment by the learning agent (using its sensors) is generally neither the best one to make a prediction, nor to take a decision (we deal with Partially Observable Markov Decision Problem). So, the perception has to be mapped in some way to a better, and relevant, state (or input) space.
Finally, an important issue of prediction regards its evaluation: how wrong may we be when we perform a prediction? For real systems to be controlled, this issue can not be simply left unanswered.
To sum-up, in SequeL , the main issues regard:
all that being understood in a sequential framework.