Benefits of Continual Learning¶
In this post, we will explore how Continual Learning can be a beneficial property of Artificial Intelligence systems.
Over the years I have tried to convince people that our current approach to Artificial Intelligence (AI) is flawed, and it does not address a fundamental property we need for AI systems. I believe we need to make Continual learning the main focus when developing AI rather than treat it like an afterthought. In most of my discussions I find that people either misunderstand what continual learning is or do not understand the benefits it brings or the flaws AI systems would have without it.
Let's start by defining what I mean by Continual Learning.
Continual Learning is when a system learns on the fly, this means there are no differences between "training" and "testing" or "inference" phase for the system, every input is a learning experience.
Current systems don't work like this, they tend to learn offline during their training phase, then the systems are "locked" before they can be used for testing or inference. More recent systems now support Test Time Training (TTT) where they learn after being trained but this still fails the fundamental requirement as it is not the way the system operates normally. If we forced current systems to do TTT every time, they would degrade in capability without human engineering support. This is because the fundamental learning algorithm used in these systems don't support online learning or non-stationary distributions out of the box. These systems tend to suffer from catastrophic forgetting if not carefully trained with a human in the loop.
Continual Learning is a fundamental property of an AI system. Artificial Intelligence capability is achieved through learning, and having a system that can learn on the fly can improve and maintain its intelligence capability for as long as it is operational. If we were to develop a system that learns without this property it's capabilities would basically become static and may not be able to function in highly dynamic environments.
Let's explore the benefits we get from Continual Learning.
1. Amortized cost of learning¶
When AI systems learn about the world, they pick up patterns and use them to generalize to different situations. These systems tend to experience different kinds of data distributions under different conditions. A simple example could be an AI system that is tasked with classifying objects, it would be unreasonable to assume we could cover the distribution of all possible things to recognize. This could be because of various reasons, for example, we could be limited by the amount if data we can collect, or we may expand the list of objects we want the system to be able to recognize later.
Humans are general purpose object recognizers, this is because they can always expend the class of objects they can recognize by just learning new ones. Because they work this way, they don't overfit to a specific class of objects and exhaust all their parameters only learning features for the limited class. Instead, they keep a more open mind and learn robust features and patterns that don't just help with the classes they have learnt but enable learning new classes. This is the Amortized cost of learning. Basically the cost of learning class A and class B this way is much cheaper than learning from scratch.
2. Better Generalization due to less inductive bias¶
A lot of current AI learning methods, don't work out of the box, they usually require a lot of human engineering. This problem extends to the limitations of the learning algorithm. When an AI system is trained, and it falls short in its capabilities, it can't just learn its way out of it. The current systems usually require human intervention, not to teach it but to modify it in an invasive way.
Because of the heavy handholding humans have to do even with the learning algorithms, we end up adding inductive biases that the system can leverage to learn. These biases can become crutches that limit the model as they can't easily see past them when they become a problem for learning. A Continual Learning system would most likely have as little bias as possible, and tend to learn its own biases, this way it can discard them when they don't prove useful. Because the system is learning them as part of its learning process rather than a human giving them beforehand, the system can adapt accordingly and learn biases that generalize better.
3. Resilience to change in distribution¶
Good Continual Learning systems have to assume non-stationary distributions to generalize well. They can't be sure their model of the world is not biased to the limited data they have seen. For example, a system that has only ever seen white swans should still be able to learn black swan when it eventually sees one. This is the stability-plasticity dilemma.
Stability: When a system is learning, we expect it to converge on some final form over many observations. Plasticity: But when it encounters new information that does not fit its model of the world, it should be able to adapt accordingly.
Continual Learning basically promises plasticity while providing good stability. A model without this would not be able to learn new information
I hope the few points above were good enough reasons to convince you that artificial intelligence as a learning problem and not a skill acquisition endeavour. We need to shift our focus from measuring how well these systems perform on tasks to how well they learn tasks. Thanks for taking the time to read this post, see you in the next one, bye.