Catastrophic Forgetting

Catastrophic forgetting refers to the phenomenon where a neural network forgets previously learned information upon learning new information. This issue is particularly prevalent in sequential learning tasks, where models are trained on a series of tasks over time. When a model is trained on a new task, it can overwrite the weights associated with the earlier tasks, leading to a significant decline in performance on those tasks. Common use cases include continual learning scenarios, where models are expected to adapt to new data without losing prior knowledge. Addressing catastrophic forgetting is crucial for developing robust AI systems that can learn incrementally without losing valuable information.