Evolving intelligent system

EISs can be implemented, for example, using neural networks or fuzzy rule-based models. The first neural networks which consider an evolving structure were published in.^{[14][15][16][17][18]} These were later expanded by N. Kasabov^[5] and P. Angelov^{[3][4][6][19]} for the neuro-fuzzy models. P. Angelov^[3][4][6][7] introduced the evolving fuzzy rule-based systems (EFSs) as the first mathematical self-learning model that can dynamically evolve its internal structure and is human interpretable and coined the phrase EFS. Contemporarily, the offline incremental approach for learning an EIS, namely, EFuNN, was proposed by N. Kasabov.^[20][21] P. Angelov, D. Filev, N. Kasabov and O. Cordon organised the first IEEE Symposium on EFSs in 2006 (the proceedings of the conference can be found in^[22]). EFSs include a formal (and mathematically sound) learning mechanism to extract it from streaming data. One of the earliest and the most widely cited comprehensive survey on EFSs was done in 2008.^[23] Later comprehensive surveys on EFS methods with real applications were done in 2011^[24] and 2016^[25] by E. Lughofer.

Other works that contributed further to this area in the following years expanded it to evolving participatory learning,^[26] evolving grammar,^[27] evolving decision trees,^[28] evolving human behaviour modelling,^[29] self-calibrating (evolving) sensors (eSensors),^[30] evolving fuzzy rule-based classifiers,^{[31][32][33][34][35]} evolving fuzzy controllers,^[36][37] autonomous fault detectors.^[38] More recently, the stability of the evolving fuzzy rule-based systems that consist of the structure learning and the fuzzily weighted recursive least square^[7] parameter update method has been proven by Rong.^[39] Generalized EFS, which allow rules to be arbitrarily rotated in the feature space and thus to improve their data representability, have been proposed in^[40] with significant extensions in^[41] towards 'smartness' of the rule bases (thus, termed as "Generalized Smart EFS"), allowing more interpretability and reducing curse of dimensionality. The generalized rule structure was also successfully used in the context of evolving neuro-fuzzy systems. Several facets and challenges for achieving more transparent and understandable rule bases in EFS have been discussed by E. Lughofer in.^[42]

EISs form the theoretical and methodological basis for the Autonomous Learning Machines (ALMA)^[43] and autonomous multi-model systems (ALMMo)^[44] as well as of the Autonomous Learning Systems.^[10] Evolving Fuzzy Rule-based classifiers,^{[31][32][33][34][35]} in particular, is a very powerful new concept that offers much more than simply incremental or online classifiers – it can cope with new classes being added or existing classes being merged. This is much more than just adapting to new data samples being added or classification surfaces being evolved. Fuzzy rule-based classifiers^[34] are the methodological basis of a new approach to deep learning that was until now considered as a form of multi-layered neural networks.^[45] Deep Learning offers high precision levels surpassing the level of human ability and grabbed the imagination of the researchers, industry and the wider public. However, it has a number of intrinsic constraints and limitations. These include:

1. The "black box", opaque internal structure which has millions of parameters and involves ad hoc decisions on the number of layers and algorithm parameters.
2. The requirement for a huge amount of training data samples, computational resources (usually requiring GPUs and/or HPC) and time (usually requiring many hours of training).
3. Iterative search.
4. Requires retraining for new situations (is not evolving).
5. Does not have proven convergence and stability.

Most, if not all, of the above limitations can be avoided with the use of the Deep (Fuzzy) Rule-based Classifiers,^[46][47] which were recently introduced based on ALMMo, while achieving similar or even better performance. The resulting prototype-based IF...THEN...models are fully interpretable and dynamically evolving (they can adapt quickly and automatically to new data patterns or even new classes). They are non-parametric and, therefore, their training is non-iterative and fast (it can take few milliseconds per data sample/image on a normal laptop which contrasts with the multiple hours the current deep learning methods require for training even when they use GPUs and HPC). Moreover, they can be trained incrementally, online, or in real-time. Another aspect of Evolving Fuzzy Rule-based classifiers has been proposed in,^[48] which, in case of multi-class classification problems, achieves the reduction of class imbalance by cascadability into class sub-spaces and an increased flexibility and performance for adding new classes on the fly from streaming samples.^[49]