"Representation learning for speech and handwriting"
Faculty of Mathematics and Computer Science at the University of Wrocław
Head of AI at NavAlgo
Jan Chorowski is an Associate Professor at Faculty of Mathematics and Computer Science at the University of Wrocław and Head of AI at NavAlgo. He received his M.Sc. degree in electrical engineering from the Wrocław University of Technology, Poland and EE PhD from the University of Louisville, Kentucky in 2012. He has worked with several research teams, including Google Brain, Microsoft Research and Yoshua Bengio’s Lab at the University of Montreal. He has led a research topic during the JSALT 2019 workshop. His research interests are applications of neural networks to problems which are intuitive and easy for humans and difficult for machines, such as speech and natural language processing.
Learning representations of data in an unsupervised way is still an open problem of machine learning. We consider representations of speech and handwriting learned using autoencoders equipped with autoregressive decoders such as WeveNets or PixelCNNs. In those autoencoders, the encoder only needs to provide the little information needed to supplement all that can be inferred by the autoregressive decoder. This allows learning a representation able to capture high level semantic content from the signal, e.g. phoneme or character identities, while being invariant to confounding low level details in the signal such as the underlying pitch contour or background noise.
Presentation will include the design choices for the autoencoder, such as the bottleneck kind its hyperparameters impact the induced latent representation. Applications will be demonstrated to unsupervised acoustic unit discovery on the ZeroSpeech task. Discussion will cover how knowledge about the average unit duration can be enforced during training, as well as during inference on new data.
"Deep Learning, Deep Knowledge Representation and Knowledge Transfer with Brain-Inspired Neural Network Architectures"
Fellow IEEE, Fellow RSNZ, DV Fellow RAE UK
Director, Knowledge Engineering and Discovery Research Institute,
Auckland University of Technology, Auckland, New Zealand, email@example.com,
Advisory/Visiting Professor Shanghai Jiao Tong University, Robert Gordon University UK
Professor Nikola Kasabov is Fellow of IEEE, Fellow of the Royal Society of New Zealand, Fellow of the INNS College of Fellows, DVF of the Royal Academy of Engineering UK and the Scottish Computer Association. He is the Founding Director of the Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland and Professor at the School of Engineering, Computing and Mathematical Sciences at Auckland University of Technology. Kasabov is the 2019 President of the Asia Pacific Neural Network Society (APNNS) and Past President of the International Neural Network Society (INNS). He is member of several technical committees of IEEE Computational Intelligence Society and Distinguished Lecturer of IEEE (2012-2014). He is Editor of Springer Handbook of Bio-Neuroinformatics, Springer Series of Bio-and Neurosystems and Springer journal Evolving Systems. He is Associate Editor of several journals, including Neural Networks, IEEE TrNN, Tr CDS, Information Sciences, Applied Soft Computing. Kasabov holds MSc and PhD from TU Sofia, Bulgaria. His main research interests are in the areas of neural networks, intelligent information systems, soft computing, bioinformatics, neuroinformatics. He has published more than 620 publications. He has extensive academic experience at various academic and research organisations in Europe and Asia, including: TU Sofia Bulgaria; University of Essex UK; University of Otago, NZ; Advisory Professor at Shanghai Jiao Tong University, Visiting Professor at ETH/University of Zurich and Robert Gordon University UK. Prof. Kasabov has received a number of awards, among them: Doctor Honoris Causa from Obuda University, Budapest; INNS Ada Lovelace Meritorious Service Award; NN Best Paper Award for 2016; APNNA ‘Outstanding Achievements Award’; INNS Gabor Award for ‘Outstanding contributions to engineering applications of neural networks’; EU Marie Curie Fellowship; Bayer Science Innovation Award; APNNA Excellent Service Award; RSNZ Science and Technology Medal; 2015 AUT Medal; Honorable Member of the Bulgarian and the Greek Societies for Computer Science. More information of Prof. Kasabov can be found on the KEDRI web site: http://www.kedri.aut.ac.nz
The talk argues and demonstrates that the third generation of artificial neural networks, the spiking neural networks (SNN), can be used to design brain-inspired architectures that are not only capable of deep learning of temporal or spatio-temporal data, but also enabling the extraction of deep knowledge representation from the learned data. Similarly to how the brain learns time-space data, these SNN models do not need to be restricted in number of layers, neurons in each layer, etc. as it is the case with the traditional deep neural network architectures. When the SNN model is designed to follow a brain template, knowledge transfer between humans and machines in both directions becomes possible through the creation of brain-inspired BCI. The presented approach is illustrated on an exemplar SNN architecture NeuCube (free software and open source available from www.kedri.aut.ac.nz/neucube) and case studies of brain and environmental data modelling and knowledge representation using incremental and transfer learning algorithms These include predictive modelling of EEG and fMRI data measuring cognitive processes and response to treatment, AD prediction, BCI, human-human and human-VR communication, hyper-scanning and other. More details can be found in the recent book: Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence, Springer,2019, https://www.springer.com/gp/book/9783662577134.
"AI-based business applications for transforming data into decisions"
University of Adelaide, Australia
Zbigniew Michalewicz received the Master of Science degree from the Technical University of Warsaw, Warsaw, Poland, in 1974; the Ph.D. degree from the Institute of Computer Science, Polish Academy of Sciences, Warsaw, in 1981, and the D.Sc. degree in Computer Science from the Polish Academy of Science in 1997. He is currently Emeritus Professor of Computer Science at the University of Adelaide, Australia. He is also a Professor with the Institute of Computer Science, Polish Academy of Sciences, the Polish-Japanese Institute of Information Technology, Warsaw, and the State Key Laboratory of Software Engineering, Wuhan University, Wuhan, China. He is also associated with the Structural Complexity Laboratory, Seoul National University, South Korea. Zbigniew Michalewicz is the Chief Scientific Officer at Complexica (www.complexica.com), a leading provider of software applications that harness the power of Artificial Intelligence and big data to improve the effectiveness of sales & marketing activities. For many years his research interests were in the field of evolutionary computation. He published several books, including a monograph Genetic Algorithms + Data Structures = Evolution Programs (3 editions, a few translations, over 20,000 citations, source: Google Scholar), and over 300 technical papers in journals and conference proceedings that are cited widely (50,000 citations, source: Google Scholar). Other books include Adaptive Business Intelligence and How to Solve It: Modern Heuristics (both published by Springer, Berlin, 2006 and 2004, respectively), Puzzle-based Learning (Hybrid Publishers, Melbourne, 2008), Winning Credibility: A Guide for Building a Business from Rags to Riches (Hybrid Publishers, Melbourne, 2007), where he described his business experiences over the last years.
Zbigniew Michalewicz was one of the editors-in-chief of the Handbook of Evolutionary Computation and the general chairman of the First IEEE International Conference on Evolutionary Computation held in Orlando, June 1994. In December 2013 Zbigniew was awarded (by the President of Poland, Mr. Bronislaw Komorowski) the Order of the Rebirth of Polish Polonia Restituta – the second highest Polish state decoration civilian for outstanding achievements in the field of education, science, sports, culture, arts, economy, national defence, social activities, the civil service and the development of good relations with other countries.
The talk would cover a few AI-based business applications for transforming data into decisions, based on work done for three companies (NuTech Solutions, SolveIT Software, and Complexica) over the last 20 years. A few general concepts (Adaptive Business Intelligence, Travelling Thief Problem, Larry – the Digital Analyst) would be discussed and illustrated by a few examples. The final part of the talk would present Complexica’s approach for increasing revenue, margin, and customer engagement through automated analysis.
Nikhil R. Pal
Title: Artificial Intelligence: "Winters", "Booms", and what we might be missing!
Professor, Electronics and communication Sciences unit
Head, Center for Artificial intelligence and Machine Learning
Indian Statistical Institute, Calcutta, India
Nikhil R. Pal is a Professor in the Electronics and Communication Sciences Unit and is the Head of the Center for Artificial Intelligence and Machine Learning of the Indian Statistical Institute. His current research interest includes brain science, computational intelligence, machine learning and data mining.
He was the Editor-in-Chief of the IEEE Transactions on Fuzzy Systems for the period January 2005-December 2010. He has served/been serving on the editorial /advisory board/ steering committee of several journals including the International Journal of Approximate Reasoning, Applied Soft Computing, International Journal of Neural Systems, Fuzzy Sets and Systems, IEEE Transactions on Fuzzy Systems and the IEEE Transactions on Cybernetics.
He is a recipient of the 2015 IEEE Computational Intelligence Society (CIS) Fuzzy Systems Pioneer Award, He has given many plenary/keynote speeches in different premier international conferences in the area of computational intelligence. He has served as the General Chair, Program Chair, and co-Program chair of several conferences. He was a Distinguished Lecturer of the IEEE CIS (2010-2012, 2016-2018.) and was a member of the Administrative Committee of the IEEE CIS (2010-2012). He has served as the Vice-President for Publications of the IEEE CIS (2013-2016). He has served as the President of the IEEE CIS (2018-2019).
He is a Fellow of the National Academy of Sciences, India, Indian National Academy of Engineering, Indian National Science Academy, International Fuzzy Systems Association (IFSA), The World Academy of Sciences, and a Fellow of the IEEE, USA. ( www.isical.ac.in/~nikhil)
In this talk I shall briefly go through the history of evolution of AI – how AI has sailed through the “ups and downs” and has come to the present state. In the recent past, we have witnessed numerous fantastic success stories of AI systems, often beating human performance, and this has caused our expectation from AI to skyrocket. In many cases, neural networks, in particular deep neural networks, are the main pillars of such systems. But are these systems comprehensible and/or biologically plausible? In most cases, they are not! It seems, implicitly we have started believing in philosophies like "bigger the better" (bigger data sets or massive architecture with millions of free parameters) and "data say all". Such approaches have been proved to be useful but raise some concerns too! In my view, comprehensibility of a system depends, at least, on the following: simplicity, transparency, explainability, trustworthiness, and in some cases the biological plausibility of such systems. Ideally, we should strive for realizing all these attributes in any AI system, but this is very difficult. I shall discuss some of these important issues where we need to pay more attention and then illustrate how one or two of these issues can be addressed (to some extent) borrowing knowledge from biological systems- some of our preliminary attempts.
"From Data to Information Granules: Quantitative and Qualitative Facets of AI"
Department of Electrical & Computer Engineering, University of Alberta, Canada
Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland
Granular Computing is about representing knowledge by means of information granules, constructing information granules, their processing, and realizing communication and interpretation carried out in the framework of information granules.
Information granules are abstract constructs that bring together individual entities because of their closeness, similarity, or resemblance. The level of abstraction makes a description of the problem manageable and problem solving strategies feasible and efficient.
We offer a rationale behind emergence of information granules, offer examples and present a variety of frameworks (sets, intervals, fuzzy sets, probabilities, rough sets, random sets, intuitionistic sets…) using which they are formally represented.
Main motivating factors are advocated. General ways of designing and evaluating information granules are discussed. A role of a variety of clustering techniques treated as a prerequisite for the formation of information granules is demonstrated. The evaluation of the quality of information granules is case ion the granulation-degranulation scheme.
We deliver a comprehensive approach to the development of information granules by means of the principle of justifiable granularity; here various construction scenarios are discussed. In the sequel, we look at the generative and discriminative aspects of information granules supporting their further usage in the formation of granular models. A symbolic manifestation of information granules is put forward and analyzed from the perspective of semantically sound descriptors of data and relationships among data. The principle provides a way to build an information granule such that it is legitimate from the perspective of coverage (experimental legitimacy of the granule) and its semantics (meaning). Along with the generic construct, discussed are various augmentations of the principle. We carefully look at the generative and discriminative aspects of information granules supporting their further usage in the formation of granular artifacts. The considerations are carried out following a general knowledge representation scheme:
data -› numeric prototypes -› information granules -› symbols
Furthermore, a symbolic characterization of information granules is put forward and analyzed from the perspective of semantically sound descriptors of data. Their linguistic summarization is offered as well. The diversity of information granules is also captured by more advanced constructs of information granules of higher type and higher order.
Some selected topics of data analytics in which information granularity is visible such as (i) imputation, (ii) time series prediction, (ii) data stream analysis, (iii) imputation, (iv) association analysis and associative memories, and (v) transfer learning are formulated and discussed.
The tutorial is made self-contained; all required prerequisite material will be made an initial part of the presentation.