Here is the paper:
Banking Information Systems continuously generate large quantities of data as inter-connected streams (transactions, events logs, time series, metrics, graphs, process, etc.). Such data streams need to be processed online to deal with critical business applications such as real-time fraud detection, network security attack prevention or predictive maintenance on information system infrastructure. Many algorithms have been proposed for data stream learning, however, most of them do not deal with the important challenges and constraints imposed by real-world applications. In particular, when we need to train models incrementally from heterogeneous data mining and deployment them within complex big data architecture. Based on banking applications and lessons learned in production environments of BNP Paribas – a major international banking group and leader in the Eurozone – we identified the most important current challenges for mining IT data streams. Our goal is to highlight the key challenges faced by data scientists and data engineers within complex industry settings for building or deploying models for real word streaming applications. We provide future research directions on Stream Learning that will accelerate the adoption of online learning models for solving real-word problems. Therefore bridging the gap between research and industry communities. Finally, we provide some recommendations to tackle some of these challenges.
]]>
Artificial Intelligence Researchers Association
Artificial Intelligence (AI) is profoundly changing how we live and work. The cumulative impact of AI
is likely to be comparable to other transformative technologies such as electricity or the internet.
As a result, it is imperative that we take a strategic approach to realising the potential benefits offered
by AI and to protecting people against the potential risks.
In this whitepaper, we discuss current AI capabilities in Aotearoa New Zealand and offer recommendations for establishing Aotearoa New Zealand as a research centre of excellence and trust in AI. Our discussion is informed and guided by the framework for developing a national AI strategy set out by the World Economic Forum.
It is important to invest in AI imbued with characteristics and values important for Aotearoa New Zealand such as sustainability, fairness, equality, data sovereignty, Te Tiriti obligations, multiculturalism, intergenerational thinking, people and whānau first, and holistic thinking. Otherwise, we risk being relegated to users of overseas technologies developed by countries with different values.
Our vision is that by 2030, Aotearoa New Zealand will have a community of cutting-edge companies producing and exporting AI technologies, supported by a strong network of researchers involved in high level fundamental and applied research
]]>As a quick example, we’ll train a logistic regression to classify the website phishing dataset. Here’s a look at the first observation in the dataset.
>>> from pprint import pprint
>>> from river import datasets
>>> dataset = datasets.Phishing()
>>> for x, y in dataset:
... pprint(x)
... print(y)
... break
{'age_of_domain': 1,
'anchor_from_other_domain': 0.0,
'empty_server_form_handler': 0.0,
'https': 0.0,
'ip_in_url': 1,
'is_popular': 0.5,
'long_url': 1.0,
'popup_window': 0.0,
'request_from_other_domain': 0.0}
True
Now let’s run the model on the dataset in a streaming fashion. We sequentially interleave predictions and model updates. Meanwhile, we update a performance metric to see how well the model is doing.
>>> from river import compose >>> from river import linear_model >>> from river import metrics >>> from river import preprocessing >>> model = compose.Pipeline( ... preprocessing.StandardScaler(), ... linear_model.LogisticRegression() ... ) >>> metric = metrics.Accuracy() >>> for x, y in dataset: ... y_pred = model.predict_one(x) # make a prediction ... metric = metric.update(y, y_pred) # update the metric ... model = model.learn_one(x, y) # make the model learn >>> metric Accuracy: 89.20%]]>
In this work, we focus on elucidating the connections among the current state-of-the-art on related fields; and clarifying open challenges in both academia and industry. We treat with special care topics that were not thoroughly investigated in past position and survey papers.
This work aims to evoke discussion and elucidate the current research opportunities, high-lighting the relationship of different subareas and suggesting courses of action when possible.
Heitor Murilo Gomes, Jesse Read, Albert Bifet, Jean Paul Barddal, João Gama: Machine learning for streaming data: state of the art, challenges, and opportunities. SIGKDD Explorations 21(2): 6-22 (2019)
]]>Heitor Murilo Gomes, Jesse Read, Albert Bifet: Streaming Random Patches for Evolving Data Stream Classification. ICDM 2019: 240-249
]]>In this paper, we present FAST AND SLOW LEARNING (FSL), a novel unified framework that sheds light on the symbiosis between batch and stream learning. FSL works by employing Fast (stream) and Slow (batch) Learners, emulating the mechanisms used by humans to make decisions.
Jacob Montiel, Albert Bifet, Viktor Losing, Jesse Read, Talel Abdessalem: Learning Fast and Slow: A Unified Batch/Stream Framework. BigData 2018: 1065-1072
Jacob Montiel PhD Thesis: “Fast and slow machine learning”
]]>
The book first offers a brief introduction to the topic, covering big data mining, basic methodologies for mining data streams, and a simple example of MOA. More detailed discussions follow, with chapters on sketching techniques, change, classification, ensemble methods, regression, clustering, and frequent pattern mining. Most of these chapters include exercises, an MOA-based lab session, or both. Finally, the book discusses the MOA software, covering the MOA graphical user interface, the command line, use of its API, and the development of new methods within MOA. The book will be an essential reference for readers who want to use data stream mining as a tool, researchers in innovation or data stream mining, and programmers who want to create new algorithms for MOA.
“Nobody can advise you and help you. Nobody. There is only one way—Go into yourself.”
I can recommend to young PhD students, only two things:
I can suggest one book to be more effective in research and in life:
The Seven Habits of Highly Effective People by Stephen Covey
]]>
In an experiment with an adaptive HT with the electricity and covertype datasets, the best performance was due to the No-Change Detector. This detector outputs change every 60 instances; it is a no-change detector in the sense that it is not detecting change in the stream. Surprisingly, the classifiers using this no-change detector are getting better results than using the standard change detectors.
Albert Bifet: Classifier Concept Drift Detection and the Illusion of Progress. ICAISC (2) 2017: 715-725