![\"seldon](\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/12\/simple-seldon.webp\" "\\"seldon\\"")
<\/figure>\n\n\n\nWe wanted to pick the best technology, so we picked Seldon-core. This great open-source project can serve any kind of ML assets (even custom ones); moreover, it offers out-of-the-box support to modern cloud-native technologies like service meshes and even accepts Kafka as a communication bus. We integrated Seldon in our Kubernetes orchestration; this backs up most of our MLOps features, like dynamic serving, continuous training, and streaming machine learning. But how do we make interact streaming workloads and ML models? Radicalbit MLOps platform offers a codeless pipeline editor, where engineers and scientists can work together to seamlessly manage and process data in motion.<\/p>\n\n\n\n Once the pipeline is built and tested (we provide a novel codeless debugging<\/strong> feature), users can deploy the topology on top of their favourite streaming engine. We provide out-of-the-box support for Apache Flink, Spark Streaming, and the Kafka streams library. Don\u2019t worry! In the end, you will get a new horizontally-scaled deployment on Kubernetes. We come up then with two main deployment families<\/strong>:<\/p>\n\n\n\n 1) Your streaming workload topology<\/strong><\/p>\n\n\n\n 2) A set of Seldon<\/em> orchestrated ML trained models<\/strong><\/p>\n\n\n\n While the deployment runs, the topology asks the models for predictions and the models respond with high-throughput and low-latency. The communication channel<\/strong> is Kafka, of course. Through this solution, we can exchange inference asynchronously using Kafka messages conveyed by Kafka topics; therefore, we can leverage fault-tolerant and highly scalable features. Consequently, it is possible to scale infinitely, the only limit is Kafka\u2019s infrastructure availability.<\/p>\n\n\n\n For Radicalbit, \u201cContinuous Training\u201d means providing customers with a set of DevOps tools, in order to help them in establishing when it is the right time to retrain a model in production. We follow one key metric only: drift. For us, drift is not only an upstream change in the data structure; but a change in data behaviour and its domain evolution.<\/p>\n\n\n\n We identify two kinds of drifts in data. The first one is the so-called \u201cData Drift Concept\u201d; at this stage, we can monitor input data behaviour. As you can see in the figure below, we track the value of the feature x\u2018s behaviour we are monitoring, and we do not consider the inference values at all. However, when the input data distribution changes in unprecedented ways, your model might not be good anymore. Therefore, you should perform a new training session. ADWIN (Adaptive Windowing) is a well-known method to spot these drifts by monitoring data streams and \u2013 in the figure below \u2013 the vertical red lines represent the ADWIN activation.<\/p>\n\n\n\n However, this approach might not properly work in some cases. For instance, for stock market data, where the trend concept is as meaningful as the values the feature assumes. Where \u201chow the values change over time\u201d represents information as well data drift is not a perfect fit anymore. So we need to move to a new monitoring strategy, where the values of the inference are also important. Thus, we built a detection model for the second kind of drift we identified, the so-called \u201cConcept Drift\u201c.<\/p>\n\n\n\n By monitoring the prediction trend, we can understand how well we are predicting. However, in order to understand, we need feedback, thus a piece of information stating the true class for a determined and already emitted prediction. By this hint, the platform is then able to compute the error rate. Methods like drift detection model track down sudden changes in the error rate value (as in the figure below as vertical red lines).<\/p>\n\n\n\n If detected, the platform alerts the clients that it is time to retrain their ML model with fresh data. Radicalbit MLOps platform offers a tiered set of notifications or triggers useful to activate actions (e.g. execute remote continuous integration pipelines aimed to re-train a model).<\/p>\n\n\n\n Like I eagerly anticipated in this article preface, Radicalbit aims to build the first production-ready platform for streaming machine learning models. This is a completely different approach for many reasons, but the obvious one is that it changes the input data that our models\u2019 crunch (therefore, goodbye traditional datasets!) while they learn. We do not have historical and finished data, but an unfinished and continuous flow of information that changes over time. Therefore, the chances of making valuable a-priori assumptions on the input data are very limited. The objective is to make our models dynamic in behaviour so they can adapt to the input data changes over time.<\/p>\n\n\n\n On the other hand, concerning Supervised Learning, we are driven by the same requirement of drifts, which is feedback. In this situation, also the paradigm is changing; indeed, for online machine learning, the prediction is firstly computed, then delivered to the downstream, and finally, in reaction to incoming feedback, the models update the learning functions based on the measured loss.<\/p>\n\n\n\n The main difference with traditional batch learning tasks is that online models predict and train the algorithm at the same time. In our latest R&D project, we built one fit for an online neural network algorithm based on a hedge algorithm. The main feature of hedging lies in the expert-advice approach: each network layer is able to provide an output prediction, shallower layers will be more valuable in earlier stages of the algorithm lifetime; then, the more events the model crunches, the more credibility will be assigned to deeper layers output predictions.<\/p>\n\n\n\n Radicalbit also proposes a first distributed implementation for it, as shown in the picture below.<\/p>\n\n\n\n The main advantage of using streaming machine learning algorithms in our platform is that you can dramatically reduce the effort of periodically re-train models because they fix themselves over time. Additionally, by building the right serving layer for streaming models, we are about to prove that for many contexts streaming algorithms can outperform traditional batch learning applications.<\/p>\n\n\n\n Stay tuned and follow us to learn more!<\/p>\n","protected":false},"excerpt":{"rendered":" Here are the challenges for a MLOps platform. We explain how serve together thousands of ML models, regardless of the underlying technology<\/p>\n","protected":false},"author":1,"featured_media":3377,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[28,30],"tags":[21,109,72,68,86],"class_list":["post-2713","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-by-radicalbit","tag-ai","tag-data","tag-machine-learning","tag-ml","tag-mlops"],"yoast_head":"\n![\"sentiment](\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/12\/pipeline-sentiment-demo.webp\" "\\"sentiment")
<\/figure>\n\n\n\n![\"kafka](\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/12\/kafka-topic-diagram.webp\" "\\"kafka")
<\/figure>\n\n\n\nLet\u2019s talk about Continuous Training<\/h2>\n\n\n\n
![\"continuous-training\"](\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/12\/continuous-training.webp\" "\\"continuous-training\\"")
<\/figure>\n\n\n\n![\"algorithm\"](\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/12\/DDM-algorithm-test.webp\" "\\"algorithm\\"")
<\/figure>\n\n\n\nLet\u2019s talk about Online Learning
How do we train machine learning models with streaming data?<\/h2>\n\n\n\n![\"data](\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/12\/data-sources.webp\" "\\"data")
<\/figure>\n\n\n\n