We are proud to announce the 1.3.0 release of Radicalbit AI Monitoring, our open source solution that helps data teams measure the effectiveness and reliability of machine learning and large language models.
This release includes a series of new, exciting features: let’s explore them in detail!
Supercharge Debugging with Agent Tracing
The black-box nature of LLM-powered applications is one of the major limitations to their implementation. To ensure compliance and accountability, organizations need to understand every step leading to the final output.
Radicalbit AI Monitoring’s new Agent Tracing feature empowers data teams to track each and every individual request, prompt and tool invoked by the agent. Thorough trace measurement is paramount both in experimental and production phase, for debugging and analyzing agent behavior.
The “Traces” dashboard gives an exhaustive overview of, well, traces – the journey of a single request to the agent – and spans – the single operations that make up a trace. Detailed information such as hierarchy, duration and metadata attributes offers much-needed insight into the behavior of the LLM-powered application.
New Algorithms for Drift Detection Flexibility
The latest Radicalbit AI Monitoring update adds a whole series of new drift detection algorithms for both categorical and numerical data.
Until now, you could only rely on good ol’ Chi-Square Test, 2-Samples-KS Test and PSI. With 1.3.0, you add to your toolbox Jensen Shannon Distance, Kullback-Liebler Divergence, Hellinger Distance for categoricals, and Wasserstein Distance, Jensen Shannon Distance, Hellinger Distance, and Kullback-Liebler Divergence for numericals.
It goes without saying that having multiple methods greatly improves reliability, sensitivity, and interpretability in different situations. Data teams now have new ways to detect variations in data and model behavior, helping preempt performance degradation and non-standard behaviors.
Enhance LLM Monitoring with Perplexity & Probability
When you ask LLMs such as GPT or Gemini to generate text, you are to be sure that the model is confident in its utterances. There is no room for uncertainty when your stakeholders rely on text created by AI-powered chatbots.
To address this, Radicalbit AI Monitoring’s 1.3.0 adds a whole new dimension to LLM monitoring. The model quality dashboard features overall metrics such as number of sentences, perplexity, and probability. It also displays detailed sequences with metric-based token coloring that provide immediate feedback on the performance and characteristics of generation.
If you want to learn more about Radicalbit AI Monitoring, deploy it for your projects and become part of our open-source journey, visit our GitHub repository!
