{"id":5035,"date":"2024-03-19T13:33:48","date_gmt":"2024-03-19T13:33:48","guid":{"rendered":"https:\/\/radicalbit.ai\/?post_type=glossary&#038;p=5035"},"modified":"2025-02-21T16:34:07","modified_gmt":"2025-02-21T16:34:07","slug":"regression","status":"publish","type":"glossary","link":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/","title":{"rendered":"Regression"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Regression is one of the most <strong>essential techniques<\/strong> in machine learning, allowing us to model relationships between variables and make predictions. It\u2019s widely used in various fields, from predicting house prices to estimating sales or forecasting weather. While there are many types of regression, each has its unique strengths and applications. This glossary entry will explore the most relevant ones, breaking them down in a clear and approachable way.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Linear Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Linear regression is the simplest and most commonly used type of regression. It works by <strong>finding the best-fitting straight line<\/strong> through a set of data points, allowing you to predict a continuous outcome based on one or more input variables. For example, you might use it to predict housing prices based on features like square footage or number of bedrooms.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If there\u2019s only one input variable, it\u2019s called simple linear regression. When there are multiple input variables, it becomes <strong>multiple linear regression.<\/strong> This technique works well when the relationship between variables is linear, meaning that as one variable increases or decreases, the other does so proportionally. However, it struggles with more complex patterns or non-linear relationships.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Logistic Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Despite its name, logistic regression is not used for predicting continuous values but rather for <strong>classification tasks.<\/strong> It\u2019s particularly useful when you need to determine whether something belongs to one of two categories\u2014such as whether an email is spam or not, or whether a patient has a certain disease.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead of predicting a number directly, logistic regression calculates the <strong>probability<\/strong> that an observation belongs to a specific category. For example, it might predict that an email has a 90% chance of being spam. These probabilities are then converted into binary outcomes (e.g., spam or not spam) using a threshold. The strength of logistic regression lies in its simplicity and effectiveness for binary classification problems.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Polynomial Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Sometimes, data doesn\u2019t follow a straight-line pattern. In such cases, polynomial regression can be used to capture more <strong>complex relationships<\/strong> by fitting a curve instead of a straight line. For instance, if you\u2019re modeling how temperature affects ice cream sales, the relationship might not be linear\u2014sales might increase sharply at first as temperatures rise but level off after reaching a certain point.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Polynomial regression achieves this by adding higher-degree terms (like squares or cubes) to the model. While this makes it more flexible than linear regression, it also comes with risks: if you add too many terms, the model may overfit the data, capturing noise rather than meaningful trends.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1354\" height=\"670\" data-id=\"8064\" src=\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1.png\" alt=\"\" class=\"wp-image-8064\" srcset=\"https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1.png 1354w, https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1-768x380.png 768w, https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1-1080x534.png 1080w, https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1-1280x633.png 1280w, https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1-980x485.png 980w, https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/03\/Screenshot-2025-02-21-at-17.23.53-1-480x238.png 480w\" sizes=\"(max-width: 1354px) 100vw, 1354px\" \/><\/figure>\n<\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Ridge Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Linear regression can run into problems when your dataset has many features that are highly correlated with each other (a situation known as multicollinearity). This can make predictions unstable and less reliable. Ridge regression addresses this issue by adding a <strong>regularization term<\/strong>\u2014a kind of penalty for overly large coefficients\u2014to the model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">By <strong>discouraging large coefficients,<\/strong> ridge regression reduces overfitting and improves the model\u2019s ability to generalize to new data. It\u2019s especially useful when working with datasets that have many features or when some features are highly correlated.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Lasso Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Lasso regression is similar to ridge regression in that it adds a <strong>penalty to prevent overfitting.<\/strong> However, while ridge regression penalizes large coefficients by shrinking them toward zero, lasso regression can <strong>shrink some coefficients all the way to zero.<\/strong> This means lasso not only reduces overfitting but also performs feature selection by effectively removing irrelevant variables from the model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This makes lasso particularly useful when dealing with high-dimensional datasets where many features may be redundant or irrelevant. By focusing only on the most important features, lasso creates simpler and more interpretable models.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Support Vector Regression (SVR)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Support Vector Regression (SVR) is an extension of support vector machines (SVMs), which are typically used for <strong>classification tasks. <\/strong>Instead of classifying data into categories, SVR predicts continuous values while maintaining certain margins of error around the true data points.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What makes SVR unique is its ability to handle both linear and non-linear relationships using kernel functions\u2014mathematical transformations that map data into higher dimensions where patterns become easier to identify. SVR is particularly effective for <strong>complex datasets and is robust against outlier<\/strong>s, making it a versatile choice for many real-world problems.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Decision Tree Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Decision tree regression provides a completely different approach compared to traditional methods like linear regression. Instead of fitting equations to data points, it splits the dataset into <strong>smaller subsets based on feature values<\/strong>. Each split creates branches in a tree-like structure until predictions are made at the leaves.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This method is <strong>highly intuitive and easy to visualize<\/strong>\u2014making it popular for understanding how decisions are made in complex systems. Additionally, decision trees can handle non-linear relationships well without requiring extensive preprocessing of data. However, they can overfit if not properly pruned or combined with ensemble methods like random forests.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Bayesian Linear Regression<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Bayesian linear regression takes traditional linear regression one step further by incorporating principles from Bayesian statistics. Instead of treating model parameters as fixed values, Bayesian methods assign them probability distributions based on prior knowledge and observed data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This approach provides more <strong>flexibility<\/strong> and allows for uncertainty in predictions\u2014something that\u2019s particularly useful when working with small datasets or when prior knowledge about the problem domain exists. Bayesian models also offer probabilistic interpretations of predictions, giving you confidence intervals rather than single-point estimates.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Choosing the Right Regression Method<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Selecting the appropriate regression method depends on understanding your ML system&#8217;s characteristics, requirements, and constraints. Consider the size and structure of your monitoring data, the complexity of relationships between ML metrics, and the importance of interpretability in your results. The computational resources available and the need for real-time predictions may also influence your choice.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>The nature of your prediction task plays a crucial role in method selection.<\/strong> Linear regression might suffice for simple relationships like dataset size vs. training time, while polynomial regression or SVR might better handle complex patterns in model performance analysis. If you have many potential ML metrics, Lasso or Elastic Net might prove more appropriate for identifying the most important factors.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Regression is one of the most essential techniques in machine learning, allowing us to model relationships between variables and make predictions. It\u2019s widely used in&hellip;<\/p>\n","protected":false},"author":1,"featured_media":4814,"menu_order":0,"template":"","meta":{"footnotes":""},"class_list":["post-5035","glossary","type-glossary","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Regression | Radicalbit<\/title>\n<meta name=\"description\" content=\"Regression understands the relationship between independent variables and dependent numerical variable aiming at predicting outcomes\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/\" \/>\n<meta property=\"og:locale\" content=\"it_IT\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Regression | Radicalbit\" \/>\n<meta property=\"og:description\" content=\"Regression understands the relationship between independent variables and dependent numerical variable aiming at predicting outcomes\" \/>\n<meta property=\"og:url\" content=\"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/\" \/>\n<meta property=\"og:site_name\" content=\"Radicalbit\" \/>\n<meta property=\"article:modified_time\" content=\"2025-02-21T16:34:07+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/02\/Simpler-faster-better-MLOps.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Tempo di lettura stimato\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minuti\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/\",\"url\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/\",\"name\":\"Regression | Radicalbit\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/radicalbit.ai\\\/wp-content\\\/uploads\\\/2024\\\/02\\\/Simpler-faster-better-MLOps.jpg\",\"datePublished\":\"2024-03-19T13:33:48+00:00\",\"dateModified\":\"2025-02-21T16:34:07+00:00\",\"description\":\"Regression understands the relationship between independent variables and dependent numerical variable aiming at predicting outcomes\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/#breadcrumb\"},\"inLanguage\":\"it-IT\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"it-IT\",\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/#primaryimage\",\"url\":\"https:\\\/\\\/radicalbit.ai\\\/wp-content\\\/uploads\\\/2024\\\/02\\\/Simpler-faster-better-MLOps.jpg\",\"contentUrl\":\"https:\\\/\\\/radicalbit.ai\\\/wp-content\\\/uploads\\\/2024\\\/02\\\/Simpler-faster-better-MLOps.jpg\",\"width\":1200,\"height\":628,\"caption\":\"Radicalbit MLOps & AI Observability Platform\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/resources\\\/glossary\\\/regression\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Regression\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/#website\",\"url\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/\",\"name\":\"Radicalbit\",\"description\":\"Simpler, faster, better MLOps\",\"publisher\":{\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"it-IT\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/#organization\",\"name\":\"Radicalbit\",\"url\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"it-IT\",\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/radicalbit.ai\\\/wp-content\\\/uploads\\\/2023\\\/11\\\/radicalbit__RGB__logo-horizontal-positive-e1701861883280.webp\",\"contentUrl\":\"https:\\\/\\\/radicalbit.ai\\\/wp-content\\\/uploads\\\/2023\\\/11\\\/radicalbit__RGB__logo-horizontal-positive-e1701861883280.webp\",\"width\":1570,\"height\":405,\"caption\":\"Radicalbit\"},\"image\":{\"@id\":\"https:\\\/\\\/radicalbit.ai\\\/it\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/company\\\/6639929\\\/\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Regression | Radicalbit","description":"Regression understands the relationship between independent variables and dependent numerical variable aiming at predicting outcomes","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/","og_locale":"it_IT","og_type":"article","og_title":"Regression | Radicalbit","og_description":"Regression understands the relationship between independent variables and dependent numerical variable aiming at predicting outcomes","og_url":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/","og_site_name":"Radicalbit","article_modified_time":"2025-02-21T16:34:07+00:00","og_image":[{"width":1200,"height":628,"url":"http:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/02\/Simpler-faster-better-MLOps.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Tempo di lettura stimato":"5 minuti"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/","url":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/","name":"Regression | Radicalbit","isPartOf":{"@id":"https:\/\/radicalbit.ai\/it\/#website"},"primaryImageOfPage":{"@id":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/#primaryimage"},"image":{"@id":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/#primaryimage"},"thumbnailUrl":"https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/02\/Simpler-faster-better-MLOps.jpg","datePublished":"2024-03-19T13:33:48+00:00","dateModified":"2025-02-21T16:34:07+00:00","description":"Regression understands the relationship between independent variables and dependent numerical variable aiming at predicting outcomes","breadcrumb":{"@id":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/#breadcrumb"},"inLanguage":"it-IT","potentialAction":[{"@type":"ReadAction","target":["https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/"]}]},{"@type":"ImageObject","inLanguage":"it-IT","@id":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/#primaryimage","url":"https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/02\/Simpler-faster-better-MLOps.jpg","contentUrl":"https:\/\/radicalbit.ai\/wp-content\/uploads\/2024\/02\/Simpler-faster-better-MLOps.jpg","width":1200,"height":628,"caption":"Radicalbit MLOps & AI Observability Platform"},{"@type":"BreadcrumbList","@id":"https:\/\/radicalbit.ai\/it\/resources\/glossary\/regression\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/radicalbit.ai\/it\/"},{"@type":"ListItem","position":2,"name":"Regression"}]},{"@type":"WebSite","@id":"https:\/\/radicalbit.ai\/it\/#website","url":"https:\/\/radicalbit.ai\/it\/","name":"Radicalbit","description":"Simpler, faster, better MLOps","publisher":{"@id":"https:\/\/radicalbit.ai\/it\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/radicalbit.ai\/it\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"it-IT"},{"@type":"Organization","@id":"https:\/\/radicalbit.ai\/it\/#organization","name":"Radicalbit","url":"https:\/\/radicalbit.ai\/it\/","logo":{"@type":"ImageObject","inLanguage":"it-IT","@id":"https:\/\/radicalbit.ai\/it\/#\/schema\/logo\/image\/","url":"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/11\/radicalbit__RGB__logo-horizontal-positive-e1701861883280.webp","contentUrl":"https:\/\/radicalbit.ai\/wp-content\/uploads\/2023\/11\/radicalbit__RGB__logo-horizontal-positive-e1701861883280.webp","width":1570,"height":405,"caption":"Radicalbit"},"image":{"@id":"https:\/\/radicalbit.ai\/it\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/6639929\/"]}]}},"_links":{"self":[{"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/glossary\/5035","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/glossary"}],"about":[{"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/types\/glossary"}],"author":[{"embeddable":true,"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":8,"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/glossary\/5035\/revisions"}],"predecessor-version":[{"id":8066,"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/glossary\/5035\/revisions\/8066"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/media\/4814"}],"wp:attachment":[{"href":"https:\/\/radicalbit.ai\/it\/wp-json\/wp\/v2\/media?parent=5035"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}