Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by James McCaffrey
News & analysis on Visual Studio, .NET, AI and beyond
The Data Science Lab Data smoothing is a classical statistics technique that removes noise from a dataset so that overall patterns and trends are visible. There are many algorithms for smoothing. Wikipedia lists over a dozen smoothing algorithms, including moving average, Chebyshev filter, exponential smoothing, kernel smoothing, Kolmogorov--Zurbenko filter and others. Each smoothing algorithm has several variations.
News & analysis on Visual Studio, .NET, AI and beyond
The Data Science Lab A hands-on guide to building and training a two-hidden-layer neural network regressor in C#, including data preparation, SGD, evaluation, and using the trained model. The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank might want to predict the maximum safe loan amount for a customer based on customer age, current savings balance, debt, income, and so on.
News & analysis on Visual Studio, .NET, AI and beyond
The Data Science Lab The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank might want to predict the maximum loan amount for a customer based on age, annual salary, bank account balance, and so on. There are approximately a dozen common regression techniques. The most primitive technique is called linear regression, or sometimes multiple linear regression, where the "multiple" indicates two or more predictor variables.
Computing Dataset Variance Inflation Factor (VIF) Using C#
The Data Science Lab The variance inflation factor (VIF) of a dataset column is a value that measures the mathematical correlation of the column with other columns in the dataset. If a dataset has two or more columns that are correlated, the dataset is called multicollinear. An extreme example of a multicollinear dataset is one in which each row represents a person, with an age column and a year-of-birth column.
Support Vector Regression with SGD Training Using C#
The Data Science Lab Support vector regression can predict numeric values effectively, and this article shows how to implement and train a kernel SVR model in C# using stochastic sub-gradient descent. The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank might want to predict the maximum safe loan amount for a customer, based on age, account balance, annual income, and so on. There are approximately a dozen common regression techniques.
AdaBoost.R2 Regression Using C#
The Data Science Lab The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank might want to predict the maximum safe loan amount for a customer based on age, account balance, current debt, and so on. Regression techniques fall into two categories: those based on decision tree structures, and those that use non-tree techniques.
Quadratic Regression with Pseudo-Inverse Training Using C#
The Data Science Lab The goal of a machine learning regression problem is to predict a single numeric value. For example, you might want to predict a person's credit rating based on age, annual salary, bank account balance, and so on. There are approximately a dozen common regression techniques. The most basic technique is called linear regression, or sometimes multiple linear regression, where the "multiple" indicates two or more predictor variables.
Decision Tree Regression from Scratch Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of decision tree regression from scratch using JavaScript. The goal of decision tree regression is to predict a single numeric value. For simplicity and better maintenance, the demo implementation uses list storage instead of pointers. For better customization and interpretability, the implementation uses list iteration instead of recursion or a stack algorithm.
Random Forest Regression Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end example of random forest regression to predict a single numeric value, implemented using C#. A random forest is a collection of basic decision tree regressors that have been trained on different subsets of the source training data. The technique reduces model overfitting to give more accurate predictions on new, previously unseen data. The goal of a machine learning regression problem is to predict a single numeric value.
Quadratic Regression with SGD Training Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of quadratic regression, with SGD training, implemented from scratch, using JavaScript. Compared to standard linear regression, quadratic regression is better able to handle data with a non-linear structure, and data with interactions between predictor variables. The goal of a machine learning regression problem is to predict a single numeric value.
Decision Tree Regression from Scratch Without Pointers or Recursion Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of decision tree regression from scratch using the C# language. The goal of decision tree regression is to predict a single numeric value. The demo implementation doesn't use pointers (references) for simplicity and does not use recursion for better maintainability and customization.
Linear Regression with Pseudo-Inverse Training Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of linear regression with pseudo-inverse training implemented using JavaScript. Compared to other training techniques, such as stochastic gradient descent, pseudo-inverse training does not require any parameters and so it is especially simple to use. The goal of a machine learning regression problem is to predict a single numeric value.
Quadratic Regression with SGD Training Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of quadratic regression, implemented from scratch, with SGD training, using C#. Compared to standard linear regression, quadratic regression is better able to handle data with a non-linear structure, and data with interactions between predictor variables. The goal of a machine learning regression problem is to predict a single numeric value.
Kernel Ridge Regression with Cholesky Inverse Training Using JavaScript
The Data Science Lab The goal of a machine learning regression problem is to predict a single numeric value. For example, you might want to predict a person's bank savings account balance based on age, annual income, and so on. There are approximately a dozen common regression techniques. Examples include linear regression, nearest neighbors regression, quadratic regression, decision tree regression (several types, such as random forest and gradient boost), and neural network regression.
Linear Regression with Pseudo-Inverse Training Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of linear regression using pseudo-inverse training. Compared to other training techniques, such as stochastic gradient descent, pseudo-inverse training does not require any parameters and so it is especially simple to use. The goal of a machine learning regression problem is to predict a single numeric value.
Anomaly Detection Using K-Means Clustering with JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of anomaly detection using k-means data clustering, implemented with JavaScript. Compared to other anomaly detection techniques, k-means anomaly detection is simple to implement, simple to interpret, and simple to customize. Anomaly detection is the process of examining a dataset to find data items that are unusual in some way.
Decision Tree Regression from Scratch with Pointers Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of decision tree regression from scratch using the C# language. The goal of decision tree regression is to predict a single numeric value. The demo implementation uses pointers (references) for efficiency but does not use any recursion for better maintainability and customization.
ANOVA Using JavaScript
The Data Science Lab Analysis of variance (ANOVA) is a classical statistics technique that's used to infer if the unknown means (averages) of three or more groups are likely to all be equal or not, based on the variances of samples from the groups. For example, imagine that there are four high schools and you want to know if the average math abilities of students in the four schools are the same or not.
Tsetlin Machine Binary Classification Using C#
The Data Science Lab The goal of a machine learning binary classification problem is to predict a variable that has exactly two possible values. For example, you might want to predict the sex of a company employee (male = 0, female = 1) based on salary, height, years of education, and so on. There are approximately a dozen common binary classification techniques.
Linear Regression with Two-Way Interactions Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of linear regression with two-way interactions between predictor variables. Standard linear regression predicts a single numeric value based only on a linear combination of predictor values. Linear regression with interactions between predictor variables can handle more complex data while retaining a high level of model interpretability.
Kernel Ridge Regression with Cholesky Inverse Training Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of the kernel ridge regression technique to predict a single numeric value. The demo uses the kernel matrix inverse (Cholesky decomposition) technique for model training. There is no single best machine learning regression technique, but when kernel ridge regression prediction works, it is often highly accurate. The goal of a machine learning regression problem is to predict a single numeric value.
Kernel Ridge Regression with Stochastic Gradient Descent Training Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of the kernel ridge regression technique to predict a single numeric value. The demo uses stochastic gradient descent, one of two possible training techniques. There is no single best machine learning regression technique, but when kernel ridge regression prediction works, it is often very accurate. The goal of a machine learning regression problem is to predict a single numeric value.
Computing the Determinant of a Matrix Using Gaussian Elimination to Row Echelon Form with C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of computing the determinant of a matrix using the C# language. In machine learning scenarios, computing the determinant of a matrix is typically used during model training to determine if a matrix has an inverse or not. One of the fundamental operations in machine learning is computing the inverse of a square matrix. But not all matrices have an inverse.
Implementing k-Nearest Neighbors Regression Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of k-nearest neighbors regression using JavaScript. There are many machine learning regression techniques, but k-nearest neighbors is especially simple to implement and the results are highly interpretable. The goal of a machine learning regression problem is to predict a single numeric value.
Kernel Ridge Regression with Stochastic Gradient Descent Training Using C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of the kernel ridge regression technique to predict a single numeric value. The demo uses stochastic gradient descent, one of two possible training techniques. There is no single best machine learning regression technique. When kernel ridge regression prediction works, it is often highly accurate. The goal of a machine learning regression problem is to predict a single numeric value.
Linear Regression Using JavaScript
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of linear regression using JavaScript. Linear regression is the simplest machine learning technique to predict a single numeric value, and a good way to establish baseline results for comparison with other more sophisticated regression techniques. The goal of a machine learning regression problem is to predict a single numeric value.
Matrix Inverse Using Cayley-Hamilton with C#
The Data Science Lab Dr. James McCaffrey presents a complete end-to-end demonstration of computing a matrix inverse using the Cayley-Hamilton technique. Compared to other matrix inverse algorithms, Cayley-Hamilton is by far the simplest and as a nice side effect gives you the matrix determinant. However, Cayley-Hamilton is not suitable for large matrices. Dozens of machine learning algorithms require computing the inverse of a matrix.
Linear Support Vector Regression Using C# with Particle Swarm Training
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the linear support vector regression (linear SVR) technique, where the goal is to predict a single numeric value. A linear SVR model uses an unusual error/loss function and cannot be trained using standard techniques, and so particle swarm optimization training is used. The goal of a machine learning regression problem is to predict a single numeric value.
Matrix Inverse Using Newton Iteration with C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of computing a matrix inverse using the Newton iteration algorithm. Compared to other algorithms, Newton iteration is simple and easy to customize, but the technique is relatively slow. Dozens of machine learning algorithms require computing the inverse of a matrix.
Linear Regression with Two-Way Interactions Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of linear regression with two-way interactions between predictor variables. Compared to standard linear regression, which predicts a single numeric value based only on a linear combination of predictor values, linear regression with interactions can handle more complex data while retaining a high level of model interpretability.
Nadaraya-Watson Kernel Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of Nadaraya-Watson kernel regression using the C# language. NW kernel regression is simple to implement and is especially effective for small datasets. The goal of a machine learning regression problem is to predict a single numeric value. For example, you might want to predict a person's income based on their age, height, high school GPA, and so on.
Linear Support Vector Regression from Scratch Using C# with Evolutionary Training
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the linear support vector regression (linear SVR) technique, where the goal is to predict a single numeric value. A linear SVR model uses an unusual error/loss function and cannot be trained using standard simple techniques, and so evolutionary optimization training is used.
Neural Network Quantile Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of neural network quantile regression. The goal of a quantile regression problem is to predict a single numeric value with an assurance such as, "The predicted y value is 0.6789 and there's roughly a 90% chance the prediction will be greater than or equal to the true y value." The goal of a machine learning regression problem is to predict a single numeric value.
Poisson Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demo of Poisson regression, where the goal is to predict a count of things arriving, such as the number of telephone calls received in a 10-minute interval at a call center. When your source data is close to mathematically Poisson distributed, Poisson regression is simple and effective. The goal of a machine learning regression problem is to predict a single numeric value.
Naive Bayes Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the naive Bayes regression technique, where the goal is to predict a single numeric value. Compared to other machine learning regression techniques, naive Bayes regression is usually less accurate, but is simple, easy to implement and customize, works on both large and small datasets, is highly interpretable, and doesn't require tuning any hyperparameters.
Random Neighborhoods Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the random neighborhoods regression technique, where the goal is to predict a single numeric value. Compared to other ML regression techniques, advantages are that it can handle both large and small datasets, and the results are highly interpretable. The goal of a machine learning regression problem is to predict a single numeric value.
Gradient Boosting Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the gradient boosting regression technique, where the goal is to predict a single numeric value. Compared to existing library implementations of gradient boosting regression, a from-scratch implementation allows much easier customization and integration with other .NET systems.
Simple k-NN Regression Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of k-nearest neighbors regression to predict a single numeric value. Compared to other machine learning regression techniques, k-NN regression is often slightly less accurate, but is very simple to implement and customize, and the results are highly interpretable. The goal of a machine learning regression problem is to predict a single numeric value.
DBSCAN Clustering and Anomaly Detection Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of data clustering and anomaly detection using the DBSCAN (Density Based Spatial Clustering of Applications with Noise) algorithm. Compared to other anomaly detection systems based on data clustering, DBSCAN can find significantly different types of anomalies.
Winnow Classification Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the Winnow classification technique. Winnow classification is used for a very specific scenario where the target variable to predict is binary and all the predictor variables are also binary. The Winnow machine learning classification algorithm is used for a very specific scenario where the target variable to predict is binary and all the predictor variables are binary too.
Implementing k-NN Classification Using C#
The Data Science Lab Dr. James McCaffrey of Microsoft Research presents a full demo of k-nearest neighbors classification on mixed numeric and categorical data. Compared to other classification techniques, k-NN is easy to implement, supports numeric and categorical predictor variables, and is highly interpretable. Multi-class classification is a machine learning technique to predict the value of a variable that can be one of three or more possible values.
Logistic Regression with Batch SGD Training and Weight Decay Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete end-to-end program that explains how to perform binary classification (predicting a variable with two possible discrete values) using logistic regression, where the prediction model is trained using batch stochastic gradient descent with weight decay. Logistic regression is a technique for binary classification -- predicting one of two discrete values.
AdaBoost Binary Classification Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a C# program that illustrates using the AdaBoost algorithm to perform binary classification for spam detection. Compared to other classification algorithms, AdaBoost is powerful and works well with small datasets, but is sometimes susceptible to model overfitting. AdaBoost ("adaptive boosting") is a powerful machine learning technique for binary classification -- predicting a variable that has two possible values.
Artificial Immune Systems for Intrusion Detection Using C#
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a demonstration program that models biological immune systems to identify network intrusion threats. The demo illustrates challenges with artificial immune systems as well as promising new approaches. An artificial immune system (AIS) for intrusion detection is a software system that loosely models some parts of the behavior of the human immune system to protect computer networks from viruses and similar cyber-attacks.
Data Anomaly Detection Using LightGBM
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a complete program that uses the Python language LightGBM system to create a custom autoencoder for data anomaly detection. You can easily adapt the demo program for your own anomaly detection scenarios. If you have a set of data items, the goal of anomaly detection is to find items that are different in some way from most of the items. Anomaly detection is sometimes called outlier detection.
Data Dimensionality Reduction Using a Neural Autoencoder with C#
The Data Science Lab Dr. James McCaffrey of Microsoft Research presents a full-code, step-by-step tutorial on creating an approximation of a dataset that has fewer columns. Imagine that you have a dataset that has many columns (dimensions). In some scenarios it's useful to create an approximation of the dataset that has fewer columns. This is called dimensionality reduction.
Binary Classification Using LightGBM
The Data Science Lab Dr. James McCaffrey from Microsoft Research presents a full-code, step-by-step tutorial on using the LightGBM tree-based system to perform binary classification (predicting a discrete variable that has exactly two possible values). A binary classification problem is one where the goal is to predict a discrete variable that has exactly two possible values.
Nearest Centroid Classification for Numeric Data Using C#
The Data Science Lab Here's a complete end-to-end demo of what Dr. James McCaffrey of Microsoft Research says is arguably the simplest possible classification technique. The goal of a machine learning classification system is to predict the value of a discrete variable. For example, you might want to predict the sex (male or female -- binary classification) of person based on their age, income, and so on.
Regression Using LightGBM
The Data Science Lab Dr. James McCaffrey of Microsoft Research presents a full-code, step-by-step tutorial on this powerful machine learning technique used to predict a single numeric value. A regression problem is one where the goal is to predict a single numeric value. For example, you might want to predict a person's annual income from their sex, age, state of residence and political leaning. There are many machine learning techniques for regression.
Clustering Mixed Categorical and Numeric Data Using k-Means with C#
The Data Science Lab Dr. James McCaffrey of Microsoft Research presents a full-code, step-by-step tutorial on a "very tricky" machine learning technique. Data clustering is the process of grouping data items together so that similar items are in the same group/cluster. For strictly numeric data, the k-means clustering technique is relatively simple, effective, and is the technique most commonly used.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with James
Contact James, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack