ML.NET Task Metrics

ML.Net has the capability of utilizing 7 different Machine Learning Tasks via the MLContext object:

1. Binary Classification
2. Multi-class/text Classification
3. Regression and Recommendation
4. Clustering
5. Ranking
6. Anomaly Detection
7. sentence similarity

 

Each task offers various performance metrics for evaluating the model after training is completed

These metrics are properties accessible via the Evaluate() method within each task object (i.e. MLContext.MLTask.Evaluate()

 

Sample Code Snippet

    static void Main(string[] args)

    {

        MLContext mlContext = new MLContext();

 

        // 1a. Create training data

        HouseData[] houseData = {

               new HouseData() { Size = 1.1F, Price = 1.2F },

               new HouseData() { Size = 1.9F, Price = 2.3F },

               new HouseData() { Size = 2.8F, Price = 3.0F },

               new HouseData() { Size = 3.4F, Price = 3.7F } };

 

        // 1b. Import training data

        IDataView trainingData = mlContext.Data.LoadFromEnumerable(houseData);

 

        // 2. Specify data preparation and model training pipeline

        var pipeline = mlContext.Transforms.Concatenate("Features", new[] { "Size" })

            .Append(mlContext.Regression.Trainers.Sdca(labelColumnName: "Price", maximumNumberOfIterations: 100));

 

        // 3. Train model

        var model = pipeline.Fit(trainingData);

 

        //***** Model Evaluation

        HouseData[] testHouseData =

        {

            new HouseData() { Size = 1.1F, Price = 0.98F },

            new HouseData() { Size = 1.9F, Price = 2.1F },

            new HouseData() { Size = 2.8F, Price = 2.9F },

            new HouseData() { Size = 3.4F, Price = 3.6F }

        };

 

        var testHouseDataView = mlContext.Data.LoadFromEnumerable(testHouseData);

        var testPriceDataView = model.Transform(testHouseDataView);

 

        var metrics = mlContext.Regression.Evaluate(testPriceDataView, labelColumnName: "Price");

        double rs = metrics.RSquared;

        double rmse = metrics.RootMeanSquaredError;

    }

 

Metrics Summary

Listed below is a summary of 6 various ML.NET Tasks and their metrics:

 

BinaryClassification	MulticlassClassification	Regression
Accuracy	Micro Accuracy	R-Squared
AUC	Macro Accuracy	MAE (Mean Absolute Error)
AreaUnderPrecisionRecallCurve	Log-Loss	MSE (Mean Squared Error)
F1-score	Log Loss Reduction	RMSE (Root Mean Square Error)

 

 

Clustering	Ranking	AnomalyDetection
Avg Distance	DCG (Discounted Cumulative Gains)	Area Under ROC Curve
Davies Boulding Index	Normalized DCG (Discounted Cumulative Gains)	Detection Rate At False Positive Count
NMI (Normalized Mutual Information)

 

 

Reference: https://learn.microsoft.com/en-us/dotnet/machine-learning/resources/metrics