Tutorial: Analyze sentiment of website comments in a web application using ML.NET Model Builder

Article
11/10/2022

Learn how to analyze sentiment from comments in real time inside a web application.

This tutorial shows you how to create an ASP.NET Core Razor Pages application that classifies sentiment from website comments in real time.

In this tutorial, you learn how to:

Create an ASP.NET Core Razor Pages application
Prepare and understand the data
Choose a scenario
Load the data
Train the model
Evaluate the model
Use the model for predictions

You can find the source code for this tutorial at the dotnet/machinelearning-samples repository.

Pre-requisites

For a list of pre-requisites and installation instructions, visit the Model Builder installation guide.

Create a Razor Pages application

Create an ASP.NET Core Razor Pages Application.

In Visual Studio open the Create a new project dialog.
In the "Create a new project" dialog, select the ASP.NET Core Web App project template.
In the Name text box, type "SentimentRazor" and select the Next button.
In the Additional information dialog, leave all the defaults as is and select the Create button.

Prepare and understand the data

Download Wikipedia detox dataset. When the webpage opens, right-click on the page, select Save As and save the file anywhere on your computer.

Each row in the wikipedia-detox-250-line-data.tsv dataset represents a different review left by a user on Wikipedia. The first column represents the sentiment of the text (0 is non-toxic, 1 is toxic), and the second column represents the comment left by the user. The columns are separated by tabs. The data looks like the following:

Sentiment	SentimentText
1	==RUDE== Dude, you are rude upload that carl picture back, or else.
1	== OK! == IM GOING TO VANDALIZE WILD ONES WIKI THEN!!!
0	I hope this helps.

Create a Model Builder config file

When first adding a machine learning model to the solution it will prompt you to create an mbconfig file. The mbconfig file keeps track of everything you do in Model Builder to allow you to reopen the session.

In Solution Explorer, right-click the SentimentRazor project, and select Add > Machine Learning Model.
In the dialog, name the Model Builder project SentimentAnalysis.mbconfig, and select Add.

Choose a scenario

To train your model, you need to select from the list of available machine learning scenarios provided by Model Builder.

For this sample, the task is text classification. In the Scenario step of the Model Builder extension, select the Text classification scenario.

Select an environment

Model Builder can train on different environments depending on the selected scenario.

Select Local (GPU) as your environment and click the Next step button.

Note

This scenario uses deep learning techniques which work best in GPU environments. If you don't have a GPU, choose the Local (CPU) environment but note that the expected time to train will be significantly longer. For more information on using GPUs with Model Builder, see the GPU support in Model Builder guide.

Load the data

Model Builder accepts data from two sources, a SQL Server database or a local file in csv or tsv format.

In the data step of the Model Builder tool, select File from the data source options.
Select the button next to the Select a file text box and use File Explorer to browse and select the wikipedia-detox-250-line-data.tsv file.
Choose Sentiment from the Column to predict (Label) dropdown.
Choose SentimentText from the Text Column dropdown.
Select the Next step button to move to the next step in Model Builder.

Train the model

The machine learning task used to train the sentiment analysis model in this tutorial is text classification. During the model training process, Model Builder trains a text classification model for your dataset using the NAS-BERT neural network architecture.

Select Start Training.
Once training is complete, the results from the training process are displayed in the Training results section of the Train screen. In addition to providing training results, three code-behind files are created under the SentimentAnalysis.mbconfig file.
- SentimentAnalysis.consumption.cs - This file contains the ModelInput and ModelOutput schemas as well as the Predict function generated for consuming the model.
- SentimentAnalysis.training.cs - This file contains the training pipeline (data transforms, trainer, trainer hyperparameters) chosen by Model Builder to train the model. You can use this pipeline for re-training your model.
- *SentimentAnalysis.zip - This is a serialized zip file which represents your trained ML.NET model.
Select the Next step button to move to the next step.

Evaluate the model

The result of the training step will be one model that has the best performance. In the evaluate step of the Model Builder tool, the output section will contain the trainer used by the best-performing model in the as well as evaluation metrics.

If you're not satisfied with your evaluation metrics, some easy ways to try to improve model performance are to use more data.

Otherwise, select the Next step button to move to the Consume step in Model Builder.

Add consumption project templates (Optional)

In the Consume step, Model Builder provides project templates that you can use to consume the model. This step is optional and you can choose the method that best fits your needs for using the model.

Console application
Web API

Add the code to make predictions

Configure the PredictionEngine pool

To make a single prediction, you have to create a PredictionEngine<TSrc,TDst>. PredictionEngine<TSrc,TDst> is not thread-safe. Additionally, you have to create an instance of it everywhere it's needed within your application. As your application grows, this process can become unmanageable. For improved performance and thread safety, use a combination of dependency injection and the PredictionEnginePool service, which creates an ObjectPool<T> of PredictionEngine<TSrc,TDst> objects for use throughout your application.

Install the Microsoft.Extensions.ML NuGet package:
1. In Solution Explorer, right-click the project and select Manage NuGet Packages.
2. Choose "nuget.org" as the Package source.
3. Select the Browse tab and search for Microsoft.Extensions.ML.
4. Select the package in the list, and select the Install button.
5. Select the OK button on the Preview Changes dialog
6. Select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.
Open the Program.cs file in the SentimentRazor project.
Add the following using directives to reference the Microsoft.Extensions.ML NuGet package and SentimentRazorML.Model project:
```
using Microsoft.Extensions.ML;
using static SentimentRazor.SentimentAnalysis;
```

Configure the PredictionEnginePool<TData,TPrediction> for your application in the Program.cs file:

builder.Services.AddPredictionEnginePool<ModelInput, ModelOutput>()
    .FromFile("SentimentAnalysis.zip");

Create sentiment analysis handler

Predictions will be made inside the main page of the application. Therefore, a method that takes the user input and uses the PredictionEnginePool<TData,TPrediction> to return a prediction needs to be added.

Open the Index.cshtml.cs file located in the Pages directory and add the following using directives:
```
using Microsoft.Extensions.ML;
using static SentimentRazor.SentimentAnalysis;
```
In order to use the PredictionEnginePool<TData,TPrediction> configured in the Program.cs file, you have to inject it into the constructor of the model where you want to use it.
Add a variable to reference the PredictionEnginePool<TData,TPrediction> inside the IndexModel class inside the Pages/Index.cshtml.cs file.
```
private readonly PredictionEnginePool<ModelInput, ModelOutput> _predictionEnginePool;
```

Modify the constructor in the IndexModel class and inject the PredictionEnginePool<TData,TPrediction> service into it.

public IndexModel(ILogger<IndexModel> logger, PredictionEnginePool<ModelInput, ModelOutput> predictionEnginePool)
{
    _logger = logger;
    _predictionEnginePool = predictionEnginePool;
}

Create a method handler that uses the PredictionEnginePool to make predictions from user input received from the web page.
1. Below the OnGet method, create a new method called OnGetAnalyzeSentiment
```
public IActionResult OnGetAnalyzeSentiment([FromQuery] string text)
{

}
```
2. Inside the OnGetAnalyzeSentiment method, return Neutral sentiment if the input from the user is blank or null.
```
if (String.IsNullOrEmpty(text)) return Content("Neutral");
```
3. Given a valid input, create a new instance of ModelInput.
```
var input = new ModelInput { SentimentText = text };
```
4. Use the PredictionEnginePool<TData,TPrediction> to predict sentiment.
```
var prediction = _predictionEnginePool.Predict(input);
```
5. Convert the predicted bool value into toxic or not toxic with the following code.
```
var sentiment = Convert.ToBoolean(prediction.PredictedLabel) ? "Toxic" : "Not Toxic";
```
6. Finally, return the sentiment back to the web page.
```
return Content(sentiment);
```

Configure the web page

The results returned by the OnGetAnalyzeSentiment will be dynamically displayed on the Index web page.

Open the Index.cshtml file in the Pages directory and replace its contents with the following code:

@page
@model IndexModel
@{
    ViewData["Title"] = "Home page";
}

<div class="text-center">
    <h2>Live Sentiment</h2>

    <p><textarea id="Message" cols="45" placeholder="Type any text like a short review"></textarea></p>

    <div class="sentiment">
        <h4>Your sentiment is...</h4>
        <p>😡 😐 😍</p>

        <div class="marker">
            <div id="markerPosition" style="left: 45%;">
                <div>▲</div>
                <label id="markerValue">Neutral</label>
            </div>
        </div>
    </div>
</div>

Next, add css styling code to the end of the site.css page in the wwwroot\css directory:

/* Style for sentiment display */

.sentiment {
    background-color: #eee;
    position: relative;
    display: inline-block;
    padding: 1rem;
    padding-bottom: 0;
    border-radius: 1rem;
}

.sentiment h4 {
    font-size: 16px;
    text-align: center;
    margin: 0;
    padding: 0;
}

.sentiment p {
    font-size: 50px;
}

.sentiment .marker {
    position: relative;
    left: 22px;
    width: calc(100% - 68px);
}

.sentiment .marker > div {
    transition: 0.3s ease-in-out;
    position: absolute;
    margin-left: -30px;
    text-align: center;
}

.sentiment .marker > div > div {
    font-size: 50px;
    line-height: 20px;
    color: green;
}

.sentiment .marker > div label {
    font-size: 30px;
    color: gray;
}

After that, add code to send inputs from the web page to the OnGetAnalyzeSentiment handler.

In the site.js file located in the wwwroot\js directory, create a function called getSentiment to make a GET HTTP request with the user input to the OnGetAnalyzeSentiment handler.

function getSentiment(userInput) {
    return fetch(`Index?handler=AnalyzeSentiment&text=${userInput}`)
        .then((response) => {
            return response.text();
        })
}

Below that, add another function called updateMarker to dynamically update the position of the marker on the web page as sentiment is predicted.

function updateMarker(markerPosition, sentiment) {
    $("#markerPosition").attr("style", `left:${markerPosition}%`);
    $("#markerValue").text(sentiment);
}

Create an event handler function called updateSentiment to get the input from the user, send it to the OnGetAnalyzeSentiment function using the getSentiment function and update the marker with the updateMarker function.

function updateSentiment() {

    var userInput = $("#Message").val();

    getSentiment(userInput)
        .then((sentiment) => {
            switch (sentiment) {
                case "Not Toxic":
                    updateMarker(100.0, sentiment);
                    break;
                case "Toxic":
                    updateMarker(0.0, sentiment);
                    break;
                default:
                    updateMarker(45.0, "Neutral");
            }
        });
}

Finally, register the event handler and bind it to the textarea element with the id=Message attribute.
```
$("#Message").on('change input paste', updateSentiment)
```

Run the application

Now that your application is set up, run the application, which should launch in your browser.

When the application launches, enter This model doesn't have enough data! into the text area. The predicted sentiment displayed should be Toxic.

Running window with the predicted sentiment window