You have a model such that the lowest score for a positive example is higher than the maximum score for a negative example. What is its ROC AUC?

Question

topGrad Admin · Accepted Answer

The model perfectly discriminates between positive and negative examples.

Pretend that the score predictions for all positive examples is 0.5 or higher, and the score predictions for all the negative examples are less than 0.5. Then all the positive examples have prediction scores of 0.5 or higher. All the negative examples have prediction scores less than 0.5. They are perfectly separated.

For any thresholds > 0.5, the specificity will be 1.0 (it correctly identifies all the negative examples), and the sensitivity will range from 0 to 1, so the points will run along the line y=1 (in the plot of the ROC curve, it will be the top horizontal edge of the chart.

At the threshold 0.5, the sensitivity (ability to correctly identify positive examples) will be 1.0 and the specificity will also be 1.0, so the point will be at the top right corner of the ROC curve.

At any threshold < 0.5, the sensitivity (ability to identify positive examples) will be 1.0 and the specificity will range from 1 to 0, so the point will be along the line x = 1 (the right side edge of the ROC Curve chart.

So the ROC curve is a box with width 1 and height 1, so the area under it is 1.0.

Q:

Data Analysis

Machine Learning

Similar Questions

You are working with the penguins dataset. You want to use the summarize() and max() functions to find the maximum value for the variable flipper_length_mm. You write the following…

You want to use the summarize() and max() functions to find the maximum rating for your data. Add the code chunk that lets you find the maximum value for the variable Rating. What is the…

You’ve fit a random forest of 10 trees with max depth 20. Your training ROC is 0.99 and test ROC is 0.54. Which of the following is NOT a reasonable thing to try?

You find that your training set has 70% negative examples and 30% positive. Which of the following techniques will NOT help for training this imbalanced dataset?

In some studies, you may have to compute the Positive predictive value (PPV) from the sensitivity, specificity and prevalence.

Model 1 has a c-index of 0.7 and Model 2 has a c-index of 0.6. Which is more accurate using a threshold of 0.5 for the risk score?

What is the sensitivity and specificity of a model which randomly assigns a score between 0 and 1 to each example (with equal probability) if we use a threshold of 0.7?

What is the sensitivity and specificity of a pneumonia model that always outputs positive? In other words, the models says that every patient has the disease.

Look at the output of model 1 and model 2: Which one will have a lower soft dice loss?

We have the following table the output of a model f on an example using subsets of the variable. What is the Shapley value for s_BP?

We have the following table the output of a model f on an example using subsets of the variable. What is the sum of the Shapley value for s_BP and d_BP?

True or False: A tree of depth 1 is more expressive than a classical linear model.

You have created a model using mean imputation. At test time, you should fill in missing values with:

Assume you have missing data on one of your features, and are considering two options: True or False: "Both options have the same performance".

A data analyst uses the bias() function to compare the actual outcome with the predicted outcome to determine if the model is biased. They get a score of 0.8. What does this mean?

Practice More Questions

Data Analysis

Machine Learning

Q:

Data Analysis

Machine Learning

Created with Fabric.js 4.6.0 Similar Questions

Created with Fabric.js 4.6.0 Practice More Questions

Data Analysis

Machine Learning

Similar Questions

Practice More Questions