Evaluation | CBSE Class 10 AI Project Cycle Notes – 7
1. What is the evaluation?
It is the process of testing and analyzing the results of an AI model for correctness.
2. Why is evaluation of an AI model in important?
a. It is used to check whether an AI model is producing results due to its learning or memorization.
b. It is used to ensure that AI model is accurate, smart and good at learning.
3. What do you mean by evaluation metrics?
It refers to the measures used to test the quality of the AI model.
4. What are different metrics used to evaluate classification-based AI models?
Recall, Precision, Accuracy, F1 Score
5. What do you mean by overfitting?
It refers to a situation when AI model always produces correct outputs using training data but it produces incorrect outputs with testing data.
6. What is underfitting?
It refers to situation when AI model is unable to capture the structure and relationships of its training data and give correct results.
7. What are two biggest causes for poor performance of AI models?
Overfitting and Underfitting.
8. What do you mean by True Positive?
Actual output is True (Positive)
Predicted output is also True (Positive).
9. What do you mean by True Negative?
Actual output is False (Negative).
Predicted output is also False (Negative).
10. What do you mean by False Positive?
Actual output is False (Negative)
Predicted output is True. (Positive)
11. What do you mean by False Negative?
Actual output is True (Positive).
Predicted output is False. (Negative).
12. What do you mean by Confusion Matrix?
It is a technique in which the performance of a Classification based AI model is represented in the form of a table by listing the actual as well as predicted output values.
13. What do you mean by accuracy of an AI model? What is its significance?
It is defined as percentage of correct predictions out of all the observations.
Formula= (TP+TN)/(TP+FN+FP+TN)
14. Is accuracy metric sufficient to determine the efficiency of an AI model? Give reason.
OR
Why is accuracy not considered a complete and sufficient metric?
It is not sufficient to determine the efficiency of an AI model because it may give misleading results if training data is not balanced.
15. What is accuracy paradox?
Accuracy is not sufficient to determine the efficiency of an AI model as it can mislead when the training data is not balanced.
16. What do you mean by precision of an AI model? What is its significance?
It is defined as the proportion of positive predictions out of all the observations that are positive(correct or incorrect). Its value lies between 0 and 1. Precision helps us to visualize the reliability of the machine learning model in classifying the model as positive.
Precision= TP/(TP+FP)
An AI model with high precision is considered trustworthy.
17. What is recall?
It is a metric that indicates that out of all actual positive values, how many are predicted positive. Its value lies between 0 and 1. Recall measures the model’s ability to detect positive samples.
Recall= TP/(TP+FN)
18. What is F1 score? What is its significance?
It is a metric that balances precision and recall. Its value lies between 0 and 1.
Recall= 2X(Precision X Recall)/(Precision +Recall)
19. Where is F1 score useful?
F1 score is useful when we need to compare two or more classification-based AI models for the same data. The AI model with higher F1 score is preferred.
20. Suggest a situation where the impact of false negatives would be costly than false positives.
Suppose we have an AI model that can make predictions about whether it will rain or not. This model given accurate predictions. If it says “it will rain”, it will definitely rain. So, we can trust it.
But it doesn’t give predictions about when “it will not rain”. In such a case, if our business is related to drying the crops in open area, the false predication will spoil the crops.
21. Suggest a situation where the impact of false positives would be costly than false negatives.
Suppose a disease is spreading very fast. Army has setup an AI model that can predict whether a soldier is suffering from this specific disease or not. So timely diagnosis and treatment is required. But individual testing is very expensive. But if AI model wrongly predicts that soldiers are suffering from disease, it will be a costly situation.
22. Give example of situation when you would go for precision metric.
Suppose, we have an AI device to predict rain. It claims that it will always make correct prediction. So Precision will be needed to evaluate this model.
23. What is the difference between Precision and Recall?
Precision: When you predict something positive, how many times they were actually positive (correct+incorrect).
Recall: Out of actual positive data, how many times you predicted correctly.
24. Give example of a situation where you would go for precision metric over recall.
Email Spam detection: This is one of the example where Precision is more important than Recall.
There is no problem if a spam email (positive case) doesn’t go to spam folder.
But, if an email is good (negative), then it must not go to spam folder. Here, Precision is more important than Recall. If AI model predicts something positive (spam), it must go to spam folder). otherwise, important emails may be missed.
25. Give example of a situation where you would go for recall metric over precision.
Cancer Prediction: This is one of the example where Recall is more important than Precision.
Cancer patients are tested for Tumors. AI model should predict most cases as tumorous (high false positives), but shouldn’t label a cancerous tumor as normal (false negative).
26. An AI model made the Sales prediction for a new mobile phone which they have recently launched.
i. Identify the total number of wrong predictions made by the model.
Total number of wrong predictions= False Positive + False Negative
= 40 +12
= 52
ii. Calculate Precision, Recall and F1 Score.
True Positive (TP) =50
True Negative (TN) =12
False Positive (FP) =50
False Negative (FN) =12
Precision=TP/(TP+FP)
= 50/(50+50)
=50/100
=0.5
Recall =TP/(TP+FN)
= 50/(50+12)
=50/62
=0.806
F1 Score=(2 X Precision X Recall)/(Precision+Recall)
= (2 X 0.5 X 0.806)/(0.5+0.806)
=0.806/1.306
=0.617
