[ML]Sklearn的scoring parameters

對於Machine Learning / Deep Learning，常會根據Task的不同而使用不同的Evaluation Metrics，例如MAE、MSE、RMSE…等等，在sklearn中當然也提供了function讓user能夠快速地進行計算，比方說如果想要算MAE的話可以這樣寫：

1
2
3
4
5
6
7
8
9
10
11
12

from sklearn.metrics import mean_absolute_error
y_true = [3, -0.5, 2, 7]
y_pred = [2.5, 0.0, 2, 8]
mean_absolute_error(y_true, y_pred)
 
y_true = [[0.5, 1], [-1, 1], [7, -6]]
y_pred = [[0, 2], [-1, 2], [8, -5]]
mean_absolute_error(y_true, y_pred)
 
mean_absolute_error(y_true, y_pred, multioutput='raw_values')
 
mean_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])

透過呼叫sklearn.metrics下的mean_absolute_error()來進行計算，但如果仔細看sklearn的document的話會發現在scoring parameter下有一些奇怪的名詞，例如：

• neg_mean_absolute_error
• neg_mean_squared_error
• neg_mean_squared_log_error
• neg_median_absolute_error

不難猜測前綴的neg_是指negative，所以這些值實際上應該是原本的值加上一個負號。

不過為什麼要將這些值加上一個負號呢？看了一下scoring這章的說明有提到：

All scorer objects follow the convention that higher return values are better than lower return values.

也就是說，這裡的所有scorer obj都被定義為分數越高越好，所以對於一些分數越低越好的評估標準(MAE、MSE…)就要加上一個負號反過來。

那為什麼要這樣做呢？在stackoverflow中找到了這篇解釋，覺得解釋得還不錯：

The sklearn.metrics.log_loss is an implementation of the error metric as typically defined, and which is as most error metrics a positive number. In this case, it is a metric which is generally minimized (e.g. as mean squared error for regression), in contrast to metrics such as accuracy which is maximized. The ‘neg_log_loss’ is a hence a technicality to make create a utility value, which allows optimizing functions and classes of sklearn to maximize this utility without having to change the function’s behavior for each metric (such include for instance named cross_val_score, GridSearchCV, RandomizedSearchCV, and others).

簡而言之，一般在做cross vaildation的時候會透過迴圈讓他自己去跑固定value range內的參數，然後去比較究竟哪一個參數的loss(或是score)越大越好，注意到在sklearn的相關方式實作都是以越大越好做為比較值，所以這裡就呼應到上述為什麼需要把一些metrics的正負翻轉過來了，例如下面的例子，在cross_val_score中指定”neg_mean_absolute_error”的scoring：

1
2
3
4
5
6
7
8
9
10

from sklearn import svm, datasets
from sklearn.model_selection import cross_val_score
iris = datasets.load_iris()
X, y = iris.data, iris.target
clf = svm.SVC(gamma='scale', random_state=0)
cross_val_score(clf, X, y, scoring='recall_macro',
                cv=5)  
 
model = svm.SVC()
cross_val_score(model, X, y, cv=5, scoring='neg_mean_absolute_error')

而最後要注意的是，雖然scoring object是negative的，但在sklearn.metric下的function仍然是正常的定義，不要搞混了！

• ← Newer Post
• Older Post →