如何不让一个超平面影响多类线性内核SVM中的决策

Question

我正在使用Scikit的LinearSVC用于3级数据集。使用One-Vs-Rest(默认(策略，我完全获得3个超平面权重向量，每个尺寸number_of_features_in_dataset。现在，最终预测是基于所有3种超平面系数的组合，但是我要排除的是，第二个超平面对最终决定做出任何贡献。

我搜索并发现内部多个超人平面投票并进行最终分类，并且在领带的情况下，考虑了与单个超平面的距离。

clf = LinearSVC()
clf.fit(x_train,y_train)
y_predict = clf.predict(x_test)
print(clf.coef_) # This prints 3xnos_of_features, where each row represents hyperplane weights
#I want to exclude say 2nd hyperplane from affecting decision made in in line 3

Answer 1

您可以在每个 hyperplane 中手动添加 bias ，以偏爱其中一个类：

from sklearn.svm import LinearSVC
from sklearn.preprocessing import LabelEncoder
import numpy as np
import warnings
warnings.filterwarnings(module='sklearn*', action='ignore', category=DeprecationWarning)

class BiasedSVC(LinearSVC):
    def __init__(self, penalty='l2', loss='squared_hinge', dual=True, tol=1e-4,
                 C=1.0, multi_class='ovr', fit_intercept=True,
                 intercept_scaling=1, class_weight=None, verbose=0,
                 random_state=None, max_iter=1000, classes=None, biases=None
                 ):
        """
        Same as LinearSVC: (https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/svm/classes.py)
        But allowing to bias the hyperplane to favor a class over another. Works for multiclass classification
        :param classes: list of the classes (all the classes myst be present in y during training).
        :type classes: list of strings
        :param biases: list of biases in the alphabetical order of the classes (ex: [0.0, +0.1, -0.1]) or dict
        containing the weights by class (ex: {"class_1": 0.0, "class_2": +0.1, "class_3": -0.1})
        :type biases: list of floats or dict
        """
        super().__init__(penalty, loss, dual, tol, C, multi_class, fit_intercept, intercept_scaling, class_weight,
                         verbose, random_state, max_iter)
        # Define new variables
        self.classes = classes
        self.biases = biases
        # Transtype Biases
        self._biases = self.get_biases(self.biases)
        # Create Norm variable
        self._w_norm = None
        # Create LabelEncoder
        self._le = LabelEncoder()
    def get_biases(self, biases):
        """ Transtype the biases to get a list of floats """
        if isinstance(biases, list):
            return biases
        elif isinstance(biases, dict):
            return [biases[class_name] for class_name in self.classes]
        else:
            return [0.0 for _ in self.classes]
    def get_w_norm(self):
        """ Get the norm of the hyperplane to normalize the distance """
        self._w_norm = np.linalg.norm(self.coef_)
    def fit(self, X, y, sample_weight=None):
        # Fit the label Encoder (to change labels to indices)
        self._le.fit(y)
        # Fit the SVM using the mother class (LinearSVC) fit method
        super().fit(X, y, sample_weight)
        # Record the norm for all the hyperplanes (useful during inference)
        self.get_w_norm()
    def predict(self, X):
        """ Performa a prediction with the biased hyerplane """
        # Get the decision output (distance to the the hyperplanes separating the different classes)
        decision_y = self.decision_function(X)
        # Add the bias to each hyperplane (normalized for each class)
        dist = decision_y / self._w_norm + self._biases
        # Return the corresponding class
        return self._le.inverse_transform(np.argmax(dist, axis=1))

注意：您在培训期间不使用偏见，只有在预测期间，因为SVC会在培训期间翻译超平面以补偿您的偏见。