2001-A Simple Approach to Ordinal Classification

2001-A Simple Approach to Ordinal Classification1.科学问题如何在不修改现有分类算法的前提下使其能够利用类别之间的顺序信息从而提升分类性能。如何在分类模型中有效引入有序关系同时保持模型的通用性和灵活性使其可以适用于能产出概率的基础分类器。2.核心思想作者将一个包含 k 个有序类别的分类问题转化为 k-1 个二分类问题。每个二分类器学习目标变量是否大于某个阈值即是否超过某一类别。最终通过组合这些二分类器的输出概率恢复原始类别的概率分布。Pr(V1)1−Pr(TargetV1) Pr(V_1) 1 - Pr(Target V_1)Pr(V1​)1−Pr(TargetV1​)Pr(Vi)Pr(TargetVi−1)×(1−Pr(TargetVi)),1ik Pr(V_i) Pr(Target V_{i-1}) \times \left(1 - Pr(Target V_i)\right), \quad 1 i kPr(Vi​)Pr(TargetVi−1​)×(1−Pr(TargetVi​)),1ikPr(Vk)Pr(TargetVk−1) Pr(V_k) Pr(Target V_{k-1})Pr(Vk​)Pr(TargetVk−1​)把多分类问题转成 k−1 个“是否超过某个等级”的问题再通过概率差分恢复原始类别概率。3.代码importnumpyasnpimportpandasaspdfromsklearn.linear_modelimportLogisticRegressionfromsklearn.baseimportclonefromsklearn.preprocessingimportKBinsDiscretizerfromsklearn.model_selectionimportRepeatedStratifiedKFoldfromsklearn.metricsimportaccuracy_scoredefmake_ordinal_target(y_continuous:np.ndarray,n_bins:int)-np.ndarray:y_2dy_continuous.reshape(-1,1)discKBinsDiscretizer(n_binsn_bins,encodeordinal,strategyquantile)y_binneddisc.fit_transform(y_2d).astype(int).ravel()returny_binnedclassOrdinalClassifier():def__init__(self,base_estimator,class_order):self.base_estimatorbase_estimator self.class_orderclass_orderdeffit(self,X:np.ndarray,y:np.ndarray)-OrdinalClassifier:self.classes_np.array(sorted(np.unique(y)))self.thresholds_self.classes_[:-1]self.n_classes_len(self.classes_)self.models[]forthresholdinself.thresholds_:y_binary(ythreshold).astype(int)modelclone(self.base_estimator)model.fit(X,y_binary)self.models.append(model)returnselfdef_predict_gt_proba(self,X:np.ndarray)-np.ndarray:gt_probs[]formodelinself.models:probamodel.predict_proba(X)# Probability of label 1, i.e. P(y threshold)pos_idxlist(model.classes_).index(1)gt_probs.append(proba[:,pos_idx])returnnp.column_stack(gt_probs)defpredict_proba(self,X:np.ndarray)-np.ndarray:gtself._predict_gt_proba(X)n_samplesgt.shape[0]kself.n_classes_ probsnp.zeros((n_samples,k),dtypefloat)# First classprobs[:,0]1.0-gt[:,0]# Middle classesforiinrange(1,k-1):probs[:,i]gt[:,i-1]*(1.0-gt[:,i])# Last classprobs[:,k-1]gt[:,k-2]# Normalize for numerical stabilityrow_sumsprobs.sum(axis1,keepdimsTrue)nonzerorow_sums.squeeze()0probs[nonzero]probs[nonzero]/row_sums[nonzero]returnprobsdefpredict(self,X:np.ndarray)-np.ndarray:probsself.predict_proba(X)returnself.classes_[np.argmax(probs,axis1)]defevaluate_models(X:np.ndarray,y_ord:np.ndarray,random_state:int42)-None:cvRepeatedStratifiedKFold(n_splits10,n_repeats10,random_staterandom_state)ord_scores[]baseLogisticRegression()fortrain_idx,test_idxincv.split(X,y_ord):X_train,X_testX[train_idx],X[test_idx]y_train,y_testy_ord[train_idx],y_ord[test_idx]ord_modelOrdinalClassifier(base_estimatorclone(base),class_orderlist(sorted(np.unique(y_ord))))ord_model.fit(X_train,y_train)ord_predord_model.predict(X_test)ord_scores.append(accuracy_score(y_test,ord_pred))print(fOrdinal mean{np.mean(ord_scores):.4f}std{np.std(ord_scores):.4f})if__name____main__:dfpd.read_csv(datasets/Abalone.csv,headerNone)Xdf.iloc[:,:-1].to_numpy()ydf.iloc[:,-1].to_numpy()y_ordmake_ordinal_target(y,len(np.array(sorted(np.unique(y)))))OrdinalClassifier.evaluate_models(X,y_ord)