如何使用Python進行超參調參和調優

InfoQ 2021-10-29 11:31:55
使用 python

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#000000","name":"user"}},{"type":"strong"}],"text":"本文最初發布於rubikscode.com網站,經原作者授權由InfoQ中文站翻譯並分享。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圍繞模型優化這一主題發展出來的許多子分支之間的差异之大往往令人難以置信。其中的一個子分支叫做超參數優化,或超參數調優。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在本文中你會學到:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"機器學習中的超參數"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"前置條件和數據"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"網格搜索超參數調優"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"隨機搜索超參數調優"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"貝葉斯超參數優化"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"减半網格搜索和减半隨機搜索"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":7,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"替代選項"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"機器學習中的超參數"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"超參數是所有機器學習和深度學習算法都包含的一部分。與由算法本身學習的標准機器學習參數(如線性回歸中的w和b,或神經網絡中的連接權重)不同,"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"超參數由工程師在訓練流程之前設置"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"它們是完全由工程師定義的一項外部因素,用來控制學習算法的行為。想看些例子?學習率是最著名的超參數之一,SVM中的C也是超參數,决策樹的最大深度同樣是一個超參數,等等。這些超參數都可以由工程師手動設置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"但是,如果我們想運行多個測試,超參數用起來可能會很麻煩。於是我們就需要對超參數做優化了。這些技術的主要目標是找到給定機器學習算法的最佳超參數,以在驗證集上獲得最佳評估性能。在本教程中,我們探索了幾種可以為你提供最佳超參數的技術。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前置條件和數據"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"前置條件和庫"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"請安裝以下Python庫,為本文接下來的內容做准備:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"NumPy——如果你需要安裝幫助,請參考這份"},{"type":"link","attrs":{"href":"https:\/\/numpy.org\/install\/","title":null,"type":null},"content":[{"type":"text","text":"指南"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"SciKit Learn——如果你需要安裝幫助,請參考這份"},{"type":"link","attrs":{"href":"https:\/\/scikit-learn.org\/stable\/install.html","title":null,"type":null},"content":[{"type":"text","text":"指南"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"SciPy——如果你需要安裝幫助,請參考這份"},{"type":"link","attrs":{"href":"https:\/\/www.scipy.org\/install.html","title":null,"type":null},"content":[{"type":"text","text":"指南"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Sci-Kit Optimization——如果你需要安裝幫助,請參考這份"},{"type":"link","attrs":{"href":"https:\/\/scikit-optimize.github.io\/stable\/install.html","title":null,"type":null},"content":[{"type":"text","text":"指南"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"安裝完成後,請確保你已導入本教程中使用的所有必要模塊。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\n\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import f1_score\n\nfrom sklearn.model_selection import GridSearchCV, RandomizedSearchCV\n\nfrom sklearn.experimental import enable_halving_search_cv\nfrom sklearn.model_selection import HalvingGridSearchCV, HalvingRandomSearchCV\n\nfrom sklearn.svm import SVC\nfrom sklearn.ensemble import RandomForestRegressor\n\nfrom scipy import stats\nfrom skopt import BayesSearchCV\nfrom skopt.space import Real, Categorical"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"除此之外,你最好起碼熟悉一下線性代數、微積分和概率論的基礎知識。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"准備數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們在本文中使用的數據來自PalmerPenguins數據集。該數據集是最近發布的,旨在作為著名的Iris數據集的替代品。它由Kristen Gorman博士和南極洲LTER的帕爾默科考站共同創建。你可以在"},{"type":"link","attrs":{"href":"https:\/\/github.com\/allisonhorst\/palmerpenguins","title":null,"type":null},"content":[{"type":"text","text":"此處"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"或通過Kaggle獲取此數據集。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"該數據集本質上是由兩個數據集組成的,每個數據集包含344只企鹅的數據。就像Iris一樣,這個數據集也有來自帕爾默群島3個島嶼的3個種類的企鹅。此外,這些數據集包含每個物種的"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"culmen"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"維度。culmen是鳥喙的上脊。在簡化的企鹅數據中,culmen長度和深度被重命名為變量culmen_length_mm和culmen_depth_mm。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/fd\/91\/fd136bce4f63dd478585782620a2d791.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"由於這個數據集已經標記過了,我們應該能驗證我們的實驗結果。但實際情况往往沒這麼簡單,聚類算法結果的驗證通常是一個艱難而複雜的過程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們先來加載並准備PalmerPenguins數據集。首先,我們加載數據集,删除本文中不會用到的特征:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"data = pd.read_csv('.\/data\/penguins_size.csv')\n\ndata = data.dropna()\ndata = data.drop(['sex', 'island', 'flipper_length_mm', 'body_mass_g'], axis=1)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"然後我們分離輸入數據並對其進行縮放:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"X = data.drop(['species'], axis=1)\n\nss = StandardScaler()\nX = ss.fit_transform(X) \n\ny = data['species']\nspicies = {'Adelie': 0, 'Chinstrap': 1, 'Gentoo': 2}\ny = [spicies[item] for item in y]\ny = np.array(y) "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"最後,我們將數據拆分為訓練和測試數據集:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=33)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"當我們繪制這裏的數據時,圖像是下面這個樣子:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/90\/41\/9085ab63af5f7a5a1e47d867ab4b8c41.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"網格搜索超參數調優"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"超參數調優的工作手動做起來又慢又煩人。所以我們開始探索第一個,也是最簡單的超參數優化技術——網格搜索。這種技術可以加快調優工作,是最常用的超參數優化技術之一。從本質上講,它會自動化試錯流程。對於這種技術,我們提供了一個包含所有超參數值的列錶,然後該算法為每個可能的組合構建模型,對其進行評估,並選擇提供最佳結果的值。它是一種通用技術,可以應用於任何模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在我們的示例中,我們使用SVM算法進行分類。我們考慮了三個超參數——C、gamma和kernel。想要更詳細地了解它們的話請查看這篇"},{"type":"link","attrs":{"href":"https:\/\/rubikscode.net\/2020\/08\/10\/back-to-machine-learning-basics-support-vector-machines\/","title":null,"type":null},"content":[{"type":"text","text":"文章"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。對於C,我們要檢查以下值:0.1、1、100、1000;對於gamma,我們使用值:0.0001、0.001、0.005、0.1、1、3、5,對於kernel,我們使用值:“linear”和“rbf”。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"網格搜索實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"下面是代碼中的樣子:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"hyperparameters = {\n 'C': [0.1, 1, 100, 1000],\n 'gamma': [0.0001, 0.001, 0.005, 0.1, 1, 3, 5],\n 'kernel': ('linear', 'rbf')\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們這裏利用了Sci-Kit Learn庫及其SVC類,其中包含SVM分類實現。除此之外,我們還使用了GridSearchCV類,用於網格搜索優化。結合起來是這個樣子:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"grid = GridSearchCV(\n estimator=SVC(),\n param_grid=hyperparameters,\n cv=5, \n\tscoring='f1_micro', \n\tn_jobs=-1)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這個類通過構造器接收幾個參數:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"estimator——實例機器學習算法本身。我們在那裏傳遞SVC類的新實例。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"param_grid——包含超參數字典。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"cv——確定交叉驗證拆分策略。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"scoring——用於評估預測的驗證指標。我們使用F1分數。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"n_jobs——錶示要並行運行的作業數。值-1錶示正在使用所有處理器。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"剩下要做的就是使用fit方法運行訓練過程:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"grid.fit(X_train, y_train)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"訓練完成後,我們可以查看最佳超參數和這些參數的得分:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'Best parameters: {grid.best_params_}')\nprint(f'Best score: {grid.best_score_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Best parameters: {'C': 1000, 'gamma': 0.1, 'kernel': 'rbf'}\nBest score: 0.9626834381551361\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"此外,我們可以打印出所有結果:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'All results: {grid.cv_results_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Allresults: {'mean_fit_time': array([0.00780015, 0.00280147, 0.00120015, 0.00219998, 0.0240006 ,\n 0.00739942, 0.00059962, 0.00600033, 0.0009994 , 0.00279789,\n 0.00099969, 0.00340114, 0.00059986, 0.00299864, 0.000597 ,\n 0.00340023, 0.00119658, 0.00280094, 0.00060058, 0.00179944,\n 0.00099964, 0.00079966, 0.00099916, 0.00100031, 0.00079999,\n 0.002 , 0.00080023, 0.00220037, 0.00119958, 0.00160012,\n 0.02939963, 0.00099955, 0.00119963, 0.00139995, 0.00100069,\n 0.00100017, 0.00140052, 0.00119977, 0.00099974, 0.00180006,\n 0.00100312, 0.00199976, 0.00220003, 0.00320096, 0.00240035,\n 0.001999 , 0.00319982, 0.00199995, 0.00299931, 0.00199928, \n..."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"好的,現在我們構建這個模型並檢查它在測試數據集上的錶現:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"model = SVC(C=500, gamma = 0.1, kernel = 'rbf')\nmodel.fit(X_train, y_train)\n\n\npreditions = model.predict(X_test)\nprint(f1_score(preditions, y_test, average='micro'))"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"0.9701492537313433"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"結果很不錯,我們的模型用建議的超參數獲得了約97%的精度。下面是繪制時模型的樣子:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/2a\/20\/2a4a17fba74d3731a28a9e2270d07420.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"隨機搜索超參數調優"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"網格搜索非常簡單,但它的計算成本也很高。特別是在深度學習領域,訓練可能需要大量時間。此外,某些超參數可能比其他超參數更重要。於是人們提出了隨機搜索的想法,本文接下來會具體介紹。事實上,這項研究錶明,隨機搜索在做超參數優化時計算成本比網格搜索更有優勢。這種技術也讓我們可以更精確地發現重要超參數的理想值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"就像網格搜索一樣,隨機搜索會創建一個超參數值網格並選擇隨機組合來訓練模型。這種方法可能會錯過最佳組合,但是與網格搜索相比,它選擇最佳結果的幾率竟然是更高的,而且需要的時間只有網格搜索的一小部分。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"隨機搜索實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們看看它是怎樣寫成代碼的。我們再次使用Sci-KitLearn庫的SVC類,但這次我們使用RandomSearchCV類進行隨機搜索優化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"hyperparameters = {\n \"C\": stats.uniform(500, 1500),\n \"gamma\": stats.uniform(0, 1),\n 'kernel': ('linear', 'rbf')\n}\nrandom = RandomizedSearchCV(\n estimator = SVC(), \n param_distributions = hyperparameters, \n n_iter = 100, \n cv = 3, \n random_state=42, \n n_jobs = -1)\nrandom.fit(X_train, y_train)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"請注意,我們對C和gamma使用了均勻分布。同樣,我們可以打印出結果:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'Best parameters: {random.best_params_}')\nprint(f'Best score: {random.best_score_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Best parameters: {'C': 510.5994578295761, 'gamma': 0.023062425041415757, 'kernel': 'linear'}\nBest score: 0.9700374531835205"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"可以看到我們的結果接近網格搜索,但並不一樣。網格搜索的超參數C的值為500,而隨機搜索的值為510.59。僅從這一點你就可以看到隨機搜索的好處,因為我們不太可能將這個值放入網格搜索列錶中。類似地,對於gamma,我們的隨機搜索結果為0.23,而網格搜索為0.1。真正令人驚訝的是隨機搜索選擇了線性kernel而不是RBF,並且它獲得了更高的F1分數。要打印所有結果,我們使用cv_results_屬性:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'All results: {random.cv_results_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Allresults: {'mean_fit_time': array([0.00200065, 0.00233404, 0.00100454, 0.00233777, 0.00100009,\n 0.00033339, 0.00099715, 0.00132942, 0.00099921, 0.00066725,\n 0.00266568, 0.00233348, 0.00233301, 0.0006667 , 0.00233285,\n 0.00100001, 0.00099993, 0.00033331, 0.00166742, 0.00233364,\n 0.00199914, 0.00433286, 0.00399915, 0.00200049, 0.01033338,\n 0.00100342, 0.0029997 , 0.00166655, 0.00166726, 0.00133403,\n 0.00233293, 0.00133729, 0.00100009, 0.00066662, 0.00066646,\n\t \n\t ...."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們來重複上面網格搜索的步驟:使用建議的超參數創建模型,檢查測試數據集的分數並繪制模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"model = SVC(C=510.5994578295761, gamma = 0.023062425041415757, kernel = 'linear')\nmodel.fit(X_train, y_train)\npreditions = model.predict(X_test)\nprint(f1_score(preditions, y_test, average='micro'))"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"0.9701492537313433"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"哇,測試數據集上的F1分數與我們使用網格搜索時的分數完全相同。查看模型:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/58\/08\/58b9ce80e945215903c1f5e83a6fb708.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"貝葉斯超參數優化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"前兩種算法有一點很棒,那就是使用各種超參數值的所有實驗都可以"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"並行"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"運行。這可以為我們節省很多時間。然而這也是它們最大的缺陷所在。由於每個實驗都是孤立運行的,我們不能在當前實驗中使用來自過去實驗的信息。有一個專門用於解决序列優化問題的領域——基於模型的序列優化(SMBO)。在該領域探索的那些算法會使用先前的實驗和對損失函數的觀察結果,然後基於它們來試圖確定下一個最佳點。其中一種算法是貝葉斯優化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這種算法就像來自SMBO組的其他算法一樣,使用先前評估的點(在這裏指的是超參數值,但我們可以推而廣之)來計算損失函數的後驗期望。該算法使用兩個重要的數學概念——"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"高斯過程"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"和"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"采集函數"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。由於高斯分布是在隨機變量上完成的,因此高斯過程是其對函數的泛化。就像高斯分布有均值和協方差一樣,高斯過程是用均值函數和協方差函數來描述的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"采集函數是我們用來評估當前損失值的函數。可以把它看作是損失函數的損失函數。它是損失函數的後驗分布函數,描述了所有超參數值的效用。最流行的采集函數是Expected Improvement(EI):"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/d6\/1d\/d6424ce7e6eaff7dfae82c785728de1d.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"其中f是損失函數,x'是當前最優的超參數集。當我們把它們放在一起時,貝葉斯優化分3個步驟完成:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"使用先前評估的損失函數點,使用高斯過程計算後驗期望。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"選擇最大化 EI的新點集"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"計算新選擇點的損失函數"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"貝葉斯優化實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"將其帶入代碼的最簡單方法是使用Sci-Kit optimization庫,通常稱為skopt。按照我們在前面示例中使用的過程,我們可以執行以下操作:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"hyperparameters = {\n \"C\": Real(1e-6, 1e+6, prior='log-uniform'),\n \"gamma\": Real(1e-6, 1e+1, prior='log-uniform'),\n \"kernel\": Categorical(['linear', 'rbf']),\n}\nbayesian = BayesSearchCV(\n estimator = SVC(), \n search_spaces = hyperparameters, \n n_iter = 100, \n cv = 5, \n random_state=42, \n n_jobs = -1)\nbayesian.fit(X_train, y_train)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"同樣,我們為超參數集定義了字典。請注意,我們使用了Sci-Kit優化庫中的Real和Categorical類。然後我們用和使用GridSearchCV或RandomSearchCV類相同的方式來使用BayesSearchCV類。訓練完成後,我們可以打印出最好的結果:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'Best parameters: {bayesian.best_params_}')\nprint(f'Best score: {bayesian.best_score_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Best parameters: \nOrderedDict([('C', 3932.2516133086), ('gamma', 0.0011646737978730447), ('kernel', 'rbf')])\nBest score: 0.9625468164794008"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"很有趣,不是嗎?使用這種優化我們得到了完全不同的結果。損失比我們使用隨機搜索時要高一些。我們甚至可以打印出所有結果:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'All results: {bayesian.cv_results_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"All results: defaultdict(, {'split0_test_score': [0.9629629629629629,\n 0.9444444444444444, 0.9444444444444444, 0.9444444444444444, 0.9444444444444444,\n 0.9444444444444444, 0.9444444444444444, 0.9444444444444444, 0.46296296296296297,\n 0.9444444444444444, 0.8703703703703703, 0.9444444444444444, 0.9444444444444444, \n 0.9444444444444444, 0.9444444444444444, 0.9444444444444444, 0.9444444444444444, \n ....."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"使用這些超參數的模型在測試數據集上的錶現如何?我們來了解一下:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"model = SVC(C=3932.2516133086, gamma = 0.0011646737978730447, kernel = 'rbf')\nmodel.fit(X_train, y_train)\npreditions = model.predict(X_test)\nprint(f1_score(preditions, y_test, average='micro'))"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"0.9850746268656716"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"太有意思了。盡管我們在驗證數據集上的結果要差一些,但我們在測試數據集上獲得了更好的分數。下面是模型:"}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/01\/d0\/01da0073yydcc8a02f0d7bb8b280a8d0.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"加點樂趣,我們可以把所有這些模型並排放置:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/b6\/f0\/b62e8590fce37eab5f71dd5f9629aff0.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"减半網格搜索和减半隨機搜索"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"幾個月前,Sci-Kit Learn庫引入了兩個新類,HalvingGridSearchCV和HalvingRandomSearchCV。他們聲稱,這兩個類“可以更快地找到一個理想參數組合”。這些類使用連續减半方法來搜索指定的參數值。該技術開始使用少量資源評估所有候選者,並使用越來越多的資源迭代地選擇最佳候選者。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"從减半網格搜索的角度來看,這意味著在第一次迭代中,所有候選者都將在少量訓練數據上進行訓練。下一次迭代將只包括在前一次迭代中錶現最好的候選者,這些模型將獲得更多資源,也就是更多的訓練數據,然後再做評估。這個過程將繼續,並且减半網格搜索將只保留前一次迭代中的最佳候選者,直到只剩最後一個為止。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"整個過程由兩個參數控制——min_samples和factor。第一個參數min_samples錶示進程開始時的數據量。每次迭代時,該數據集將按factor定義的值增長。該過程類似於HalvingRandomSearchCV。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"减半網格搜索和减半隨機搜索實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這裏的代碼與前面的示例類似,我們只是使用了不同的類。我們先從HalvingGridSearch開始:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"hyperparameters = {\n 'C': [0.1, 1, 100, 500, 1000],\n 'gamma': [0.0001, 0.001, 0.01, 0.005, 0.1, 1, 3, 5],\n 'kernel': ('linear', 'rbf')\n}\n\n\ngrid = HalvingGridSearchCV(\n estimator=SVC(),\n param_grid=hyperparameters,\n cv=5, \n scoring='f1_micro', \n n_jobs=-1)\ngrid.fit(X_train, y_train)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"有趣的是這段代碼只運行了0.7秒。相比之下,使用GridSearchCV類的相同代碼跑了3.6秒。前者的速度快得多,但結果有點不同:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"print(f'Best parameters: {grid.best_params_}')\nprint(f'Best score: {grid.best_score_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Best parameters: {'C': 500, 'gamma': 0.005, 'kernel': 'rbf'}\nBest score: 0.9529411764705882"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們得到了相似的結果,但並不相同。如果我們使用這些值創建模型將獲得以下精度和圖:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"model = SVC(C=500, gamma = 0.005, kernel = 'rbf')\nmodel.fit(X_train, y_train)\npreditions = model.predict(X_test)\nprint(f1_score(preditions, y_test, average='micro'))"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"0.9850746268656716"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/d6\/7a\/d63a4d52435405e8d476a73ba309137a.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"然後我們對减半隨機搜索重複上述過程。有趣的是,使用這種方法我們得到了最奇怪的結果。我們可以說以這種方式創建的模型很難過擬合:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"hyperparameters = {\n \"C\": stats.uniform(500, 1500),\n \"gamma\": stats.uniform(0, 1),\n 'kernel': ('linear', 'rbf')\n}\nrandom = HalvingRandomSearchCV(\n estimator = SVC(), \n param_distributions = hyperparameters, \n cv = 3, \n random_state=42, \n n_jobs = -1)\nrandom.fit(X_train, y_train)\nprint(f'Best parameters: {random.best_params_}')\nprint(f'Best score: {random.best_score_}')"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"Best parameters: {'C': 530.8767414437036, 'gamma': 0.9699098521619943, 'kernel': 'rbf'}\nBest score: 0.9506172839506174"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/2c\/29\/2c4a3b1f9613afb1252c5af6c1ab1729.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"其他替代品"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"一般來說,前面描述的這些方法是最流行和最常用的。但是,如果上面介紹的方案不適合你,你還可以考慮多種替代方案。其中之一是基於梯度的超參數值優化。該技術會計算關於超參數的梯度,然後使用梯度下降算法對其進行優化。這種方法的問題在於,要讓梯度下降過程正常運行,我們需要凸且平滑的函數,但超參數這個領域並不是總有這樣的條件。另一種方法是使用進化算法進行優化。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"小結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在本文中,我們介紹了幾種眾所周知的超參數優化和調整算法。我們學習了如何使用網格搜索、隨機搜索和貝葉斯優化來獲得超參數的最佳值。我們還學到了如何利用Sci-KitLearn類和方法在代碼中做到這一點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"感謝閱讀!"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"原文鏈接:"},{"type":"link","attrs":{"href":"https:\/\/rubikscode.net\/2021\/08\/17\/ml-optimization-pt-3-hyperparameter-optimization-with-python\/","title":null,"type":null},"content":[{"type":"text","text":"https:\/\/rubikscode.net\/2021\/08\/17\/ml-optimization-pt-3-hyperparameter-optimization-with-python\/"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]}]}]}
版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢
https://pythonmana.com/2021/10/20211029113109037s.html

  1. Python code reading (Chapter 14): List Union
  2. Lecture du Code Python (article 25): diviser les chaînes multilignes en listes
  3. Python self study notes -- operators
  4. Formation python - différences entre http et HTTPS
  5. Implementation of automatic timing comment function on Python CSDN platform
  6. python+tkinter+treeview子控件快捷键
  7. Raccourcis clavier pour les sous - contrôles Python + tkinter + treeview
  8. Analyse des données Python
  9. python+tkinter+treeview子控件快捷鍵
  10. Devine si je peux attraper Maotai avec la programmation python? Tout est ouvert à github
  11. À propos de pygame.display.set in Python Un petit problème avec mode ()
  12. Implementation of automatic timing comment function on Python CSDN platform
  13. python:dataframe进行iteritem遍历时如何将输出结果按照列分别输出为该列最后一行
  14. python:dataframe進行iteritem遍曆時如何將輸出結果按照列分別輸出為該列最後一行
  15. Python: comment le dataframe affiche les résultats de sortie par colonne à la dernière ligne de la colonne lors de la traversée de l'itemitem
  16. Écrivez un gadget de bureau pour votre fille préférée en python et elle dit que c'est génial!
  17. Introduction to closures in Python 3
  18. Global / nonlocal usage in Python 3
  19. Introduction to context manager in Python 3
  20. Python crawler selenium framework. You can start with these five questions | Python skill tree
  21. Common standard library random, python introductory tutorial 5 or 6 questions a day | Python skill tree
  22. It is said that Python is omnipotent. It's really good to see Liyang photography circle with Python this time
  23. 【Python 爬虫】 4、爬虫基本原理
  24. 【Python 爬蟲】 4、爬蟲基本原理
  25. 【 Python crawler】 4. Principes de base du crawler
  26. 这道python题到底应该要怎么做
  27. Que doit faire exactement ce problème Python
  28. Après l'importation des variables du module Python, les valeurs imprimées sont fixes.
  29. Nouveau singe Muzi Lee: 0 cours de formation Python de base types de hachage pour les opérations Python redis
  30. Looking at problems from a fresh perspective: analyzing selenium principle from the perspective of Python
  31. Insérez le format de date dans la base de données MySQL en python et ne l'exécutez pas.
  32. Try Python 3.10 with CONDA
  33. Répondez en python et demandez à quelqu'un de vous aider.
  34. Un simple problème de travail Python, qui ne fonctionne pas
  35. Problèmes d'écriture Python pour la boucle
  36. Comment Python exécute les commandes du programme à plusieurs reprises au lieu de quitter
  37. YYDS! Dexplot: one line of Python code to easily draw statistical charts!
  38. pandas生成的透视表如何和源数据一起保存
  39. pandas生成的透視錶如何和源數據一起保存
  40. Comment sauvegarder le tableau pivot généré par pandas avec les données sources
  41. 10 fois plus efficace avec cache dans le développement de Django
  42. 求Python *.svg文件操作方法
  43. 求Python *.svg文件操作方法
  44. Trouver la méthode de fonctionnement du fichier Python *.Svg
  45. 【 python】 Internal Guide for Unit Test Practice
  46. 用Python编程佩尔数列pell数列循环结构
  47. 【 python】 échafaudage fastapi: spécification du développement du projet d'interface arrière fastapi
  48. [Python] restful Specification Practice Based on fastapi
  49. Python代码阅读(第26篇):将列表映射成字典
  50. How to use Python to make a screen color extractor with Exe file
  51. Lecture du Code Python (article 26): cartographie des listes dans les dictionnaires
  52. Python代码阅读(第26篇):将列表映射成字典
  53. Python代碼閱讀(第26篇):將列錶映射成字典
  54. Lecture du Code Python (article 26): cartographie des listes dans les dictionnaires
  55. 使用 Python 进行数据可视化之Seaborn
  56. Real time access to stock data, free—— Python crawler Sina stock actual combat
  57. Seaborn pour la visualisation des données en python
  58. 浅识XPath(熟练掌握XPath的语法)【python爬虫入门进阶】(03)
  59. Python中if else语句进行操作的时候哪里除了错,搞不懂
  60. Python题,我刚学,还不会