## Statistical learning theory -- implementation of perceptron learning algorithm in python (primitive form and dual form)

A good function 2020-11-13 02:56:56
statistical learning theory implementation perceptron

##### The learning algorithm of perceptron can be divided into primitive form and dual form .

Concrete model theory ： Statistical learning —— perceptron
In this paper, two algorithms in the book are introduced , utilize python It's done .
When the machine learning algorithm is implemented , You can use powerful mathematical operations numpy library . But in this article , of no avail numpy library . Data sets are entered as arrays , When you do the relevant vector operations , Use the subscript of the array to operate on the data in the array .

## 1. The original form of perceptron learning algorithm

``````def vectorInnerProduct(vector1,vector2): # The realization of two vectors of inner product
result = 0
length = len(vector1)
for i in range(length):
result += vector1[i] * vector2[i]
return result
for i in range(len(vector1)):
vector1[i] += vector2[i]
return vector1
def numberMultiply(num,vector): # Realize the multiplication of vectors
tempVector = []
for i in range(len(vector)):
tempVector.append(vector[i] * num)
return tempVector
# You can't modify the original directly vector, Otherwise, the main function brought into the perceptron will modify the weight and bias ,
# While modifying the original dataset
"""
The three functions above , It's for the convenience of data （ vector ） Do the relevant operations , Write auxiliary function . It's simple
"""
def perceptron(bias,dataSet,learnRate):
# The original algorithm of perceptron , You need to enter three variables , bias , Data sets and learning rates .
weightVector = [0 for i in range(len(dataSet))] # The weight vector is initialized to 0
while True:
# Because we have to traverse the training set , Until there are no misclassification points , So use a while loop , And a variable that records the number of misclassification points errornum
errorNum = 0
for data in dataSet: # Go through the data set over and over again , To iterate
if data * (vectorInnerProduct(weightVector,data)+bias) <= 0:
errorNum += 1
bias += learnRate * data
if errorNum == 0: # If there is no misclassification point , Exit loop
break
return weightVector,bias # Return model parameters
``````
##### You can go on and on , Use the learning model to judge the classification of any instance point . This function only needs a simple if Statement can be implemented .
``````bias = 0
dataSet = [[[3,3],1],[[4,3],1],[[1,1],-1]]
learnRate = 1
print(perceptron(bias,dataSet,learnRate))
# This example is from the statistical learning method book , The return result is ([1, 1], -3)
``````

## 2. Dual form of perceptron learning algorithm

In the learning process of dual algorithm , The examples in the training set only appear in the form of inner product , We can calculate the inner product of the data in the training set in advance , And store it in matrix form , namely Gram matrix

``````def vectorInnerProduct(vector1,vector2): # Vector inner product
result = 0
length = len(vector1)
for i in range(length):
result += vector1[i] * vector2[i]
return result
def gramMatrix(dataSet): # Calculation gram matrix
length = len(dataSet)
gramMatrix = [[0 for i in range(length)] for i in range(length)]
for i in range(length):
for j in range(length):
gramMatrix[i][j] = vectorInnerProduct(dataSet[i],dataSet[j])
return gramMatrix
for i in range(len(vector1)):
vector1[i] += vector2[i]
return vector1
def numberMultiply(num,vector): # Number multiplication
tempVector = []
for i in range(len(vector)):
tempVector.append(vector[i] * num)
return tempVector
def perceptron(dataSet,learnRate): # The dual variables are data set and learning rate ,alpha And the offset is set to... In the function 0
n = len(dataSet)
alphaList= [0 for i in range(n)]
bias = 0
gram = gramMatrix(dataSet)
while True: # The specific idea is the same as the original algorithm , It's just some details （ Judging conditions and learning expressions ） Modified it
errorNum = 0
for i in range(n):
tempSum = 0
for j in range(n):
tempSum += alphaList[j] * dataSet[j] * gram[j][i]
if dataSet[i] * (tempSum + bias) <= 0:
errorNum += 1
alphaList[i] += learnRate
bias += learnRate * dataSet[i]
if errorNum == 0:
break
# What you learn in the process of learning is alpha, Take advantage of learning alpha Calculate the final weight vector
weightVector = numberMultiply(alphaList*dataSet,dataSet)
for i in range(1,n):
``````dataSet = [[[3,3],1],[[4,3],1],[[1,1],-1]]