企业官网建设流程全解析

1.决策树的相关知识

在之前的接触中决策树直观印象应该就是if-else的循环，if会怎么样，else之后再继续if-else直至最终的结果。在上节讲的kNN它其实已经可以完成很多任务，但是它最大的缺点就是无法给数据集的内在含义，决策树的主要优势在于数据形式非常容易理解。

我们来对决策树进行一个总的概括，然后对程序进行解释。优点：计算复杂度不高，输出结果易于理解，对中间值的缺失不敏感，可以处理不相关特征数据。缺点：可能会产生过度匹配的问题。适用数据类型：数值型和标称型。

标称型：标称型目标变量的结果只在有限目标集中取值，如真与假(标称型目标变量主要用于分类)

数值型：数值型目标变量则可以从无限的数值集合中取值，如0.100，42.001等 (数值型目标变量主要用于回归)

2.信息增益

讲到信息增益，或者信息论也总会提到一个名人克劳德 · 香农，集合信息度量方式称为香农熵或者简称为熵，这个名字就是来源于信息之父香农。（其实，我这里会有一个经验的分享啦，就是在学习过程中，经常会遇到一些之前从来没有听过的名词或者专业术语，文人之间的咬文嚼字真的我是很无感的，但是理工科是以严谨和细致著称的，所以看论文或书籍的时候经常看到没听过的词，这时候可千万不要放弃或者烦操，看一下公式或者解释，也许会豁然开朗，不过是名字罢了，追根究底有可能只是一行公式）

熵定义为信息的期望值，在明晰这个概念之前，我们必须明白信息的定义。如果待分类的事物可能划分在多个分类之中，则符号xix_ixi​的信息定义为如下所示：

l(xi)=−log2p(xi)l(x_i) = -log_2 p(x_i)l(xi​)=−log2​p(xi​)

其中p(xi)p(x_i)p(xi​)是选择该分类的概率。

为了计算熵，我们需要计算所有类别可能值包含的信息期望值，通过下面的公式得到:

H=−∑i=1np(xi)log2p(xi)H=-\sum_{i=1}^n p(x_i)log_2 p(x_i)H=−∑i=1n​p(xi​)log2​p(xi​)

是不是看到熵，心里就一颤，莫名的烦躁呢。。。也有可能只有LZ是这样的，但是如果静下心来，会发现只是个代号，内容还是比较平易近人的哈。

3.决策树的代码实现
这里主要有两个.py文件。
第一个是trees.py，代码如下：

#coding:utf-8#导入一些操作符frommathimportlogimportoperator#创建一个简单的数据集defcreateDataSet():dataSet=[[1,1,'yes'],[1,1,'yes'],[1,0,'no'],[0,1,'no'],[0,1,'no']]labels=['no surfacing','flippers']#change to discrete valuesreturndataSet,labels#计算给定数据集的香农熵defcalcShannonEnt(dataSet):numEntries=len(dataSet)labelCounts={}forfeatVecindataSet:#the the number of unique elements and their occurancecurrentLabel=featVec[-1]ifcurrentLabelnotinlabelCounts.keys():labelCounts[currentLabel]=0labelCounts[currentLabel]+=1shannonEnt=0.0forkeyinlabelCounts:prob=float(labelCounts[key])/numEntries shannonEnt-=prob*log(prob,2)#log base 2returnshannonEnt#按照给定特征划分数据集defsplitDataSet(dataSet,axis,value):retDataSet=[]forfeatVecindataSet:iffeatVec[axis]==value:reducedFeatVec=featVec[:axis]#chop out axis used for splittingreducedFeatVec.extend(featVec[axis+1:])retDataSet.append(reducedFeatVec)returnretDataSet#选择最好的数据集划分方式defchooseBestFeatureToSplit(dataSet):numFeatures=len(dataSet[0])-1#the last column is used for the labelsbaseEntropy=calcShannonEnt(dataSet)bestInfoGain=0.0;bestFeature=-1foriinrange(numFeatures):#iterate over all the featuresfeatList=[example[i]forexampleindataSet]#create a list of all the examples of this featureuniqueVals=set(featList)#get a set of unique valuesnewEntropy=0.0forvalueinuniqueVals:subDataSet=splitDataSet(dataSet,i,value)prob=len(subDataSet)/float(len(dataSet))newEntropy+=prob*calcShannonEnt(subDataSet)infoGain=baseEntropy-newEntropy#calculate the info gain; ie reduction in entropyif(infoGain>bestInfoGain):#compare this to the best gain so farbestInfoGain=infoGain#if better than current best, set to bestbestFeature=ireturnbestFeature#returns an integer#返回出现次数最多的分类名称defmajorityCnt(classList):classCount={}forvoteinclassList:ifvotenotinclassCount.keys():classCount[vote]=0classCount[vote]+=1sortedClassCount=sorted(classCount.iteritems(),key=operator.itemgetter(1),reverse=True)returnsortedClassCount[0][0]#创建树的函数代码defcreateTree(dataSet,labels):classList=[example[-1]forexampleindataSet]ifclassList.count(classList[0])==len(classList):returnclassList[0]#stop splitting when all of the classes are equaliflen(dataSet[0])==1:#stop splitting when there are no more features in dataSetreturnmajorityCnt(classList)bestFeat=chooseBestFeatureToSplit(dataSet)bestFeatLabel=labels[bestFeat]myTree={bestFeatLabel:{}}del(labels[bestFeat])featValues=[example[bestFeat]forexampleindataSet]uniqueVals=set(featValues)forvalueinuniqueVals:subLabels=labels[:]#copy all of labels, so trees don't mess up existing labelsmyTree[bestFeatLabel][value]=createTree(splitDataSet(dataSet,bestFeat,value),subLabels)returnmyTree#使用决策树执行分类defclassify(inputTree,featLabels,testVec):#python2# firstStr = inputTree.keys()[0]#python3.5firstSides=list(inputTree.keys())firstStr=firstSides[0]secondDict=inputTree[firstStr]featIndex=featLabels.index(firstStr)key=testVec[featIndex]valueOfFeat=secondDict[key]ifisinstance(valueOfFeat,dict):classLabel=classify(valueOfFeat,featLabels,testVec)else:classLabel=valueOfFeatreturnclassLabel#使用pickle模块存储决策树defstoreTree(inputTree,filename):importpickle fw=open(filename,'w')pickle.dump(inputTree,fw)fw.close()defgrabTree(filename):importpickle fr=open(filename)returnpickle.load(fr)

第二个主要是绘制决策树图treePlotter.py

#coding:utf-8#导入matplotlib.pyplot,如果显示mo module,请自行安装importmatplotlib.pyplotasplt#定义文本框和箭头格式decisionNode=dict(boxstyle="sawtooth",fc="0.8")leafNode=dict(boxstyle="round4",fc="0.8")arrow_args=dict(arrowstyle="<-")#获取叶节点数defgetNumLeafs(myTree):numLeafs=0#如果使用的是python2，用注释掉的一行# firstStr = myTree.keys()[0]#LZ使用的是python3firstSides=list(myTree.keys())firstStr=firstSides[0]secondDict=myTree[firstStr]forkeyinsecondDict.keys():iftype(secondDict[key]).__name__=='dict':#test to see if the nodes are dictonaires, if not they are leaf nodesnumLeafs+=getNumLeafs(secondDict[key])else:numLeafs+=1returnnumLeafs#获取树的层数defgetTreeDepth(myTree):maxDepth=0#python2的用法# firstStr = myTree.keys()[0]#python3使用下面两行firstSides=list(myTree.keys())firstStr=firstSides[0]secondDict=myTree[firstStr]forkeyinsecondDict.keys():iftype(secondDict[key]).__name__=='dict':#test to see if the nodes are dictonaires, if not they are leaf nodesthisDepth=1+getTreeDepth(secondDict[key])else:thisDepth=1ifthisDepth>maxDepth:maxDepth=thisDepthreturnmaxDepth#绘制带箭头的注解defplotNode(nodeTxt,centerPt,parentPt,nodeType):createPlot.ax1.annotate(nodeTxt,xy=parentPt,xycoords='axes fraction',xytext=centerPt,textcoords='axes fraction',va="center",ha="center",bbox=nodeType,arrowprops=arrow_args)#在父子节点填充文本信息defplotMidText(cntrPt,parentPt,txtString):xMid=(parentPt[0]-cntrPt[0])/2.0+cntrPt[0]yMid=(parentPt[1]-cntrPt[1])/2.0+cntrPt[1]createPlot.ax1.text(xMid,yMid,txtString,va="center",ha="center",rotation=30)#plotTree函数defplotTree(myTree,parentPt,nodeTxt):#if the first key tells you what feat was split onnumLeafs=getNumLeafs(myTree)#this determines the x width of this treedepth=getTreeDepth(myTree)#python2# firstStr = myTree.keys()[0] #the text label for this node should be this#python3firstSides=list(myTree.keys())firstStr=firstSides[0]cntrPt=(plotTree.xOff+(1.0+float(numLeafs))/2.0/plotTree.totalW,plotTree.yOff)plotMidText(cntrPt,parentPt,nodeTxt)plotNode(firstStr,cntrPt,parentPt,decisionNode)secondDict=myTree[firstStr]plotTree.yOff=plotTree.yOff-1.0/plotTree.totalDforkeyinsecondDict.keys():iftype(secondDict[key]).__name__=='dict':#test to see if the nodes are dictonaires, if not they are leaf nodesplotTree(secondDict[key],cntrPt,str(key))#recursionelse:#it's a leaf node print the leaf nodeplotTree.xOff=plotTree.xOff+1.0/plotTree.totalW plotNode(secondDict[key],(plotTree.xOff,plotTree.yOff),cntrPt,leafNode)plotMidText((plotTree.xOff,plotTree.yOff),cntrPt,str(key))plotTree.yOff=plotTree.yOff+1.0/plotTree.totalD#if you do get a dictonary you know it's a tree, and the first element will be another dictdefcreatePlot(inTree):fig=plt.figure(1,facecolor='white')fig.clf()axprops=dict(xticks=[],yticks=[])createPlot.ax1=plt.subplot(111,frameon=False,**axprops)#no ticks#createPlot.ax1 = plt.subplot(111, frameon=False) #ticks for demo puropsesplotTree.totalW=float(getNumLeafs(inTree))plotTree.totalD=float(getTreeDepth(inTree))plotTree.xOff=-0.5/plotTree.totalW;plotTree.yOff=1.0;plotTree(inTree,(0.5,1.0),'')plt.show()#def createPlot():# fig = plt.figure(1, facecolor='white')# fig.clf()# createPlot.ax1 = plt.subplot(111, frameon=False) #ticks for demo puropses# plotNode('a decision node', (0.5, 0.1), (0.1, 0.5), decisionNode)# plotNode('a leaf node', (0.8, 0.1), (0.3, 0.8), leafNode)# plt.show()defretrieveTree(i):listOfTrees=[{'no surfacing':{0:'no',1:{'flippers':{0:'no',1:'yes'}}}},{'no surfacing':{0:'no',1:{'flippers':{0:{'head':{0:'no',1:'yes'}},1:'no'}}}}]returnlistOfTrees[i]#createPlot(thisTree)

这里贴出代码和代码配套的数据集，有兴趣的小伙伴可以自行下载。链接: https://pan.baidu.com/s/1gf664dP 密码: 32r5
需要注意的是因为LZ笔记本上装的是python3.5，为了确保代码编译成功，把原本的python2版本的代码改了一部分，注释里都有，小伙伴可以根据自己的需求进行选择O(∩_∩)O