machinelearning_notebook/3_kmeans/1-k-means.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# k-Means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "根据训练样本中是否包含标签信息，机器学习可以分为 **监督学习** 和 **无监督学习**。`聚类算法`是典型的无监督学习，其训练的样本中值包含`样本的特征`，**不包含样本的标签信息**。在聚类算法中，利用样本的特征，将具有相似特征空间分布的样本划分到同一类别中。\n",
    "\n",
    "\n",
    "![cluster illustration](images/kmeans-illustration.jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 方法\n",
    "\n",
    "由于具有出色的速度和良好的可扩展性，K-Means聚类算法最经典的聚类方法。***k-Means算法是一个重复移动类中心点（重心，centroids）的过程***:\n",
    "* 移动中心点到其包含成员的平均位置;\n",
    "* 然后重新划分其内部成员。\n",
    "\n",
    "`k`是算法中的超参数，表示类的数量；k-Means可以自动分配样本到不同的类，但是不能决定究竟要分几个类。`k`必须是一个比训练集样本数小的正整数。有时，类的数量是由问题内容指定的。例如，一个鞋厂有三种新款式，它想知道每种新款式都有哪些潜在客户，于是它调研客户，然后从数据里找出三类。也有一些问题没有指定聚类的数量，最优的聚类数量是不确定的。\n",
    "\n",
    "k-Means的参数是类的重心位置和其内部观测值的位置。与广义线性模型和决策树类似，k-Means参数的最优解也是以代价函数最小化为目标。k-Means代价函数公式如下：\n",
    "$$\n",
    "J = \\sum_{k=1}^{K} \\sum_{i \\in C_k} | x_i - u_k|^2\n",
    "$$\n",
    "\n",
    "$u_k$是第$k$个类的重心位置，定义为：\n",
    "$$\n",
    "u_k = \\frac{1}{|C_k|} \\sum_{i \\in C_k} x_i\n",
    "$$\n",
    "\n",
    "\n",
    "成本函数是各个类畸变程度（distortions）之和。每个类的畸变程度等于该类重心与其内部成员位置距离的平方和。若类内部的成员彼此间越紧凑则类的畸变程度越小，反之，若类内部的成员彼此间越分散则类的畸变程度越大。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 算法\n",
    "求解成本函数最小化的参数就是一个重复配置每个类包含的观测值，并不断移动类重心的过程。\n",
    "\n",
    "输入：$T=\\{ x_1, x_2, ..., x_N\\}$，其中$x_i \\in R_n$，i=1,2...N\n",
    "\n",
    "输出：聚类集合$C_k$, 聚类中心$u_k$, 其中k=1,2,...K\n",
    "\n",
    "1. 初始化类的重心$u_k$，可以随机选择样本作为聚类中心\n",
    "2. 每次迭代的时候，把所有样本分配到离它们最近的类，即更新聚类集合$C_k$\n",
    "3. 然后把重心移动到该类全部成员位置的平均值那里，即更新$u_k$\n",
    "4. 若达到最大迭代步数，或两次迭代差小于设定的阈值则算法结束，否则重复步骤2\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 计算过程演示"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAN7klEQVR4nO3dT2zkd3nH8c/H3sxmdqGAlMhVd6N6Dwi0QqqCVzRDVDTqcICC4NJDqMIBH3wpIfypUIJUcar2ghA50EpRMlwYwWGJqiqNSCrjOVQzWrG7iRR2F6QoQP4Q1OVAwVH1G7Lz9GC7s4286zHrn7+Px++XZGn9dx89Hr89/vnP1xEhAEBec6UHAADcGqEGgOQINQAkR6gBIDlCDQDJHanjjd51112xuLhYx5ue2ptvvqnjx48XnSELdjHBLibYxUSGXVy8ePE3EXH3ds+rJdSLi4u6cOFCHW96av1+X+12u+gMWbCLCXYxwS4mMuzC9i9v9jwufQBAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHJThdr2l2xftv0T29+zfWfdgwEANuwYatsnJH1B0pmI+ICkeUkP1D0YgPoMh0P1ej0Nh8PSo2AK0176OCKpafuIpGOSflXfSADqNBwO1el01O121el0iPUBsOPhthHxuu1vSHpF0v9Iei4innv7y9lekbQiSQsLC+r3+3s86u6sr68XnyELdjHBLqRer6eqqjQej1VVlbrdrqqqKj1WUelvFxFxywdJ75H0I0l3S7pD0r9KevBWr7O0tBSlra2tlR4hDXYxwS4iBoNBNJvNmJubi2azGYPBoPRIxWW4XUi6EDdp6jSXPj4q6ecRcS0i/iDpKUkfruWzBoDatVotra6uanl5Waurq2q1WqVHwg52vPShjUse99k+po1LHx1JF2qdCkCtWq2Wqqoi0gfEjveoI+K8pHOSLkl6cfN1Hq95LgDApmnuUSsivi7p6zXPAgDYBr+ZCADJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDVqNxwOdfbsWQ5RFbu4UZZdHIQT2af6e9TAH2vrxOvRaKRGo3Goj35iFxNZdrE1R1VV6vV6ad8n3KNGrfr9vkajka5fv67RaJT7pOeasYuJLLvYmmM8Hqd+nxBq1KrdbqvRaGh+fl6NRkPtdrv0SMWwi4ksu9iaY25uLvX7hEsfqNXWidf9fl/tdjvll5X7hV1MZNnF1hzdblfLy8tp3yeEGrVrtVppPwD2G7uYyLKLg3AiO5c+ACA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJDcVKG2/W7b52z/1PZV23n/HiAAzJhp/x71Y5J+GBF/a7sh6ViNMwEAbrDjPWrb75L0EUlPSlJEjCLitzXPBey5g3DaNLCdaS59nJJ0TdJ3bD9v+wnbx2ueC9hTW6dNd7tddTodYo0DZZpLH0ckfVDSQxFx3vZjkh6R9I83vpDtFUkrkrSwsFD8NN/19fXiM2TBLqRer6eqqjQej1VVlbrdrqqqKj1WUdwuJtLvIiJu+SDpTyX94obH/0rSv9/qdZaWlqK0tbW10iOkwS4iBoNBNJvNmJubi2azGYPBoPRIxXG7mMiwC0kX4iZN3fHSR0T8WtKrtt+3+aSOpCv1fNoA6rF12vTy8rJWV1dTH2QKvN20P/XxkKTe5k98vCzpc/WNBNTjIJw2DWxnqlBHxAuSztQ7CgBgO/xmIgAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9TAPhoOhzp79ixnNopd7Ma0BwcAuE1bB+yORiM1Go1DfdIMu9gd7lED+6Tf72s0Gun69esajUa5D1OtGbvYHUIN7JN2u61Go6H5+Xk1Gg212+3SIxXDLnaHSx/APtk6YLff76vdbh/qL/XZxe4QamAftVotorSJXUyPSx8AkByhBoDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByU4fa9rzt520/XedAAID/bzf3qB+WdLWuQWYRpywD2AtTnfBi+6SkT0j6J0lfrnWiGcEpywD2yrRHcX1L0lclvfNmL2B7RdKKJC0sLBQ/VXh9fb3oDL1eT1VVaTweq6oqdbtdVVVVZJbSu8iEXUywi4n0u4iIWz5I+qSkf978d1vS0zu9ztLSUpS2trZW9P8fDAbRbDZjfn4+ms1mDAaDYrOU3kUm7GKCXUxk2IWkC3GTpk5zj/p+SZ+y/TeS7pT0J7a/GxEP1vOpYzZwyjKAvbJjqCPiUUmPSpLttqR/INLT4ZRlAHuBn6MGgOSm/WaiJCki+pL6tUwCANgW96gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhRu04jR24Pbv6e9TAbnEaO3D7uEeNWvX7fY1GI12/fl2j0Sj3Sc9AUoQatWq322o0Gpqfn1ej0VC73S49EnDgcOkDteI0duD2EWrUjtPYgdvDpQ8ASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJLdjqG3fY3vN9hXbl20/vB+DAQA2TPP3qN+S9JWIuGT7nZIu2v6PiLhS82wAAE1xjzoi3oiIS5v//r2kq5JO1D0Y9sZwOFSv1+MEcOAA29U1atuLku6VdL6WabCntk4A73a76nQ6xBo4oKY+isv2OyT9QNIXI+J32zx/RdKKJC0sLBQ/bXp9fb34DKX1ej1VVaXxeKyqqtTtdlVVVemxiuJ2McEuJtLvIiJ2fJB0h6RnJX15mpdfWlqK0tbW1kqPUNxgMIhmsxlzc3PRbDZjMBiUHqk4bhcT7GIiwy4kXYibNHWan/qwpCclXY2Ib9b6WQN7ausE8OXlZa2urnLALHBATXPp435Jn5X0ou0XNp/2tYh4prapsGdarZaqqiLSwAG2Y6gj4j8leR9mAQBsg99MBIDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkCDUAJEeoASA5Qg0AyRFqAEiOUANAcoQaAJIj1ACQHKEGgOQINQAkR6gBILmpQm37Y7Z/Zvsl24/UPRQAYGLHUNuel/RtSR+XdFrSZ2yfrnuw2zEcDtXr9TQcDkuPAgC3bZp71B+S9FJEvBwRI0nfl/Tpesf64w2HQ3U6HXW7XXU6HWIN4MA7MsXLnJD06g2PvybpL9/+QrZXJK1I0sLCgvr9/l7Mt2u9Xk9VVWk8HquqKnW7XVVVVWSWLNbX14u9P7JhFxPsYiL7LqYJ9VQi4nFJj0vSmTNnot1u79Wb3pWjR4/+X6yPHj2q5eVltVqtIrNk0e/3Ver9kQ27mGAXE9l3Mc2lj9cl3XPD4yc3n5ZSq9XS6uqqlpeXtbq6eugjDeDgm+Ye9Y8lvdf2KW0E+gFJf1frVLep1WqpqioiDWAm7BjqiHjL9uclPStpXlI3Ii7XPhkAQNKU16gj4hlJz9Q8CwBgG/xmIgAkR6gBIDlCDQDJEWoASI5QA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEgOUINAMkRagBIjlADQHKEGgCSI9QAkByhBoDkHBF7/0bta5J+uedveHfukvSbwjNkwS4m2MUEu5jIsIs/j4i7t3tGLaHOwPaFiDhTeo4M2MUEu5hgFxPZd8GlDwBIjlADQHKzHOrHSw+QCLuYYBcT7GIi9S5m9ho1AMyKWb5HDQAzgVADQHIzGWrbH7P9M9sv2X6k9Dyl2L7H9prtK7Yv23649Ewl2Z63/bztp0vPUpLtd9s+Z/untq/abpWeqRTbX9r82PiJ7e/ZvrP0TNuZuVDbnpf0bUkfl3Ra0mdsny47VTFvSfpKRJyWdJ+kvz/Eu5CkhyVdLT1EAo9J+mFEvF/SX+iQ7sT2CUlfkHQmIj4gaV7SA2Wn2t7MhVrShyS9FBEvR8RI0vclfbrwTEVExBsRcWnz37/XxgfkibJTlWH7pKRPSHqi9Cwl2X6XpI9IelKSImIUEb8tOlRZRyQ1bR+RdEzSrwrPs61ZDPUJSa/e8PhrOqRxupHtRUn3SjpfeJRSviXpq5LGheco7ZSka5K+s3kZ6Anbx0sPVUJEvC7pG5JekfSGpP+OiOfKTrW9WQw13sb2OyT9QNIXI+J3pefZb7Y/Kem/IuJi6VkSOCLpg5L+JSLulfSmpEP5fRzb79HGV9unJP2ZpOO2Hyw71fZmMdSvS7rnhsdPbj7tULJ9hzYi3YuIp0rPU8j9kj5l+xfauBT217a/W3akYl6T9FpEbH1ldU4b4T6MPirp5xFxLSL+IOkpSR8uPNO2ZjHUP5b0XtunbDe08c2Bfys8UxG2rY1rkVcj4pul5yklIh6NiJMRsaiN28OPIiLlPae6RcSvJb1q+32bT+pIulJwpJJekXSf7WObHysdJf3G6pHSA+y1iHjL9uclPauN7+J2I+Jy4bFKuV/SZyW9aPuFzad9LSKeKTcSEnhIUm/zjszLkj5XeJ4iIuK87XOSLmnjJ6SeV9JfJedXyAEguVm89AEAM4VQA0ByhBoAkiPUAJAcoQaA5Ag1ACRHqAEguf8FNFbkKND8AT8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "X0 = np.array([7, 5, 7, 3, 4, 1, 0, 2, 8, 6, 5, 3])\n",
    "X1 = np.array([5, 7, 7, 3, 6, 4, 0, 2, 7, 8, 5, 7])\n",
    "plt.figure()\n",
    "plt.axis([-1, 9, -1, 9])\n",
    "plt.grid(True)\n",
    "plt.plot(X0, X1, 'k.');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "假设K-Means初始化时，将第一个类的重心设置在第5个样本，第二个类的重心设置在第11个样本.那么我们可以把每个实例与两个重心的距离都计算出来，将其分配到最近的类里面。计算结果如下表所示：\n",
    "![data_0](images/data_0.png)\n",
    "\n",
    "新的重心位置和初始聚类结果如下图所示。第一类用X表示，第二类用点表示。重心位置用稍大的点突出显示。\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVAklEQVR4nO3df3Dcd33n8ec7cmRwzCVtnRMXR0ZhoPRy5CCx0yJy9KSKOUhJ4K9LU4Jd8HR81zmaQOLLUdKkLdRNJ01ToNNyQ6nLJNHg8wSGaUL4dbK2w7UiYzvkLiQmM7lEsYKhDdD8UMJJtvy+P3bFyo4srWytvx9Jz8eMRvr+2O++9+31Sx99vrv7jcxEklSuM6ouQJI0N4NakgpnUEtS4QxqSSqcQS1JhTOoJalwBrXaKiLeFhGPVVzDRyPis1XWcCoioi8inq66DlXHoBYAEfHBiNgXERMR8bkF3G40It5+ou2Z+c3MfEOr+5+q2UItM/8oM3+zXfd5urW7hyrPqqoLUDEOAX8IvAN4ZcW1zCoiAojMPFp1LbOJiFWZeaTqOrT8OKIWAJn5xcz8EvCj47dFxLqIuC8ino2IH0fENyPijIi4C9gA3BsR4xFx4yy3/ekI90T7R8RbIuIfGsf/3xHRN+P2tYjYERF/D7wEvDYiPhARByLihYh4IiL+U2Pfs4CvAOc1jj8eEedFxO9HxN0zjvnuiHikcX+1iPjXM7aNRsT2iPg/EfFcRPyPiHjFbD2LiPdHxN9HxJ9FxI+A34+I1RFxe0QcjIh/jIj/HhGvnKuPjW0ZEa+bcezPRcQfznKfL+thRLwiIu6OiB81jr03Irpm/YfWkmRQqxU3AE8D5wJdwEeBzMzNwEHgysxcm5m3zXWQ2faPiPXAl6mP5n8W2A58ISLOnXHTzcA24FXAU8A/AVcA/wL4APBnEXFJZr4IXA4cahx/bWYemllDRPw88HngQ43Hcz/10OucsdtVwDuBC4B/C7x/jof1S8ATjb7sAP4Y+HngzcDrgPXALY19Z+3jXD073gl6/hvA2UA38HPAfwZ+spDjqmwGtVpxGPhXwGsy83Bj3nmxPiTmfcD9mXl/Zh7NzG8A+4BfnbHP5zLzkcw80rj/L2fm/826vwO+Drytxfv7NeDLmfmNzDwM3E59quetM/b5VGYeyswfA/dSD90TOZSZf96Y8vh/1H+hfDgzf5yZLwB/BFzd2LddfTxMPaBfl5lTmbk/M59fhOOqEAa1WvEnwOPA1xtTDR9ZxGO/BviPjT/Zn42IZ4F/Rz3Qpo3NvEFEXB4R32pMHzxLPdTXtXh/51EflQPQmO8eoz7ynfaDGT+/BKyd43gzazsXWAPsn/FYvtpYD+3r413A14BdEXEoIm6LiDMX6dgqgEGteWXmC5l5Q2a+Fng3cH1EDExvXujhjlseA+7KzHNmfJ2VmX88220iYjXwBeoj4a7MPIf69EW0WM8h6r8cpo8X1KcMvrfAx/Gy2oAfUp9y+DczHsvZmbkW5u3jS9RDftqrW7xPGqPzP8jMC6n/ZXAFsOUkH48KZFALqL9ioXHSrAPoaJygWtXYdkVEvK4Ras8BU8D0Ky/+EXjtAu7q+P3vBq6MiHdExPT99kXE+Se4fSewGngGOBIRlwP/4bjj/1xEnH2C2+8G3hURA41R5w3ABPAPC3gMs2qMzv+K+pz5vwSIiPUR8Y7Gz3P18SHgvY0evBP493Pc1TE9jIj+iLgoIjqA56lPhRT5yhidHINa036X+mjwI9TnjX/SWAfweuB/AuPACPCXmTnc2HYr8LuNP/W3t3A/x+yfmWPAe6ifWHuG+gj7v3KC52Zj3vda6oH7z8B7gb+dsf271E8WPtG4j/OOu/1jjcf359RHwFdSPzE32ULtrfhv1Kc3vhURz1Pv2/TryOfq43WNWp4FrgG+NMd9HN/zVwP3UA/pA8DfUZ8O0TIRXjhAksrmiFqSCmdQS1LhDGpJKpxBLUmFa8uHMq1bty57enraceiWvfjii5x11lmV1lAKe9FkL5rsRVMJvdi/f/8PM/Pc2ba1Jah7enrYt29fOw7dslqtRl9fX6U1lMJeNNmLJnvRVEIvIuKpE21z6kOSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIK11JQR8SHI+KRiPhORHw+Il7R7sIktcFtt8Hw8LHrhofr61WseYM6ItYD1wKbMvONQAdwdbsLk9QGl14KV13VDOvh4frypZdWW5fm1Oo1E1cBr4yIw8Aa4FD7SpLUNv39sHs3XHUVPZdfDl/5Sn25v7/qyjSHyMz5d4q4DtgB/AT4emZeM8s+24BtAF1dXRt37dq1yKUuzPj4OGvXrq20hlLYiyZ7Udezcyc9d93F6ObNjG7dWnU5lSvhedHf378/MzfNujEz5/wCfgbYA5wLnAl8CXjfXLfZuHFjVm14eLjqEophL5rsRWbu2ZO5bl0+uXlz5rp19eUVroTnBbAvT5CprZxMfDvwZGY+k5mHgS8Cbz313x+STrvpOendu+sj6cY0yMtOMKoorQT1QeAtEbEmIgIYAA60tyxJbbF377Fz0tNz1nv3VluX5jTvycTMfCAi7gEeBI4A3wY+0+7CJLXBjTe+fF1/vycTC9fSqz4y8/eA32tzLZKkWfjOREkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1Gofr3jdZC90CgxqtY9XvG6yFy8zMjbCrd+8lZGxkcrrGDw4WHkdc2n1KuTSws244jW/9Vvw6U+v3Cte24tjjIyNMHDnAJNTk3R2dDK0ZYje7t7K6pg4MsHg2GBldczHEbXaq7+/Hkwf/3j9+woNJsBezFAbrTE5NclUTjE5NUlttFZpHUc5Wmkd8zGo1V7Dw/XR480317+v5Iuo2ouf6uvpo7Ojk47ooLOjk76evkrrOIMzKq1jPk59qH1mXPH6p9flm7m8ktiLY/R29zK0ZYjaaI2+nr7Kphum69g5vJOt/VuLnPYAg1rtNNcVr1daONmLl+nt7i0iGHu7e5nYMFFELSdiUKt9vOJ1k73QKXCOWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBreVptktfnYiXxFLhDGotT8df+upEvCSWloCWgjoizomIeyLiuxFxICLK/TxACY699NWJwvr4z4iWCtXqiPqTwFcz8xeANwEH2leStEimw/rKK+GOO47ddscd9fWGtJaAeT+POiLOBn4ZeD9AZk4Ck+0tS1ok/f3wsY/B9u315UsuqYf09u1w++2GtJaEVi4ccAHwDPA3EfEmYD9wXWa+2NbKpMVy/fX179u38+Y3vhG+8516SE+vlwoXmTn3DhGbgG8Bl2XmAxHxSeD5zLz5uP22AdsAurq6Nu7atatNJbdmfHyctWvXVlpDKexF3ZuvvZZzHn6YZy+6iIc+9amqy6mcz4umEnrR39+/PzM3zboxM+f8Al4NjM5Yfhvw5blus3Hjxqza8PBw1SUUw15k5p/+aWZE/vNFF2VG1JdXOJ8XTSX0AtiXJ8jUeac+MvMHETEWEW/IzMeAAeDRxfotIrXdjDnphy65hL4HH2zOWTv9oSWg1Yvb/jYwGBGdwBPAB9pXkrSIhofhlluac9K1WjOcb7kFLr7YE4oqXktBnZkPAbPPnUilmn6d9L33vjyMr7++HtK+jlpLgO9M1PLUyptZWnlTjFQAg1rL0969rY2Up8N6797TU5d0Elqdo5aWlhtvbH3f/n6nPlQ0R9SSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS6fJ4MOD9HyihzP+4Ax6PtHD4MODVZekJcLP+pBOg8GHB9l27zZeOvwSAE899xTb7t0GwDUXXVNlaZUZGRuhNlqjr6eP3u7eqsspmkEtnQY3Dd3005Ce9tLhl7hp6KYVGdQjYyMM3DnA5NQknR2dDG0ZMqzn4NSHdBocfO7ggtYvd7XRGpNTk0zlFJNTk9RGa1WXVDSDWjoNNpy9YUHrl7u+nj46OzrpiA46Ozrp6+mruqSiGdTSabBjYAdrzlxzzLo1Z65hx8COiiqqVm93L0Nbhvh4/8ed9miBc9TSaTA9D33T0E0cfO4gG87ewI6BHStyfnpab3evAd0ig1o6Ta656JoVHcw6eU59SFLhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCtRzUEdEREd+OiPvaWdCycNttMDx87Lrh4fp6SVqghYyorwMOtKuQZeXSS+Gqq5phPTxcX7700mrrkrQktRTUEXE+8C7gs+0tZ5no74fdu+vhfMst9e+7d9fXS9ICRWbOv1PEPcCtwKuA7Zl5xSz7bAO2AXR1dW3ctWvXIpe6MOPj46xdu7bSGnp27qTnrrsY3byZ0a1bK6ujhF6Uwl402YumEnrR39+/PzM3zboxM+f8Aq4A/rLxcx9w33y32bhxY1ZteHi42gL27Mlcty7z5pvr3/fsqayUyntREHvRZC+aSugFsC9PkKmtTH1cBrw7IkaBXcCvRMTdp/77YxmbnpPevRs+9rHmNMjxJxglqQXzBnVm/k5mnp+ZPcDVwJ7MfF/bK1vK9u49dk56es56795q65K0JHkV8na48caXr+vv92SipJOyoKDOzBpQa0slkqRZ+c5ESSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1ptNzI2wq3fvJWRsZGqS5GWJC8coLYaGRth4M4BJqcm6ezoZGjLEL3dvVWXJS0pjqjVVrXRGpNTk0zlFJNTk9RGa1WXJC05BrXaqq+nj86OTjqig86OTvp6+qouSVpynPpQW/V29zK0ZYjaaI2+nj6nPaSTYFCr7Xq7ew1o6RQ49SFJhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCjdvUEdEd0QMR8SjEfFIRFx3OgqTJNW18nnUR4AbMvPBiHgVsD8ivpGZj7a5NkkSLYyoM/P7mflg4+cXgAPA+nYXpsUxMjbC4MFBrwAuLWELmqOOiB7gYuCBtlSjRTV9BfCdT+5k4M4Bw1paolq+FFdErAW+AHwoM5+fZfs2YBtAV1cXtVptsWo8KePj45XXULXBg4NMHJngKEeZODLBzuGdTGyYqLqsSvm8aLIXTaX3IjJz/p0izgTuA76WmXfMt/+mTZty3759i1DeyavVavT19VVaQ9WmR9QTRyZYvWo1Q1uGVvy1C31eNNmLphJ6ERH7M3PTbNtaedVHAH8NHGglpFWO6SuAb71gqyEtLWGtTH1cBmwGHo6IhxrrPpqZ97etKi2a3u5eJjZMGNLSEjZvUGfm/wLiNNQiSZqF70yUpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIK11JQR8Q7I+KxiHg8Ij7S7qIkSU3zBnVEdAB/AVwOXAj8ekRc2O7CTsXI2AiDBwcZGRupuhRJOmWtjKh/EXg8M5/IzElgF/Ce9pZ18kbGRhi4c4CdT+5k4M4Bw1rSkreqhX3WA2Mzlp8Gfun4nSJiG7ANoKuri1qtthj1LdjgwUEmjkxwlKNMHJlg5/BOJjZMVFJLKcbHxyv79yiNvWiyF02l96KVoG5JZn4G+AzApk2bsq+vb7EOvSCrx1YzOFYP69WrVrO1fyu93b2V1FKKWq1GVf8epbEXTfaiqfRetDL18T2ge8by+Y11Rert7mVoyxBbL9jK0JahFR/Skpa+VkbUe4HXR8QF1AP6auC9ba3qFPV29zKxYcKQlrQszBvUmXkkIj4IfA3oAHZm5iNtr0ySBLQ4R52Z9wP3t7kWSdIsfGeiJBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwkZmLf9CIZ4CnFv3AC7MO+GHFNZTCXjTZiyZ70VRCL16TmefOtqEtQV2CiNiXmZuqrqME9qLJXjTZi6bSe+HUhyQVzqCWpMIt56D+TNUFFMReNNmLJnvRVHQvlu0ctSQtF8t5RC1Jy4JBLUmFW5ZBHRHvjIjHIuLxiPhI1fVUJSK6I2I4Ih6NiEci4rqqa6pSRHRExLcj4r6qa6lSRJwTEfdExHcj4kBE9FZdU1Ui4sON/xvfiYjPR8Qrqq5pNssuqCOiA/gL4HLgQuDXI+LCaquqzBHghsy8EHgL8F9WcC8ArgMOVF1EAT4JfDUzfwF4Eyu0JxGxHrgW2JSZbwQ6gKurrWp2yy6ogV8EHs/MJzJzEtgFvKfimiqRmd/PzAcbP79A/T/k+mqrqkZEnA+8C/hs1bVUKSLOBn4Z+GuAzJzMzGcrLapaq4BXRsQqYA1wqOJ6ZrUcg3o9MDZj+WlWaDjNFBE9wMXAAxWXUpVPADcCRyuuo2oXAM8Af9OYBvpsRJxVdVFVyMzvAbcDB4HvA89l5terrWp2yzGodZyIWAt8AfhQZj5fdT2nW0RcAfxTZu6vupYCrAIuAT6dmRcDLwIr8jxORPwM9b+2LwDOA86KiPdVW9XslmNQfw/onrF8fmPdihQRZ1IP6cHM/GLV9VTkMuDdETFKfSrsVyLi7mpLqszTwNOZOf2X1T3Ug3slejvwZGY+k5mHgS8Cb624plktx6DeC7w+Ii6IiE7qJwf+tuKaKhERQX0u8kBm3lF1PVXJzN/JzPMzs4f682FPZhY5cmq3zPwBMBYRb2isGgAerbCkKh0E3hIRaxr/VwYo9MTqqqoLWGyZeSQiPgh8jfpZ3J2Z+UjFZVXlMmAz8HBEPNRY99HMvL+6klSA3wYGGwOZJ4APVFxPJTLzgYi4B3iQ+iukvk2hbyX3LeSSVLjlOPUhScuKQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIK9/8Bsi7Q+mRmA4QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C1 = [1, 4, 5, 9, 11]\n",
    "C2 = list(set(range(12)) - set(C1))\n",
    "X0C1, X1C1 = X0[C1], X1[C1]\n",
    "X0C2, X1C2 = X0[C2], X1[C2]\n",
    "plt.figure()\n",
    "plt.title('1st iteration results')\n",
    "plt.axis([-1, 9, -1, 9])\n",
    "plt.grid(True)\n",
    "plt.plot(X0C1, X1C1, 'rx')\n",
    "plt.plot(X0C2, X1C2, 'g.')\n",
    "plt.plot(4,6,'rx',ms=12.0)\n",
    "plt.plot(5,5,'g.',ms=12.0);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们重新计算两个类的重心，把重心移动到新位置，并重新计算各个样本与新重心的距离，并根据距离远近为样本重新归类。结果如下表所示：\n",
    "\n",
    "![data_1](images/data_1.png)\n",
    "\n",
    "画图结果如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUz0lEQVR4nO3dfZBddX3H8feXDQmEKKjBpULCpg+KjI61CeLCaHcbxpFC1Zm2FMWgpk5aWiw6WitSFLXUqe1YdVQciqEFtmYyaFtEWtGwa32IlASYIgRba0ICgsQHHjbobh6+/eOe9S7hZvcu2Zvz2933a2Zn99577jnf8927nz33d+69v8hMJEnlOqzuAiRJEzOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1Br2kTEmyPiGwe4bWlEDEdE16Gua1wN50XEzXVt/2BFRE9EZETMq7sWHVoG9RwWEQsi4rMRcV9EPB4Rd0bEmZ3YVmZuz8xFmbm32vZQRLy1E9uq1v+UUMvMgcx8Vae2eah1uocqh0E9t80DdgC/CRwN/CWwPiJ66iyqHXUemU/GI15NN4N6DsvMXZl5WWZuy8x9mXkjsBVYDhARfRFxf0S8MyIejogHI+ItY/ePiOdExA0R8VhE/BfwKwfa1vgj3Ii4HHgF8MlqOOST1TInRcRXIuInEfHdiDhn3P3/MSKuiIibImIX0B8RZ0XEHdX2d0TEZeM2+Z/V90eqbfTuPzQTEadFxG0R8Wj1/bRxtw1FxIci4pvVs42bI2LxAfZtrE9/EREPAVdHxGER8Z6I+L+I+HFErI+IZ1fLHxER11XXP1Jtu7u6bVtEnDFu3ZdFxHUttvmUHkbD31e/q8ci4q6IeNGBfieaOQxq/UIVFs8H7h539XE0jraPB/4Q+FREPKu67VPAz4FfAlZXX5PKzEuArwMXVsMhF0bEUcBXgH8GngucC3w6Ik4ed9c3AJcDzwC+AewCzgeOAc4CLoiI11XLvrL6fky1jY377euzgS8BnwCeA3wU+FJEPGe/7b2lqmc+8K4Jdus44NnAicAa4G3A62g8W3ke8FMa/QJ4E42eLqm2/cfAzyZY91O06iHwqmq/n1+t/xzgx1NZr8pkUAuAiDgcGAD+KTPvHXfTbuCDmbk7M28ChoEXVEMPvwu8rzoy/w7wTwdRwtnAtsy8OjP3ZOYdwOeB3x+3zL9l5jero/+fZ+ZQZt5VXf5v4HM0grEdZwH/m5nXVtv7HHAv8Dvjlrk6M/8nM38GrAd+fYL17QPen5kj1fJ/DFySmfdn5ghwGfB71bDIbhoB/auZuTczN2fmY23WPZHdNP6JnQREZm7JzAenYb2qmUEtIuIw4FpgFLhwv5t/nJl7xl1+AlgEHEtzjHvMfQdRxonAqdVQwCMR8QhwHo0j1THjt0VEnBoRgxGxMyIepRGOLYcnWnhei3rvo/HMYcxD434e2+8D2ZmZPx93+UTgX8btyxZgL9BNo9dfBtZFxA8i4iPVP8qDkpm3AJ+kceT+cERcGRHPPNj1qn4G9RwXEQF8lkaA/G5m7m7zrjuBPTSevo9ZOoVN7/+xjTuAr2XmMeO+FmXmBRPc55+BG4AlmXk08BkgDrDs/n5AI0zHWwo80PYePFmr/Tlzv/05IjMfqJ6dfCAzTwZOo/Fs4vzqfruAhePWcxwH9pR9zMxPZOZy4GQaQyB//jT3RwUxqHUF8ELgd6qn7G2pXmb3BeCyiFhYjSW/aQrb/SHwy+Mu3wg8PyJWRcTh1dcpEfHCCdbxDOAnmfnziHgZjTHlMTtpDEf8cst7wk3V9t5QneD8AxrhduMU9mEinwEuj4gTASLi2Ih4bfVzf0S8uBo+eozGkMW+6n53AudW+78C+L0JtvGkHlb9OrU6Ot9F4/zBvgPdWTOHQT2HVSHyRzTGXh+qXj0wHBHntbmKC2kMBzwE/CNw9RQ2/3EaY7Y/jYhPZObjNE6GnUvjaPch4G+ABROs40+AD0bE48D7aIwjA5CZT9A48fjNavjh5ePvmJk/pnEk+04aJ9zeDZydmT+awj5Mtn83ADdX9X0bOLW67TjgehohvQX4Go3hEIBLabx65qfAB2g8a5hoG7/oIfBM4B+q+95X7dffTtP+qEbhxAGSVDaPqCWpcAa1JBXOoJakwhnUklS4jnx4zOLFi7Onp6cTq27brl27OOqoo2qtoRT2osleNNmLphJ6sXnz5h9l5rGtbutIUPf09LBp06ZOrLptQ0ND9PX11VpDKexFk71oshdNJfQiIg74zl6HPiSpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwbQV1RLwjIu6OiO9ExOci4ohOFyapAz7yERgcfPJ1g4ON61WsSYM6Io4H/gxYkZkvArqAcztdmKQOOOUUOOecZlgPDjYun3JKvXVpQu3OmTgPODIidgMLgR90riRJHdPfD+vXwznn0HPmmfDv/9643N9fd2WaQGTm5AtFXARcDvwMuDkzz2uxzBpgDUB3d/fydevWTXOpUzM8PMyiRYtqraEU9qLJXjT0rF1Lz7XXsm3VKratXl13ObUr4XHR39+/OTNXtLwxMyf8Ap4F3AIcCxwO/Cvwxonus3z58qzb4OBg3SUUw1402YvMvOWWzMWLc+uqVZmLFzcuz3ElPC6ATXmATG3nZOIZwNbM3JmZu4EvAKcd/P8PSYfc2Jj0+vWNI+lqGOQpJxhVlHaCejvw8ohYGBEBrAS2dLYsSR1x221PHpMeG7O+7bZ669KEJj2ZmJm3RsT1wO3AHuAO4MpOFyapA9797qde19/vycTCtfWqj8x8P/D+DtciSWrBdyZKUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqBW5zjjdZO9aCqlF6XU0QaDWp3jjNdN9qKplF6UUkcb2p2FXJq6cTNec8EFcMUVc3fGa3vRVEovZtCM7B5Rq7P6+xt/jB/6UON7gX8Eh4y9aCqlF1UdPddeW/TvxKBWZw0ONo6YLr208X0uT6JqL5pK6UVVx7ZVq4r+nRjU6pxxM17zwQ/O7Rmv7UVTKb2YQTOyG9TqHGe8brIXTaX0opQ62uDJRHWOM1432YumUnpRSh1t8IhakgpnUEtS4QxqzVyt3ll2IIW+40xqh0GtmWv/d5YdSMHvOJPaYVBr5hr/DrcDhfX4l4IVeJJIaodBrZltorA2pDVLGNSa+VqFtSGtWcTXUWt2KOWDfqQO8Ihas0cpH/QjTTODWrNHKR/0I00zg1qzQykf9CN1gEGtma/VicN2XronzRAGtWa2iV7dYVhrlmgrqCPimIi4PiLujYgtEdHb6cKkSbXzEjzDWrNAu0fUHwf+IzNPAl4CbOlcSVKb9v884YmWu/jiJ3/OsJ/9oRlk0tdRR8TRwCuBNwNk5igw2tmypDa0+jzhVsY+E2T9+sbl8Ufi0gzQzhtelgE7gasj4iXAZuCizNzV0cqk6TKDZpuWWonMnHiBiBXAt4HTM/PWiPg48FhmXrrfcmuANQDd3d3L161b16GS2zM8PMyiRYtqraEU9qKhZ+1aeq69lm2rVjXmyJvjfFw0ldCL/v7+zZm5ouWNmTnhF3AcsG3c5VcAX5roPsuXL8+6DQ4O1l1CMexFZt5yS+bixbl11arMxYsbl+c4HxdNJfQC2JQHyNRJTyZm5kPAjoh4QXXVSuCeafgHIh0aM2i2aamVdl/18TZgICL+G/h14K87VpE03WbQbNNSK219el5m3gm0HjuRSjeDZpuWWvGdiZJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1dAht3LGRD3/9w2zcsbHuUmpnL9rX1udRSzp4G3dsZOU1KxndO8r8rvlsOH8DvUt66y6rFvZiajyilg6RoW1DjO4dZW/uZXTvKEPbhuouqTb2YmoMaukQ6evpY37XfLqii/ld8+nr6au7pNrYi6lx6EM6RHqX9LLh/A0MbRuir6dvTj/VtxdTY1BLh1Dvkl5DqWIv2ufQhyQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUzRw1wA9H+vhsA8cRs/Hehi4a6DukjTL+el50hQM3DXAmi+u4YndTwBw36P3seaLawA478Xn1VmaZjGPqKUpuGTDJb8I6TFP7H6CSzZcUlNFmgvaDuqI6IqIOyLixk4WJJVs+6Pbp3S9NB2mckR9EbClU4XMRs6yPPssPXrplK6XpkNbQR0RJwBnAVd1tpzZY2yW5UsHL2XlNSsN61ni8pWXs/DwhU+6buHhC7l85eU1VaS5IDJz8oUirgc+DDwDeFdmnt1imTXAGoDu7u7l69atm+ZSp2Z4eJhFixbVtv2B7QOs3bqWfezjMA5j9bLVnLe0npNNdfeiJNPRi6/+8KtctfUqHh55mOcueC5vXfZWzug+Y5oqPHR8XDSV0Iv+/v7Nmbmi5Y2ZOeEXcDbw6ernPuDGye6zfPnyrNvg4GCt2//W9m/lkX91ZHZ9oCuP/Ksj81vbv1VbLXX3oiT2osleNJXQC2BTHiBT23l53unAayLit4EjgGdGxHWZ+cZp+CcyaznLsqTpMmlQZ+bFwMUAEdFHY+jDkG6DsyxLmg6+jlqSCjeldyZm5hAw1JFKJEkteUQtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUKvjnI1dOjhT+jxqaarGZmMf3TvK/K75bDh/g7PeSFPkEbU6amjbEKN7R9mbexndO8rQtqG6S5JmHINaHdXX08f8rvl0RRfzu+bT19NXd0nSjOPQhzrK2dilg2dQq+OcjV06OA59SFLhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCTRrUEbEkIgYj4p6IuDsiLjoUhUmSGtr5POo9wDsz8/aIeAawOSK+kpn3dLg2SRJtHFFn5oOZeXv18+PAFuD4Them6bFxx0YGtg84A7g0g01pjDoieoCXArd2pBpNq7EZwNduXcvKa1Ya1tIM1fZUXBGxCPg88PbMfKzF7WuANQDd3d0MDQ1NV41Py/DwcO011G1g+wAje0bYxz5G9oywdnAtI0tH6i6rVj4umuxFU+m9iMycfKGIw4EbgS9n5kcnW37FihW5adOmaSjv6RsaGqKvr6/WGuo2dkQ9smeEBfMWsOH8DXN+7kIfF032oqmEXkTE5sxc0eq2dl71EcBngS3thLTKMTYD+Oplqw1paQZrZ+jjdGAVcFdE3Fld997MvKljVWna9C7pZWTpiCEtzWCTBnVmfgOIQ1CLJKkF35koSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVrq2gjohXR8R3I+J7EfGeThclSWqaNKgjogv4FHAmcDLw+og4udOFHYyNOzYysH2AjTs21l2KJB20do6oXwZ8LzO/n5mjwDrgtZ0t6+nbuGMjK69Zydqta1l5zUrDWtKMN6+NZY4Hdoy7fD9w6v4LRcQaYA1Ad3c3Q0ND01HflA1sH2Bkzwj72MfInhHWDq5lZOlILbWUYnh4uLbfR2nsRZO9aCq9F+0EdVsy80rgSoAVK1ZkX1/fdK16ShbsWMDAjkZYL5i3gNX9q+ld0ltLLaUYGhqirt9HaexFk71oKr0X7Qx9PAAsGXf5hOq6IvUu6WXD+RtYvWw1G87fMOdDWtLM184R9W3Ar0XEMhoBfS7who5WdZB6l/QysnTEkJY0K0wa1Jm5JyIuBL4MdAFrM/PujlcmSQLaHKPOzJuAmzpciySpBd+ZKEmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqXGTm9K80Yidw37SveGoWAz+quYZS2Isme9FkL5pK6MWJmXlsqxs6EtQliIhNmbmi7jpKYC+a7EWTvWgqvRcOfUhS4QxqSSrcbA7qK+suoCD2osleNNmLpqJ7MWvHqCVptpjNR9SSNCsY1JJUuFkZ1BHx6oj4bkR8LyLeU3c9dYmIJRExGBH3RMTdEXFR3TXVKSK6IuKOiLix7lrqFBHHRMT1EXFvRGyJiN66a6pLRLyj+tv4TkR8LiKOqLumVmZdUEdEF/Ap4EzgZOD1EXFyvVXVZg/wzsw8GXg58KdzuBcAFwFb6i6iAB8H/iMzTwJewhztSUQcD/wZsCIzXwR0AefWW1Vrsy6ogZcB38vM72fmKLAOeG3NNdUiMx/MzNurnx+n8Qd5fL1V1SMiTgDOAq6qu5Y6RcTRwCuBzwJk5mhmPlJrUfWaBxwZEfOAhcAPaq6npdkY1McDO8Zdvp85Gk7jRUQP8FLg1ppLqcvHgHcD+2quo27LgJ3A1dUw0FURcVTdRdUhMx8A/g7YDjwIPJqZN9dbVWuzMai1n4hYBHweeHtmPlZ3PYdaRJwNPJyZm+uupQDzgN8ArsjMlwK7gDl5HicinkXj2fYy4HnAURHxxnqram02BvUDwJJxl0+orpuTIuJwGiE9kJlfqLuempwOvCYittEYCvutiLiu3pJqcz9wf2aOPbO6nkZwz0VnAFszc2dm7ga+AJxWc00tzcagvg34tYhYFhHzaZwcuKHmmmoREUFjLHJLZn607nrqkpkXZ+YJmdlD4/FwS2YWeeTUaZn5ELAjIl5QXbUSuKfGkuq0HXh5RCys/lZWUuiJ1Xl1FzDdMnNPRFwIfJnGWdy1mXl3zWXV5XRgFXBXRNxZXffezLypvpJUgLcBA9WBzPeBt9RcTy0y89aIuB64ncYrpO6g0LeS+xZySSrcbBz6kKRZxaCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1Jhft/uEEZ1c5o3CIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C1 = [1, 2, 4, 8, 9, 11]\n",
    "C2 = list(set(range(12)) - set(C1))\n",
    "X0C1, X1C1 = X0[C1], X1[C1]\n",
    "X0C2, X1C2 = X0[C2], X1[C2]\n",
    "plt.figure()\n",
    "plt.title('2nd iteration results')\n",
    "plt.axis([-1, 9, -1, 9])\n",
    "plt.grid(True)\n",
    "plt.plot(X0C1, X1C1, 'rx')\n",
    "plt.plot(X0C2, X1C2, 'g.')\n",
    "plt.plot(3.8,6.4,'rx',ms=12.0)\n",
    "plt.plot(4.57,4.14,'g.',ms=12.0);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们再重复一次上面的做法，把重心移动到新位置，并重新计算各个样本与新重心的距离，并根据距离远近为样本重新归类。结果如下表所示：\n",
    "![data_2](images/data_2.png)\n",
    "\n",
    "画图结果如下：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUlklEQVR4nO3dfZBddX3H8feXPEkIgp1gLJCw+ARSHLUEJFLbXeOM4gN2OlOKYhgbnbS0KlhtFCgVRcRaR8ERaaNEBbemDKKjCGIn7E5ljAgBWh4CHUpCNggFHxAWcEPIt3/cE+4l7m7usnv3/Hb3/Zq5s3vvOfec7/3m5rO/+7u79xeZiSSpXHvVXYAkaXQGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqjVtEdEfEtlG2D0bEiyezpt3O//qIuLuu80+EiMiIeGnddageBrWIiG9GxAMR8WhE/E9EvG8ij5+ZCzLz3upcX4+IT03k8Xe3e6hl5o8z87BOnnMyTUYPVRaDWgDnA12Z+XzgBOBTEXHUcDtGxOxJrayw84+m5No0tRnUIjPvyMyhXVery0ugOa0RER+NiAeBr0XE3tWo7tcRcSdw9GjH3zXCjYhVwMnA6mo65PvV9gMj4tsR8XBEbI6ID7bc95yIuKIa9T8KvCcijomIDRHxSPVK4EsRMbfa/z+ru/5XdY6/2H1qJiJeERH91f3viIgTWrZ9PSIuiogfRMRjEXFDRLxkhMfVVT2290bEVuC66vaVEbGp6s+1EXFIdXtExBci4qHq1cttEXFkta2/9ZVMRLwnIq4f5pwj9fCjEXF/VfPdEbF8tH8TTTGZ6cULwJeBJ2iE9M3Agur2bmAH8E/APGBv4DPAj4HfAxYDtwPbRjl2Ai+tvv868KmWbXsBG4F/BOYCLwbuBd5UbT8HeAr402rfvYGjgGOB2UAXsAk4fbjztTyGbdX3c4B7gDOr870BeAw4rKW+XwLHVMfvBdaN8Li6qnNdCuxT1faO6vivqO7/D8BPqv3fVD3W/YGo9vn9als/8L6WY78HuL7NHh4GDAAHttT1krqfU14m7uKIWgBk5t8A+wKvB64Ehlo27wQ+nplDmfkkcCJwXmb+KjMHgC+O49RHAwdk5iczc3s25rK/ApzUss+GzPxuZu7MzCczc2Nm/jQzd2TmFuBfgT9p83zHAguAz1Tnuw64Cnhnyz7fycyfZeYOGkH96j0c85zMfLzqzV8D52fmpur+nwZeXY2qn6LR48OBqPZ5oM26R/M0jR+iR0TEnMzckpn/OwHHVSEMaj0jM5/OzOuBg4FTWzY9nJm/bbl+II0R3C73jeO0hwAHVtMQj0TEIzRGu4ta9mk9FxHx8oi4KiIerKZDPg0sbPN8BwIDmbmz5bb7gINarj/Y8v0TNIJ9NK31HQJc2PJYfkVj9HxQ9UPhS8BFwEMRsSYint9m3SPKzHuA02m8+ngoItZFxIHjPa7KYVBrOLOp5qgru3/E4gM0pjx2WTKGY+9+rAFgc2bu33LZNzPfMsp9LgbuAl6WjTdAz6QRhu34ObA4Ilqf+0uA+9t/CL+jtb4B4K92ezx7Z+ZPADLzi5l5FHAE8HLg76v7PQ7MbznOi9o8H9Vx/y0z/4jGD4qkMVWlacKgnuEi4oURcVJELIiIWRHxJhrTAOtHudvlwBkR8YKIOBj4wBhO+X805qF3+RnwWPVm2N5VDUdGxGhvUO4LPAoMRsThPHv0P9w5Wt1AY5S8OiLmREQ38HZg3Rgew2j+hUZv/gAgIvaLiD+vvj86Il4bEXNoBPNvaUwrAdwK/FlEzK9+tfC9o5zjWY8vIg6LiDdExLzqmE+2HFfTgEGtpBF024BfA5+j8cbc90a5zydoTBdsBn4EXDaG811CYy71kYj4bmY+DbyNxjzwZuAXwFeB/UY5xkeAd9F4E/ArwL/vtv0c4BvVOU5s3ZCZ22kE8/HVub4MnJKZd43hMYwoM79DYzS7rpqWub06F8Dzq3p/TaN/vwT+udr2BWA7jRD+Bo258ZE8q4c05qc/Uz2eB4EXAmdMxONRGSLThQMkqWSOqCWpcAa1JBXOoJakwhnUklS4jnyIzMKFC7Orq6sTh27b448/zj777FNrDaWwF032osleNJXQi40bN/4iMw8YbltHgrqrq4ubbrqpE4duW39/P93d3bXWUAp70WQvmuxFUwm9iIgR/8LXqQ9JKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqXFtBHREfiog7IuL2iPhWRDyv04VJ6oDPfhb6+p59W19f43YVa49BHREHAR8ElmbmkcAs4KROFyapA44+Gk48sRnWfX2N60cfXW9dGlW7aybOBvaOiKeA+cDPO1eSpI7p6YHLL4cTT6Tr+OPhmmsa13t66q5Mo4jM3PNOEacB5wFPAj/KzJOH2WcVsApg0aJFR61bt26CSx2bwcFBFixYUGsNpbAXTfaioWvtWrouu4wtK1awZeXKusupXQnPi56eno2ZuXTYjZk56gV4AXAdcAAwB/gu8O7R7nPUUUdl3fr6+uouoRj2osleZOZ112UuXJibV6zIXLiwcX2GK+F5AdyUI2RqO28mvhHYnJkPZ+ZTwJXA68b/80PSpNs1J3355Y2RdDUN8jtvMKoo7QT1VuDYiJgfEQEsBzZ1tixJHXHjjc+ek941Z33jjfXWpVHt8c3EzLwhIq4AbgZ2ALcAazpdmKQOWL36d2/r6fHNxMK19Vsfmflx4OMdrkWSNAz/MlGSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqqWTDLZ01EpfUmrYMaqlkuy+dNRKX1JrWDGqpZC1LZ40Y1i2fMe2n4E1PBrU6xxWvm8bTi9HCeiqGdCnPi1LqaINBrc5xxeum8fZiuLCeiiEN5TwvSqmjHSOt0TWei2smlqXWXlTr8+XZZxexPt+U78UE9nPK92IC6yhh/UjGuWai9Nz19MCpp8K55za+TqWR30SbiF5Ml36W8jiqOrouu6zofhrU6qy+Prj4Yjj77MbXmbyI6kT0Yrr0s5THUdWxZcWKsvs50lB7PBenPspSWy92vbzd9XJy9+s1mNK9mOB+TuleTHAdfX19tT8/cepDtXDF66bx9mK4Nw7b+dW9EpXyvCiljnaMlODjuTiiLou9aJqSvdjTSO85jgSnZC86pIRe4IhamqLa+RW8qTqyVtsMaqlku788H0nJL9s1brPrLkDSKFavbn/fnp5if71M4+OIWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1Lh2grqiNg/Iq6IiLsiYlNELOt0YZKkhnZH1BcCP8zMw4FXAZs6V5I0wabQatPScPYY1BGxH/DHwCUAmbk9Mx/pcF3SxJlKq01Lw2jn0/MOBR4GvhYRrwI2Aqdl5uMdrUyaKC2f19x1/PFwzTXtfXSoVIhoLCwwyg4RS4GfAsdl5g0RcSHwaGaevdt+q4BVAIsWLTpq3bp1HSq5PYODgyxYsKDWGkphLxq61q6l67LL2LJiBVtWrqy7nNr5vGgqoRc9PT0bM3PpsBtHWvpl1wV4EbCl5frrgR+Mdh+X4iqLvchnlqvavGJF7QvslsLnRVMJvWA8S3Fl5oPAQEQcVt20HLhzAn6ASJOjZTmrLStXumyVppx2f+vjA0BvRPw38Grg0x2rSJpoU2m1aWkYbS3FlZm3AsPPnUilG245K5et0hTiXyZKUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWJoML7DbZizEzqKXJ4AK7TfZizNr6PGpJ49SywC6nngoXXzxzF9i1F2PmiFqaLD09jWA699zG15kcTPZiTAxqabL09TVGj2ef3fg6k9dstBdjYlBLk6FlgV0++cmZvcCuvRgzg1qaDC6w22Qvxsw3E6XJ4AK7TfZizBxRS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCtR3UETErIm6JiKs6WZAk6dnGMqI+DdjUqUKmow0DGzj/x+ezYWBD3aVImsLaWjggIg4G3gqcB/xdRyuaJjYMbGD5pcvZ/vR25s6ay/pT1rNs8bK6y5I0BbW7wssFwGpg35F2iIhVwCqARYsW0d/fP97axmVwcLDWGnq39jK0Y4id7GRoxxBr+9YytGSollrq7kVJ7EWTvWgqvRd7DOqIeBvwUGZujIjukfbLzDXAGoClS5dmd/eIu06K/v5+6qxh3sA8egd6nxlRr+xZWduIuu5elMReNNmLptJ70c6I+jjghIh4C/A84PkR8c3MfHdnS5vali1exvpT1tO/pZ/urm6nPSQ9Z3sM6sw8AzgDoBpRf8SQbs+yxcsMaEnj5u9RS1Lh2n0zEYDM7Af6O1KJJGlYjqglqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEManWcq7FL4zOmz6OWxsrV2KXxc0Stjurf0s/2p7fzdD7N9qe307+lv+6SpCnHoJ6hem/rpeuCLvb6xF50XdBF7229HTlPd1c3c2fNZVbMYu6suXR3dXfkPNJ05tTHDNR7Wy+rvr+KJ556AoD7fnMfq76/CoCTX3nyhJ7L1dil8TOoZ6Cz1p/1TEjv8sRTT3DW+rMmPKjB1dil8XLqYwba+putY7pdUr0M6hloyX5LxnS7pHoZ1DPQecvPY/6c+c+6bf6c+Zy3/LyaKpI0GoN6Bjr5lSez5u1rOGS/QwiCQ/Y7hDVvX9OR+WlJ4+ebiTPUya882WCWpghH1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVLg9BnVELI6Ivoi4MyLuiIjTJqMwSVJDO39CvgP4cGbeHBH7Ahsj4j8y884O1yZJoo0RdWY+kJk3V98/BmwCDup0YZoYGwY20Lu11xXApSlsTHPUEdEFvAa4oSPVaELtWgF87ea1LL90uWEtTVFtf3peRCwAvg2cnpmPDrN9FbAKYNGiRfT3909Ujc/J4OBg7TXUrXdrL0M7htjJToZ2DLG2by1DS4bqLqtWPi+a7EVT6b2IzNzzThFzgKuAazPz83vaf+nSpXnTTTdNQHnPXX9/P93d3bXWULddI+qhHUPMmz2P9aesn/FrF/q8aLIXTSX0IiI2ZubS4ba181sfAVwCbGonpFWOXSuArzx0pSEtTWHtTH0cB6wAbouIW6vbzszMqztWlSbMssXLGFoyZEhLU9gegzozrwdiEmqRJA3Dv0yUpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFM6glqXAGtSQVzqCWpMIZ1JJUOINakgpnUEtS4QxqSSqcQS1JhTOoJalwBrUkFc6glqTCGdSSVDiDWpIK11ZQR8SbI+LuiLgnIj7W6aIkSU17DOqImAVcBBwPHAG8MyKO6HRh47FhYAO9W3vZMLCh7lIkadzaGVEfA9yTmfdm5nZgHfCOzpb13G0Y2MDyS5ezdvNall+63LCWNOXNbmOfg4CBluvbgNfuvlNErAJWASxatIj+/v6JqG/Merf2MrRjiJ3sZGjHEGv71jK0ZKiWWkoxODhY279HaexFk71oKr0X7QR1WzJzDbAGYOnSpdnd3T1Rhx6TeQPz6B1ohPW82fNY2bOSZYuX1VJLKfr7+6nr36M09qLJXjSV3ot2pj7uBxa3XD+4uq1IyxYvY/0p61l56ErWn7J+xoe0pKmvnRH1jcDLIuJQGgF9EvCujlY1TssWL2NoyZAhLWla2GNQZ+aOiHg/cC0wC1ibmXd0vDJJEtDmHHVmXg1c3eFaJEnD8C8TJalwBrUkFc6glqTCGdSSVDiDWpIKZ1BLUuEMakkqnEEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCmdQS1LhDGpJKpxBLUmFi8yc+INGPAzcN+EHHpuFwC9qrqEU9qLJXjTZi6YSenFIZh4w3IaOBHUJIuKmzFxadx0lsBdN9qLJXjSV3gunPiSpcAa1JBVuOgf1mroLKIi9aLIXTfaiqeheTNs5akmaLqbziFqSpgWDWpIKNy2DOiLeHBF3R8Q9EfGxuuupS0Qsjoi+iLgzIu6IiNPqrqlOETErIm6JiKvqrqVOEbF/RFwREXdFxKaIWFZ3TXWJiA9V/zduj4hvRcTz6q5pONMuqCNiFnARcDxwBPDOiDii3qpqswP4cGYeARwL/O0M7gXAacCmuosowIXADzPzcOBVzNCeRMRBwAeBpZl5JDALOKneqoY37YIaOAa4JzPvzcztwDrgHTXXVIvMfCAzb66+f4zGf8iD6q2qHhFxMPBW4Kt111KniNgP+GPgEoDM3J6Zj9RaVL1mA3tHxGxgPvDzmusZ1nQM6oOAgZbr25ih4dQqIrqA1wA31FxKXS4AVgM7a66jbocCDwNfq6aBvhoR+9RdVB0y837gc8BW4AHgN5n5o3qrGt50DGrtJiIWAN8GTs/MR+uuZ7JFxNuAhzJzY921FGA28IfAxZn5GuBxYEa+jxMRL6DxavtQ4EBgn4h4d71VDW86BvX9wOKW6wdXt81IETGHRkj3ZuaVdddTk+OAEyJiC42psDdExDfrLak224BtmbnrldUVNIJ7JnojsDkzH87Mp4ArgdfVXNOwpmNQ3wi8LCIOjYi5NN4c+F7NNdUiIoLGXOSmzPx83fXUJTPPyMyDM7OLxvPhuswscuTUaZn5IDAQEYdVNy0H7qyxpDptBY6NiPnV/5XlFPrG6uy6C5hombkjIt4PXEvjXdy1mXlHzWXV5ThgBXBbRNxa3XZmZl5dX0kqwAeA3mogcy/wlzXXU4vMvCEirgBupvEbUrdQ6J+S+yfkklS46Tj1IUnTikEtSYUzqCWpcAa1JBXOoJakwhnUklQ4g1qSCvf/X4HY9SMyqPMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C1 = [0, 1, 2, 4, 8, 9, 10, 11]\n",
    "C2 = list(set(range(12)) - set(C1))\n",
    "X0C1, X1C1 = X0[C1], X1[C1]\n",
    "X0C2, X1C2 = X0[C2], X1[C2]\n",
    "plt.figure()\n",
    "plt.title('3rd iteration results')\n",
    "plt.axis([-1, 9, -1, 9])\n",
    "plt.grid(True)\n",
    "plt.plot(X0C1, X1C1, 'rx')\n",
    "plt.plot(X0C2, X1C2, 'g.')\n",
    "plt.plot(5.5,7.0,'rx',ms=12.0)\n",
    "plt.plot(2.2,2.8,'g.',ms=12.0);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再重复上面的方法就会发现类的重心不变了，k-Means会在条件满足的时候停止重复聚类过程。通常，条件是前后两次迭代的成本函数值的差达到了限定值，或者是前后两次迭代的重心位置变化达到了限定值。如果这些停止条件足够小，k-Means就能找到最优解。不过这个最优解不一定是全局最优解。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Program"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {