examples/getting-started-ja.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started（日本語）\n",
    "\n",
    "## このノートブックについて\n",
    "\n",
    "### 目的\n",
    "\n",
    "SSH Kernelの利用に最低限必要な設定と、使い方を述べます。\n",
    "\n",
    "### ノートブックの構成\n",
    "\n",
    "* 説明用のSSHデーモンをDockerコンテナとして作成し、\n",
    "  SSH Kernelの利用に必要な接続先ユーザ名やポートなどの設定・検証手順を述べます\n",
    "* SSH Kernel独自の記法を述べ、リモートコマンド実行する手順を述べます\n",
    "\n",
    "### ノートブック作成環境\n",
    "\n",
    "* OS: Ubuntu 16.04\n",
    "* Docker: 18.06.1-ce\n",
    "* Python: 3.6.2\n",
    "* Jupyter notebook: 5.7.0\n",
    "* SSH Kernel: 0.6.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:08:29.786270Z",
     "start_time": "2018-12-06T01:08:29.779027Z"
    }
   },
   "source": [
    "## 説明用SSH接続先デーモンの起動\n",
    "\n",
    "説明用のsshdコンテナを作成\n",
    "\n",
    "* host: localhost\n",
    "* port: 10022\n",
    "* login: root\n",
    "* password: root\n",
    "\n",
    "注：localhost以外のホストで実行する場合や、ポート番号を変更する場合は、次章の説明を読み替えてください"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:08:44.877006Z",
     "start_time": "2018-12-06T01:08:41.777447Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "18.04: Pulling from rastasheep/ubuntu-sshd\n",
      "Digest: sha256:1df808ccf95c13b8e62548ad434829bc28ee701a65624e5c0440fc24482e4a62\n",
      "Status: Image is up to date for rastasheep/ubuntu-sshd:18.04\n"
     ]
    }
   ],
   "source": [
    "! docker pull rastasheep/ubuntu-sshd:18.04"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T00:58:42.266620Z",
     "start_time": "2018-12-06T00:58:41.550025Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "41b562ffa1b35b2aea62255d7e5645bc4bb0afd7a725c06cccf3c52dcafcb1e5\r\n"
     ]
    }
   ],
   "source": [
    "!docker run -d --rm -p 10022:22 rastasheep/ubuntu-sshd:18.04"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T00:59:05.200178Z",
     "start_time": "2018-12-06T00:59:05.000565Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "41b562ffa1b3        rastasheep/ubuntu-sshd:18.04     \"/usr/sbin/sshd -D\"      24 seconds ago      Up 22 seconds       0.0.0.0:10022->22/tcp                                lucid_roentgen\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps |grep ubuntu-sshd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SSHの設定"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:13:31.821040Z",
     "start_time": "2018-12-06T02:13:31.814176Z"
    }
   },
   "source": [
    "### 接続先ユーザ名・ポート・公開鍵などの設定と確認"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 設定ファイルの作成\n",
    "\n",
    "ターミナルでの作業：\n",
    "\n",
    "* 次の内容の `~/.ssh/config` ファイルを作成します"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:14:29.892837Z",
     "start_time": "2018-12-06T01:14:29.724071Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Host test\n",
      "  Hostname localhost\n",
      "  User root\n",
      "  Port 10022\n",
      "  IdentityFile /tmp/id_rsa_test\n"
     ]
    }
   ],
   "source": [
    "!head -n5 ~/.ssh/config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 公開鍵ペアの作成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T00:59:15.307416Z",
     "start_time": "2018-12-06T00:59:15.302808Z"
    }
   },
   "source": [
    "ターミナルでの作業：\n",
    "\n",
    "* パスフレーズ無し秘密鍵作成\n",
    "\n",
    "```\n",
    "ssh-keygen -f /tmp/id_rsa_test -t rsa -N ''\n",
    "\n",
    "```\n",
    "\n",
    "* ホスト鍵記憶、パスワードログインの検証\n",
    "\n",
    "```\n",
    "% ssh test\n",
    "The authenticity of host '[localhost]:10022 ([::1]:10022)' can't be established.\n",
    "ECDSA key fingerprint is SHA256:Rath9QRSP1hKeFkIGwL1c1WUV+haHdJrxTyilRrRNnE.\n",
    "Are you sure you want to continue connecting (yes/no)? yes\n",
    "Warning: Permanently added '[localhost]:10022' (ECDSA) to the list of known hosts.\n",
    "root@localhost's password: \n",
    "    初期パスワード'root'と入力\n",
    "root@41b562ffa1b3:~#\n",
    "root@41b562ffa1b3:~# exit\n",
    "logout\n",
    "Connection to localhost closed.\n",
    "```\n",
    "\n",
    "* 公開鍵の登録\n",
    "\n",
    "```\n",
    "% ssh-copy-id -i /tmp/id_rsa_test test\n",
    "/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: \"/tmp/id_rsa_test.pub\"\n",
    "/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed\n",
    "/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys\n",
    "root@localhost's password:\n",
    "\n",
    "Number of key(s) added: 1\n",
    "\n",
    "Now try logging into the machine, with:   \"ssh 'test'\"\n",
    "and check to make sure that only the key(s) you wanted were added.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 設定の検証"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 次のセルの結果が uid=0(root) gid=0(root) groups=0(root) であればOK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:15:10.090387Z",
     "start_time": "2018-12-06T01:15:09.794085Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uid=0(root) gid=0(root) groups=0(root)\n"
     ]
    }
   ],
   "source": [
    "!ssh test id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SSH Kernelの使い方"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:05:00.511032Z",
     "start_time": "2018-12-06T02:05:00.505163Z"
    }
   },
   "source": [
    "### 概要\n",
    "\n",
    "SSH Kernel独自の記法 `%login {host}` 以降のセルを、リモートで実行します。\n",
    "\n",
    "リモート実行を終了するには `%logout`を実行します"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:15:12.158571Z",
     "start_time": "2018-12-06T01:15:11.956774Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ssh] Login to test...\n",
      "[DEBUG] host=test hostname=localhost other_conf={'user': 'root', 'port': 10022, 'keyfile': ['/tmp/id_rsa_test']}\n",
      "[ssh] Successfully logged in.\n"
     ]
    }
   ],
   "source": [
    "%login test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:15:17.854995Z",
     "start_time": "2018-12-06T01:15:17.756438Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] host = localhost\n",
      "uid=0(root) gid=0(root) groups=0(root)\n"
     ]
    }
   ],
   "source": [
    "id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:48:50.646792Z",
     "start_time": "2018-12-06T01:48:50.589277Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ssh] Successfully logged out.\n"
     ]
    }
   ],
   "source": [
    "%logout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:59:53.013743Z",
     "start_time": "2018-12-06T01:59:53.008178Z"
    }
   },
   "source": [
    "### 独自の記法\n",
    "\n",
    "#### %login記法\n",
    "\n",
    "`%login {host}` は、対象ホストとのSSHコネクションを生成する、SSH Kernel独自の記法です。\n",
    "先に作成した`~/.ssh/config`の設定が有効です。\n",
    "\n",
    "コネクション生成に成功すると成功メッセージが表示され、失敗するとエラーメッセージが表示されます。\n",
    "\n",
    "#### %logout記法\n",
    "\n",
    "`%logout` は、生成したコネクションを切断する、独自の記法です。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### エラー例\n",
    "\n",
    "%login前、もしくは、%logout後、に !をつけずにセル実行するとエラーメッセージが表示されます"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:53:14.666256Z",
     "start_time": "2018-12-06T01:53:14.658352Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m[ssh] Not connected\n",
      "\u001b[0m\u001b[0;31mTraceback (most recent call last):\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/ssh_kernel/kernel.py\", line 81, in do_execute_direct\n",
      "    self.assert_connected()\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/ssh_kernel/kernel.py\", line 179, in assert_connected\n",
      "    raise SSHKernelNotConnectedException\n",
      "ssh_kernel.exception.SSHKernelNotConnectedException\n",
      "\n",
      "\u001b[0m"
     ]
    },
    {
     "ename": "abort",
     "evalue": "not connected",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "%login時に名前解決に失敗するとエラーメッセージが表示されます"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:20:48.375980Z",
     "start_time": "2018-12-06T02:20:48.231109Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ssh] Login to notfound...\n",
      "[DEBUG] host=notfound hostname=notfound other_conf={}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m[ssh] Login to notfound failed.\n",
      "\u001b[0m\u001b[0;31mError in calling magic 'login' on line:\n",
      "    [Errno -5] No address associated with hostname\n",
      "    args: ['notfound']\n",
      "    kwargs: {}\n",
      "\u001b[0m\u001b[0;31mTraceback (most recent call last):\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/env/lib/python3.6/site-packages/metakernel/magic.py\", line 82, in call_magic\n",
      "    func(*args, **kwargs)\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/ssh_kernel/magics.py\", line 28, in line_login\n",
      "    raise e\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/ssh_kernel/magics.py\", line 25, in line_login\n",
      "    self.kernel.sshwrapper.connect(host)\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/ssh_kernel/ssh_wrapper_plumbum.py\", line 88, in connect\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/env/lib/python3.6/site-packages/plumbum/machines/paramiko_machine.py\", line 261, in __init__\n",
      "    self._client.connect(host, **kwargs)\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/env/lib/python3.6/site-packages/paramiko/client.py\", line 334, in connect\n",
      "    to_try = list(self._families_and_addresses(hostname, port))\n",
      "  File \"/home/masaru/go/src/gitlab.com/m-ueno/ssh_kernel/env/lib/python3.6/site-packages/paramiko/client.py\", line 204, in _families_and_addresses\n",
      "    hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM\n",
      "  File \"/home/masaru/.asdf/installs/python/3.6.2/lib/python3.6/socket.py\", line 743, in getaddrinfo\n",
      "    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):\n",
      "socket.gaierror: [Errno -5] No address associated with hostname\n",
      "\n",
      "\u001b[0m\u001b[0;31m%login HOST\n",
      "\n",
      "SSH login to the remote host.\n",
      "Cells below this line will be executed remotely.\n",
      "\n",
      "Example:\n",
      "    [~/.ssh/config]\n",
      "    Host myserver\n",
      "        Hostname 10.0.0.10\n",
      "        Port 2222\n",
      "\n",
      "    %login myserver\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "%login notfound"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### %param記法\n",
    "\n",
    "`%param {変数名} {変数値}` はノートブック上で使えるパラメータを宣言します。\n",
    "\n",
    "詳細は [parameterized-notebook-ja](./parameterized-notebook-ja.ipynb) を参照して下さい。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### その他\n",
    "\n",
    "実装の仕組み上使えるものの未検証であったり主眼でない機能について触れます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:06:05.528490Z",
     "start_time": "2018-12-06T02:06:05.522175Z"
    }
   },
   "source": [
    "#### ! 記法\n",
    "\n",
    "<strong><u>非推奨</u></strong>\n",
    "\n",
    "`!` で始まるセル内容は、ローカルシェルで実行されます。\n",
    "\n",
    "* 注\n",
    "    * この記法は、コマンドが異常終了したとしても正常終了になってしまうため、利用を推奨しません。コマンドの異常終了を検知したい場合には`%login localhost`実行後にコマンドを入力してください\n",
    "    * この記法は、IPythonやipykernelの記法に似たものですが、IPythonやipykernelのように行内に書くことはできません"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T01:53:18.487470Z",
     "start_time": "2018-12-06T01:53:18.317517Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uid=1000(masaru) gid=1000(masaru) groups=1000(masaru),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),110(lxd),115(lpadmin),116(sambashare),121(libvirtd),999(docker)\n"
     ]
    }
   ],
   "source": [
    "!id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### %python記法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:36:19.899777Z",
     "start_time": "2018-12-06T02:36:19.894556Z"
    }
   },
   "outputs": [],
   "source": [
    "%python x = 1+1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:36:20.374683Z",
     "start_time": "2018-12-06T02:36:20.369185Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%python x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:37:44.201456Z",
     "start_time": "2018-12-06T02:37:44.194121Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x is 4\n"
     ]
    }
   ],
   "source": [
    "%%python\n",
    "\n",
    "x *= 2\n",
    "print('x is ' + str(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### %help\n",
    "\n",
    "実行すると、オンラインヘルプがポップアップ表示されます."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:36:45.829363Z",
     "start_time": "2018-12-06T02:36:45.824993Z"
    }
   },
   "outputs": [],
   "source": [
    "%help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:36:52.346193Z",
     "start_time": "2018-12-06T02:36:52.339609Z"
    }
   },
   "outputs": [],
   "source": [
    "%help %login"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-12-06T02:38:03.388841Z",
     "start_time": "2018-12-06T02:38:03.384790Z"
    }
   },
   "outputs": [],
   "source": [
    "%help %%python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 制限事項\n",
    "\n",
    "主に実装の都合により、次の制限があります。\n",
    "\n",
    "### 対話コマンド\n",
    "\n",
    "実行中にキー入力を待ち受ける対話コマンドは使えません。\n",
    "\n",
    "対策：\n",
    "\n",
    "* Jupyter Notebookの\"Terminal\"を使う\n",
    "* `sudo`コマンドは、パスワード入力なしで実行できるよう設定を変える\n",
    "* 「対話モード」の代わりに「バッチモード」で実行する\n",
    "    * e.g. `top` => `top -b`\n",
    "* 間違って実行してしまったら、Jupyter Notebookの「interrupt the kernel」か「restart kernel」を実行し、コマンド実行を中断する\n",
    "    * ![](./interrupt.png)\n",
    "\n",
    "### パスワードログイン\n",
    "\n",
    "上と同様の理由で、対象サーバへのパスワードでのログインはできません。\n",
    "\n",
    "対策：\n",
    "\n",
    "* SSH公開鍵認証の設定をする\n",
    "+ SSH秘密鍵をパスフレーズで保護し、Jupyter Notebookをssh-agentクライアントとして起動する\n",
    "\n",
    "### シェル変数\n",
    "\n",
    "セルをまたぐ場合、一部のシェル変数の値が、SSH端末と異なることがあります（例：`$?`, `$$`）。\n",
    "これは、コマンド実行の度に[SSHチャネル](http://docs.paramiko.org/en/2.4/api/channel.html)を生成しシェル変数がリセットされるためです。\n",
    "同一セル内であればリセットされません。\n",
    "\n",
    "### !記法\n",
    "\n",
    "* !記法 (`%shell` magic) は[Metakernel](https://github.com/Calysto/metakernel/)の実装を利用しており、IPython Kernelの実装・挙動と異なります。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "SSH",
   "language": "bash",
   "name": "ssh"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "ssh"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}