{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1ef797b6",
   "metadata": {},
   "source": [
    "Copyright (c) 2023 Graphcore Ltd. All rights reserved."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c4421e7c",
   "metadata": {},
   "source": [
    "Node Classification on IPU using Cluster-GCN - Training\n",
    "==================\n",
    "\n",
    "This notebook demonstrates training a Cluster GCN model presented in [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://arxiv.org/pdf/1905.07953.pdf) with PyTorch Geometric on the Graphcore IPU. We will use the Reddit dataset from [Inductive Representation Learning on Large Graphs](https://arxiv.org/abs/1706.02216) and train the model to predict the community a post belongs to.\n",
    "\n",
    "|  Domain | Tasks | Model | Datasets | Workflow |   Number of IPUs   | Execution time |\n",
    "|---------|-------|-------|----------|----------|--------------------|----------------|\n",
    "|   GNNs   |  Node Classification  | CGCN | Reddit | Training, evaluation | recommended: 4 | ~6 minutes |\n",
    "\n",
    "This notebook assumes some familiarity with PopTorch as well as PyTorch Geometric (PyG). For additional resources please consult:\n",
    "\n",
    "* [PopTorch Documentation](https://docs.graphcore.ai/projects/poptorch-user-guide/en/latest/index.html),\n",
    "* [PopTorch Examples and Tutorials](https://docs.graphcore.ai/en/latest/examples.html#pytorch),\n",
    "* [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/latest/)\n",
    "* [PopTorch Geometric Documentation](https://docs.graphcore.ai/projects/poptorch-geometric-user-guide/en/latest/index.html)\n",
    "\n",
    "Requirements:\n",
    "\n",
    "A Poplar SDK environment enabled (see the Getting Started guide for your IPU system Python packages installed with `pip install -r ../requirements.txt`"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a4609220",
   "metadata": {},
   "source": [
    "## Environment setup\n",
    "\n",
    "The best way to run this demo is on Paperspace Gradient’s cloud IPUs because everything is already set up for you. To improve your experience, we preload datasets and pre-install packages. This can take a few minutes. If you experience errors immediately after starting a session, please try restarting the kernel before contacting support. If a problem persists or you want to give us feedback on the content of this notebook, please reach out to through our community of developers using our [Slack channel](https://www.graphcore.ai/join-community) or raise a [GitHub issue](https://github.com/graphcore/Gradient-PyTorch-Geometric/issues).\n",
    "\n",
    "\n",
    "To run the demo using other IPU hardware, you need to have the Poplar SDK enabled. Refer to the [Getting Started guide](https://docs.graphcore.ai/en/latest/getting-started.html#getting-started) for your system for details on how to enable the Poplar SDK. Also refer to the [Jupyter Quick Start guide](https://docs.graphcore.ai/projects/jupyter-notebook-quick-start/en/latest/index.html) for how to set up Jupyter to be able to run this notebook on a remote IPU machine.\n",
    "\n",
    "Requirements:\n",
    "\n",
    "* Python packages installed with `pip install -r requirements.txt`"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "38efcf59",
   "metadata": {},
   "source": [
    "In order to improve usability and support for future users, Graphcore would like to collect information about the\n",
    "applications and code being run in this notebook. The following information will be anonymised before being sent to Graphcore:\n",
    "\n",
    "- User progression through the notebook\n",
    "- Notebook details: number of cells, code being run and the output of the cells\n",
    "- Environment details\n",
    "\n",
    "You can disable logging at any time by running `%unload_ext graphcore_cloud_tools.notebook_logging.gc_logger` from any cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21adb3ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -r requirements.txt\n",
    "%load_ext graphcore_cloud_tools.notebook_logging.gc_logger"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9e3a6bea",
   "metadata": {},
   "source": [
    "Lets import the required packages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aea1fc67",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import os.path as osp\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import poptorch\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "from poptorch_geometric import FixedSizeOptions, OverSizeStrategy\n",
    "from poptorch_geometric.cluster_loader import FixedSizeClusterLoader\n",
    "from torch_geometric.loader import ClusterData, ClusterLoader\n",
    "from torch_geometric.datasets import Reddit\n",
    "from torch_geometric.nn import SAGEConv\n",
    "from tqdm import tqdm\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d6671b1c",
   "metadata": {},
   "source": [
    "And for compatibility with the Paperspace environment variables we will do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73116e72",
   "metadata": {},
   "outputs": [],
   "source": [
    "poptorch.setLogLevel(\"ERR\")\n",
    "executable_cache_dir = (\n",
    "    os.getenv(\"POPLAR_EXECUTABLE_CACHE_DIR\", \"/tmp/exe_cache/\") + \"/pyg-clustergcn\"\n",
    ")\n",
    "dataset_directory = os.getenv(\"DATASETS_DIR\", \"data\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "db80fb10",
   "metadata": {},
   "source": [
    "Now we are ready to start!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d9278ac0",
   "metadata": {},
   "source": [
    "### Reddit Dataset\n",
    "\n",
    "PyG provides a convenient dataset class that manages downloading the Reddit dataset to local storage. The Reddit dataset contains one single graph which contains 232,965 Reddit posts. The graph is homogeneous and undirected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51bf2bab",
   "metadata": {},
   "outputs": [],
   "source": [
    "reddit_root = osp.join(dataset_directory, \"Reddit\")\n",
    "dataset = Reddit(reddit_root)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5c878a92",
   "metadata": {},
   "source": [
    "We can check the `len` on the dataset to see this is one single large graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b4186c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(dataset)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "455f0f88",
   "metadata": {},
   "source": [
    "And we can view the data within the graph. We can see there are 232965 nodes each with a feature size of 602. The dataset contains masks for training, validation and test which we will apply during those stages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d68585ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset[0]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "13beffc9",
   "metadata": {},
   "source": [
    "### Clustering\n",
    "\n",
    "As this dataset is a single large graph the computational cost grows exponentially as the layers increase. There is also a large memory requirement to keep the entire graph and node embeddings in memory. It is therefore useful to consider a sampling approach to mitigate these problems. In this example we use cluster sampling, which attempts to group the nodes into clusters of a similar size which minimises edge cuts.\n",
    "\n",
    "The following code clusters the original dataset into 1500 clusters using [METIS](https://epubs.siam.org/doi/10.1137/S1064827595287997)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2dfc2aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "total_num_clusters = 1500\n",
    "\n",
    "cluster_data = ClusterData(\n",
    "    dataset[0], num_parts=total_num_clusters, recursive=False, save_dir=reddit_root\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2410adad",
   "metadata": {},
   "source": [
    "We can now see we now have multiple items in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "873f9e08",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(cluster_data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "76d471dd",
   "metadata": {},
   "source": [
    "Each with a reduced set of the original data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdfef299",
   "metadata": {},
   "outputs": [],
   "source": [
    "cluster_data[0]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "295cf8d9",
   "metadata": {},
   "source": [
    "It can be useful to plot the distribution of nodes in each cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a430e217",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_nodes_per_cluster = []\n",
    "num_edges_per_cluster = []\n",
    "\n",
    "for cluster in cluster_data:\n",
    "    num_nodes_per_cluster.append(cluster.y.shape[0])\n",
    "    num_edges_per_cluster.append(cluster.edge_index.shape[1])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "30eb6fbc",
   "metadata": {},
   "source": [
    " As you can see the number of nodes per cluster is relatively balanced."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b268b5af",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.hist(np.array(num_nodes_per_cluster), 20)\n",
    "plt.xlabel(\"Number of nodes per cluster\")\n",
    "plt.ylabel(\"Counts\")\n",
    "plt.title(\"Histogram of nodes in each cluster\")\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "271ffde4",
   "metadata": {},
   "source": [
    "But the number of edges per cluster is not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84cfc57b",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.hist(np.array(num_edges_per_cluster), 20)\n",
    "plt.xlabel(\"Number of edges per cluster\")\n",
    "plt.ylabel(\"Counts\")\n",
    "plt.title(\"Histogram of edges in each cluster\")\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ee6b1b57",
   "metadata": {},
   "source": [
    "We will have to take this into consideration when loading our data for the IPU. Next we will look at how to load our clusters."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "abf3cb7d",
   "metadata": {},
   "source": [
    "## Data Loading and Batching\n",
    "\n",
    "A batch in the cluster GCN algorithm is created by:\n",
    "* Randomly select a number of clusters\n",
    "* Combine the clusters into a single graph and add the edges between the nodes in this new graph that were removed in clustering\n",
    "* This is our batch, a single graph that is a selection of clusters\n",
    "\n",
    "When using the IPU we need our inputs to be fixed size. Combining the clusters will result in a graph of a different size each batch and so we need the result of our combined clusters to be fixed size. Lets see how to do that.\n",
    "\n",
    "First let's create a cluster data loader in the normal way. This data loader will produce batches with dynamic size, but we will use it to calculate the number of nodes and edges to make our batches up to fixed size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8e88839",
   "metadata": {},
   "outputs": [],
   "source": [
    "clusters_per_batch = 6\n",
    "\n",
    "dynamic_size_dataloader = ClusterLoader(\n",
    "    cluster_data,\n",
    "    batch_size=clusters_per_batch,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "692e2ccf",
   "metadata": {},
   "source": [
    "Now we can sample from this data loader and calculate the maximum number of nodes and edges of each batch. We can use the method `FixedSizeOptions.from_loader` to help us with this. This will sample from the dynamic size data loader and measure the maximum nodes and edges in each sampled mini-batch. It will use these to initialise a `FixedSizeOptions` object that we can use to pad our batches up to fixed size in the data loader."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c437439",
   "metadata": {},
   "outputs": [],
   "source": [
    "fixed_size_options = FixedSizeOptions.from_loader(\n",
    "    dynamic_size_dataloader, sample_limit=10\n",
    ")\n",
    "print(fixed_size_options)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "815c9ba1",
   "metadata": {},
   "source": [
    "Now we can use these `fixed_size_options` with the fixed size version of the cluster loader that produces batches of fixed size, padding up to the maximum nodes and edges set in `fixed_size_options`. Notice how we set `over_size_strategy` to `TrimNodesAndEdges`. This is to ensure that if our combined clusters have a number of edges greater than the number of edges we have set, then the edges will be randomly removed to achieve the requested size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d448190b",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataloader = FixedSizeClusterLoader(\n",
    "    cluster_data,\n",
    "    batch_size=clusters_per_batch,\n",
    "    fixed_size_options=fixed_size_options,\n",
    "    over_size_strategy=OverSizeStrategy.TrimNodesAndEdges,\n",
    "    num_workers=8,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5a9cf5b9",
   "metadata": {},
   "source": [
    "Lets take a look at the first few items in the dataloader:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8cc6eb72",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataloader_iter = iter(train_dataloader)\n",
    "\n",
    "print(next(train_dataloader_iter))\n",
    "print(next(train_dataloader_iter))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6997812b",
   "metadata": {},
   "source": [
    "You can see that these two samples have the same sizes corresponding to our specified maximum nodes and edges per batch. Now we have our dataloader set up, we can start training our model. We will do this in the next section."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1bcb10d7",
   "metadata": {},
   "source": [
    "## Training\n",
    "\n",
    "Now we are in the position to start creating and training our cluster GCN model.\n",
    "\n",
    "### Model Architecture\n",
    "\n",
    "We take a very simple model to demonstrate the Cluster GCN approach, this is shown below. One key thing to note is we mask out the labels by setting the target at the mask locations to `-100`, which will be ignored by default in the loss function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a493df95",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Net(torch.nn.Module):\n",
    "    def __init__(self, in_channels, out_channels):\n",
    "        super().__init__()\n",
    "        self.conv_1 = SAGEConv(in_channels, 128)\n",
    "        self.conv_2 = SAGEConv(128, out_channels)\n",
    "\n",
    "    def forward(self, x, edge_index, mask=None, target=None):\n",
    "        x = self.conv_1(x, edge_index)\n",
    "        x = F.relu(x)\n",
    "        x = F.dropout(x, p=0.5, training=self.training)\n",
    "        x = self.conv_2(x, edge_index)\n",
    "        out = F.log_softmax(x, dim=-1)\n",
    "\n",
    "        if self.training:\n",
    "            # Mask out the nodes we don't care about\n",
    "            target = torch.where(mask, target, -100)\n",
    "            return out, F.nll_loss(out, target)\n",
    "        return out"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b7fcaef5",
   "metadata": {},
   "source": [
    "Lets create the `poptorch.Options` object with device iterations set to 4. Device iterations will increase the number of loops our model runs before returning to the host and can have a positive affect on our models throughput performance. For more information refer to the following resources for additional background:\n",
    "* PopTorch documentation [Efficient data batching](https://docs.graphcore.ai/projects/poptorch-user-guide/en/latest/batching.html#efficient-data-batching),\n",
    "* PopTorch tutorial: [Efficient data loading](https://github.com/graphcore/tutorials/tree/sdk-release-2.5/tutorials/pytorch/tut2_efficient_data_loading),\n",
    "\n",
    "We also enable outputting the results for each iteration as well as allowing the executable to be cached to avoid recompilation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dc528cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "options = poptorch.Options()\n",
    "options.deviceIterations(4)\n",
    "options.outputMode(poptorch.OutputMode.All)\n",
    "options.enableExecutableCaching(executable_cache_dir)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c03f859f",
   "metadata": {},
   "source": [
    "We can now use those options to instantiate our dataloader again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c538a7f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataloader = FixedSizeClusterLoader(\n",
    "    cluster_data,\n",
    "    batch_size=clusters_per_batch,\n",
    "    fixed_size_options=fixed_size_options,\n",
    "    over_size_strategy=OverSizeStrategy.TrimNodesAndEdges,\n",
    "    num_workers=8,\n",
    "    options=options,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "50dd81a4",
   "metadata": {},
   "source": [
    "Now inspecting our first two batches you can see that the items are larger than previously. This is because we have increased the device iterations to 4. PopTorch will slice this batch for us and distribute it over each of the device iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0d41973",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataloader_iter = iter(train_dataloader)\n",
    "\n",
    "print(next(train_dataloader_iter))\n",
    "print(next(train_dataloader_iter))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5f242c71",
   "metadata": {},
   "source": [
    "Lets create the model and prepare for training with PopTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37614d22",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Net(dataset.num_features, dataset.num_classes)\n",
    "model.train()\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=0.005)\n",
    "poptorch_model = poptorch.trainingModel(model, optimizer=optimizer, options=options)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "96962936",
   "metadata": {},
   "source": [
    "Now we can run the training loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "156df724",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_epochs = 10\n",
    "train_losses = torch.empty(num_epochs, len(train_dataloader))\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    bar = tqdm(train_dataloader)\n",
    "    for i, data in enumerate(bar):\n",
    "        # Performs forward pass, loss function evaluation,\n",
    "        # backward pass and weight update in one go on the device.\n",
    "        _, mini_batch_loss = poptorch_model(\n",
    "            data.x, data.edge_index, data.train_mask, data.y\n",
    "        )\n",
    "        train_losses[epoch, i] = float(mini_batch_loss.mean())\n",
    "        bar.set_description(\n",
    "            f\"Epoch {epoch} training loss: {train_losses[epoch, i].item():0.6f}\"\n",
    "        )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5b69d404",
   "metadata": {},
   "source": [
    "Finally we can detach the training model from the IPU:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb20e802",
   "metadata": {},
   "outputs": [],
   "source": [
    "poptorch_model.detachFromDevice()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6898f4b9",
   "metadata": {},
   "source": [
    "And finally lets take a look at the loss curve:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16806fe2",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(train_losses.mean(dim=1))\n",
    "plt.xlabel(\"Epoch\")\n",
    "plt.ylabel(\"Mean loss\")\n",
    "plt.legend([\"Training loss\"])\n",
    "plt.grid(True)\n",
    "plt.xticks(torch.arange(0, num_epochs, 2))\n",
    "plt.gcf().set_dpi(150)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "03b66110",
   "metadata": {},
   "source": [
    "We have successfully trained our simple model to do node classification on the Reddit dataset. In the next section we will see how we can run validation and test on our trained model."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0de40801",
   "metadata": {},
   "source": [
    "## Optional - Validation and Test"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f637f1c3",
   "metadata": {},
   "source": [
    "Now we can run validation and test on our trained model. For this we will need to do a single execution on the full graph on the CPU. This can take a while so we have left this section commented, feel free to uncomment and run validation and test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8baed13",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "data = dataset[0]\n",
    "\n",
    "model = Net(dataset.num_features, dataset.num_classes)\n",
    "model.load_state_dict(poptorch_model.state_dict())\n",
    "model.eval()\n",
    "out = model.forward(data.x, data.edge_index)\n",
    "y_pred = out.argmax(dim=-1)\n",
    "\n",
    "accs = []\n",
    "for mask in [data.val_mask, data.test_mask]:\n",
    "    correct = y_pred[mask].eq(data.y[mask]).sum().item()\n",
    "    accs.append(correct / mask.sum().item())\n",
    "\n",
    "print(\"Validation accuracy: {accs[0]}\")\n",
    "print(\"Test accuracy: {accs[1]}\")\n",
    "\"\"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2551a4cb",
   "metadata": {},
   "source": [
    "## Follow up\n",
    "\n",
    "We have successfully trained a simple model to do node classification on a large graph, using sampling to reduce the size of our batch.\n",
    "\n",
    "Next you could try:\n",
    "* Experiment with the dataloading to achieve higher throughput.\n",
    "* Try other sampling approaches with our PopTorch Geometric tools to achieve fixed size outputs."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}