How To Draw Cnn Architecture In Tensorflow With Tensorboard
intermediate/tensorboard_tutorial
Run in Google Colab
Colab
Download Notebook
Notebook
View on GitHub
GitHub
Visualizing Models, Data, and Training with TensorBoard¶
In the 60 Minute Blitz, we evidence you how to load in data, feed information technology through a model we ascertain as a bracket of nn.Module , railroad train this model on training data, and exam it on test data. To encounter what's happening, we print out some statistics as the model is training to become a sense for whether training is progressing. However, we tin do much better than that: PyTorch integrates with TensorBoard, a tool designed for visualizing the results of neural network training runs. This tutorial illustrates some of its functionality, using the Fashion-MNIST dataset which can exist read into PyTorch using torchvision.datasets.
In this tutorial, we'll learn how to:
- Read in information and with appropriate transforms (most identical to the prior tutorial).
- Ready TensorBoard.
- Write to TensorBoard.
- Inspect a model compages using TensorBoard.
- Use TensorBoard to create interactive versions of the visualizations nosotros created in last tutorial, with less lawmaking
Specifically, on point #v, we'll run across:
- A couple of ways to inspect our training information
- How to track our model's performance as information technology trains
- How to assess our model'southward operation once it is trained.
We'll begin with similar boilerplate code as in the CIFAR-10 tutorial:
# imports import matplotlib.pyplot every bit plt import numpy as np import torch import torchvision import torchvision.transforms as transforms import torch.nn every bit nn import torch.nn.functional every bit F import torch.optim equally optim # transforms transform = transforms . Etch ( [ transforms . ToTensor (), transforms . Normalize (( 0.5 ,), ( 0.5 ,))]) # datasets trainset = torchvision . datasets . FashionMNIST ( './data' , download = True , railroad train = True , transform = transform ) testset = torchvision . datasets . FashionMNIST ( './information' , download = Truthful , train = False , transform = transform ) # dataloaders trainloader = torch . utils . data . DataLoader ( trainset , batch_size = 4 , shuffle = Truthful , num_workers = two ) testloader = torch . utils . data . DataLoader ( testset , batch_size = 4 , shuffle = False , num_workers = 2 ) # constant for classes classes = ( 'T-shirt/top' , 'Trouser' , 'Pullover' , 'Dress' , 'Glaze' , 'Sandal' , 'Shirt' , 'Sneaker' , 'Handbag' , 'Ankle Kick' ) # helper function to testify an image # (used in the `plot_classes_preds` role below) def matplotlib_imshow ( img , one_channel = False ): if one_channel : img = img . mean ( dim = 0 ) img = img / 2 + 0.v # unnormalize npimg = img . numpy () if one_channel : plt . imshow ( npimg , cmap = "Greys" ) else : plt . imshow ( np . transpose ( npimg , ( 1 , 2 , 0 ))) We'll ascertain a similar model architecture from that tutorial, making only pocket-size modifications to business relationship for the fact that the images are at present one channel instead of iii and 28x28 instead of 32x32:
class Internet ( nn . Module ): def __init__ ( self ): super ( Net , self ) . __init__ () self . conv1 = nn . Conv2d ( 1 , 6 , 5 ) self . pool = nn . MaxPool2d ( two , 2 ) self . conv2 = nn . Conv2d ( vi , xvi , 5 ) self . fc1 = nn . Linear ( 16 * 4 * 4 , 120 ) self . fc2 = nn . Linear ( 120 , 84 ) self . fc3 = nn . Linear ( 84 , 10 ) def frontward ( self , x ): ten = cocky . pool ( F . relu ( self . conv1 ( x ))) x = self . pool ( F . relu ( self . conv2 ( x ))) x = x . view ( - 1 , 16 * four * 4 ) x = F . relu ( self . fc1 ( ten )) x = F . relu ( self . fc2 ( x )) x = self . fc3 ( x ) return x cyberspace = Internet () Nosotros'll define the same optimizer and criterion from earlier:
criterion = nn . CrossEntropyLoss () optimizer = optim . SGD ( net . parameters (), lr = 0.001 , momentum = 0.ix ) 1. TensorBoard setup¶
At present we'll prepare TensorBoard, importing tensorboard from torch.utils and defining a SummaryWriter , our key object for writing data to TensorBoard.
from torch.utils.tensorboard import SummaryWriter # default `log_dir` is "runs" - we'll exist more specific here writer = SummaryWriter ( 'runs/fashion_mnist_experiment_1' ) Notation that this line solitary creates a runs/fashion_mnist_experiment_1 folder.
ii. Writing to TensorBoard¶
Now let's write an paradigm to our TensorBoard - specifically, a grid - using make_grid.
# go some random training images dataiter = iter ( trainloader ) images , labels = dataiter . adjacent () # create grid of images img_grid = torchvision . utils . make_grid ( images ) # bear witness images matplotlib_imshow ( img_grid , one_channel = True ) # write to tensorboard writer . add_image ( 'four_fashion_mnist_images' , img_grid ) Now running
tensorboard -- logdir = runs from the command line and then navigating to http://localhost:6006 should prove the post-obit.
Now you know how to use TensorBoard! This example, however, could be done in a Jupyter Notebook - where TensorBoard really excels is in creating interactive visualizations. Nosotros'll cover one of those next, and several more by the finish of the tutorial.
3. Inspect the model using TensorBoard¶
1 of TensorBoard'southward strengths is its ability to visualize complex model structures. Allow's visualize the model nosotros congenital.
author . add_graph ( net , images ) writer . close () Now upon refreshing TensorBoard you should see a "Graphs" tab that looks like this:
Go alee and double click on "Net" to see it aggrandize, seeing a detailed view of the individual operations that make up the model.
TensorBoard has a very handy feature for visualizing high dimensional information such as image information in a lower dimensional space; nosotros'll cover this adjacent.
4. Adding a "Projector" to TensorBoard¶
We tin can visualize the lower dimensional representation of higher dimensional data via the add_embedding method
# helper part def select_n_random ( data , labels , north = 100 ): ''' Selects n random datapoints and their respective labels from a dataset ''' assert len ( data ) == len ( labels ) perm = torch . randperm ( len ( data )) return information [ perm ][: n ], labels [ perm ][: n ] # select random images and their target indices images , labels = select_n_random ( trainset . data , trainset . targets ) # get the class labels for each epitome class_labels = [ classes [ lab ] for lab in labels ] # log embeddings features = images . view ( - 1 , 28 * 28 ) writer . add_embedding ( features , metadata = class_labels , label_img = images . unsqueeze ( 1 )) writer . close () At present in the "Projector" tab of TensorBoard, you lot can see these 100 images - each of which is 784 dimensional - projected down into iii dimensional space. Furthermore, this is interactive: you can click and drag to rotate the three dimensional projection. Finally, a couple of tips to make the visualization easier to encounter: select "color: label" on the summit left, as well every bit enabling "night fashion", which will brand the images easier to encounter since their background is white:
Now we've thoroughly inspected our data, let's evidence how TensorBoard can make tracking model training and evaluation clearer, starting with training.
5. Tracking model grooming with TensorBoard¶
In the previous instance, we simply printed the model'south running loss every 2000 iterations. Now, we'll instead log the running loss to TensorBoard, along with a view into the predictions the model is making via the plot_classes_preds function.
# helper functions def images_to_probs ( cyberspace , images ): ''' Generates predictions and corresponding probabilities from a trained network and a list of images ''' output = net ( images ) # convert output probabilities to predicted form _ , preds_tensor = torch . max ( output , 1 ) preds = np . clasp ( preds_tensor . numpy ()) return preds , [ F . softmax ( el , dim = 0 )[ i ] . detail () for i , el in zip ( preds , output )] def plot_classes_preds ( cyberspace , images , labels ): ''' Generates matplotlib Effigy using a trained network, along with images and labels from a batch, that shows the network's superlative prediction along with its probability, alongside the actual characterization, coloring this information based on whether the prediction was correct or not. Uses the "images_to_probs" role. ''' preds , probs = images_to_probs ( net , images ) # plot the images in the batch, along with predicted and truthful labels fig = plt . figure ( figsize = ( 12 , 48 )) for idx in np . arange ( four ): ax = fig . add_subplot ( 1 , 4 , idx + one , xticks = [], yticks = []) matplotlib_imshow ( images [ idx ], one_channel = True ) ax . set_title ( " {0} , {1:.1f} % \n (label: {ii} )" . format ( classes [ preds [ idx ]], probs [ idx ] * 100.0 , classes [ labels [ idx ]]), color = ( "green" if preds [ idx ] == labels [ idx ] . particular () else "red" )) render fig Finally, let's train the model using the same model training code from the prior tutorial, but writing results to TensorBoard every g batches instead of printing to console; this is done using the add_scalar office.
In addition, every bit we train, we'll generate an image showing the model'due south predictions vs. the actual results on the four images included in that batch.
running_loss = 0.0 for epoch in range ( 1 ): # loop over the dataset multiple times for i , data in enumerate ( trainloader , 0 ): # become the inputs; information is a listing of [inputs, labels] inputs , labels = data # cypher the parameter gradients optimizer . zero_grad () # forward + backward + optimize outputs = net ( inputs ) loss = criterion ( outputs , labels ) loss . backward () optimizer . step () running_loss += loss . item () if i % 1000 == 999 : # every thou mini-batches... # ...log the running loss writer . add_scalar ( 'training loss' , running_loss / 1000 , epoch * len ( trainloader ) + i ) # ...log a Matplotlib Figure showing the model'due south predictions on a # random mini-batch writer . add_figure ( 'predictions vs. actuals' , plot_classes_preds ( cyberspace , inputs , labels ), global_step = epoch * len ( trainloader ) + i ) running_loss = 0.0 print ( 'Finished Training' ) Y'all tin can at present expect at the scalars tab to come across the running loss plotted over the fifteen,000 iterations of preparation:
In addition, we can look at the predictions the model made on arbitrary batches throughout learning. See the "Images" tab and scroll down under the "predictions vs. actuals" visualization to see this; this shows us that, for example, after just 3000 training iterations, the model was already able to distinguish between visually distinct classes such as shirts, sneakers, and coats, though it isn't as confident as it becomes later on in training:
In the prior tutorial, we looked at per-form accuracy in one case the model had been trained; here, nosotros'll apply TensorBoard to plot precision-recall curves (good explanation here) for each class.
6. Assessing trained models with TensorBoard¶
# 1. gets the probability predictions in a test_size x num_classes Tensor # 2. gets the preds in a test_size Tensor # takes ~10 seconds to run class_probs = [] class_label = [] with torch . no_grad (): for data in testloader : images , labels = data output = net ( images ) class_probs_batch = [ F . softmax ( el , dim = 0 ) for el in output ] class_probs . suspend ( class_probs_batch ) class_label . append ( labels ) test_probs = torch . cat ([ torch . stack ( batch ) for batch in class_probs ]) test_label = torch . cat ( class_label ) # helper function def add_pr_curve_tensorboard ( class_index , test_probs , test_label , global_step = 0 ): ''' Takes in a "class_index" from 0 to 9 and plots the respective precision-recall curve ''' tensorboard_truth = test_label == class_index tensorboard_probs = test_probs [:, class_index ] writer . add_pr_curve ( classes [ class_index ], tensorboard_truth , tensorboard_probs , global_step = global_step ) writer . close () # plot all the pr curves for i in range ( len ( classes )): add_pr_curve_tensorboard ( i , test_probs , test_label ) You will now run across a "PR Curves" tab that contains the precision-recall curves for each class. Go ahead and poke around; y'all'll see that on some classes the model has almost 100% "area nether the curve", whereas on others this expanse is lower:
And that's an intro to TensorBoard and PyTorch's integration with it. Of form, you could do everything TensorBoard does in your Jupyter Notebook, only with TensorBoard, you gets visuals that are interactive by default.
Source: https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html
Posted by: marcucciancessitere.blogspot.com
0 Response to "How To Draw Cnn Architecture In Tensorflow With Tensorboard"
Post a Comment