TensorBoard

The complexity of a computation graph gets high even for moderately sized problems. Large computational graphs that represent complex machine learning models can become quite confusing and hard to understand. Visualization helps in easy understanding and interpretation of computation graphs, and thus accelerates the debugging and optimizations of TensorFlow programs. TensorFlow comes with a built-in tool that allows us to visualize computation graphs, namely, TensorBoard.

TensorBoard visualizes computation graph structure, provides statistical analysis and plots the values captured as summaries during the execution of computation graphs. Let's see how it works in practice.

A TensorBoard minimal example

1. Start by defining the variables and placeholders for our linear model:

# Assume Linear Model y = w * x + b
# Define model parameters
w = tf.Variable([.3], name='w',dtype=tf.float32)
b = tf.Variable([-.3], name='b', dtype=tf.float32)
# Define model input and output
x = tf.placeholder(name='x',dtype=tf.float32)
y = w * x + b

2. Initialize a session, and within the context of this session, do the following steps:
   • Initialize global variables
   • Create tf.summary.FileWriter that would create the output in the tflogs folder with the events from the default graph
   • Fetch the value of node y, effectively executing our linear model

with tf.Session() as tfs:
    tfs.run(tf.global_variables_initializer())
    writer=tf.summary.FileWriter('tflogs',tfs.graph)
    print('run(y,{x:3}) : ', tfs.run(y,feed_dict={x:3}))

3. We see the following output:

run(y,{x:3}) :  [ 0.60000002]

As the program executes, the logs are collected in the tflogs folder that would be used by TensorBoard for visualization. Open the command line interface, navigate to the folder from where you were running the ch-01_TensorFlow_101 notebook, and execute the following command:

tensorboard --logdir='tflogs'

You would see an output similar to this:

Starting TensorBoard b'47' at http://0.0.0.0:6006

Open a browser and navigate to http://0.0.0.0:6006. Once you see the TensorBoard dashboard, don't worry about any errors or warnings shown and just click on the GRAPHS tab at the top. You will see the following screen:

TensorBoard console

You can see that TensorBoard has visualized our first simple model as a computation graph:

Computation graph in TensorBoard

Let's now try to understand how TensorBoard works in detail.

TensorBoard details

TensorBoard works by reading log files generated by TensorFlow. Thus, we need to modify the programming model defined here to incorporate additional operation nodes that would produce the information in the logs that we want to visualize using TensorBoard. The programming model or the flow of programs with TensorBoard can be generally stated as follows:

1. Create the computational graph as usual.
2. Create summary nodes. Attach summary operations from the tf.summary package to the nodes that output the values that you wish to collect and analyze.
3. Run the summary nodes along with running your model nodes. Generally, you would use the convenience function, tf.summary.merge_all(), to merge all the summary nodes into one summary node. Then executing this merged node would basically execute all the summary nodes. The merged summary node produces a serialized Summary ProtocolBuffers object containing the union of all the summaries.

4. Write the event logs to disk by passing the Summary ProtocolBuffers object to a tf.summary.FileWriter object.
5. Start TensorBoard and analyze the visualized data.

In this section, we did not create summary nodes but used TensorBoard in a very simple way. We will cover the advanced usage of TensorBoard later in this book.