Index

A

• activation function
  • about / Biological motivation and connections, Activation functions
  • sigmoid / Sigmoid
  • Tanh / Tanh
  • ReLU / ReLU
• advertising example, linear regression model
  • about / Advertising – a financial example
  • dependencies / Dependencies
  • data, importing with pandas / Importing data with pandas
  • advertising data / Understanding the advertising data
  • data analysis / Data analysis and visualization
  • visualization / Data analysis and visualization
  • simple regression model / Simple regression model
• alpha / Leaky ReLU
• apparent error / Apparent (training set) error
• architecture design, feed-forward neural network
  • input layer / Hidden units and architecture design
  • hidden layer / Hidden units and architecture design
  • output layer / Hidden units and architecture design
• autoencoder, for compressing MNIST dataset
  • building / Compressing the MNIST dataset
  • MNIST dataset, obtaining / The MNIST dataset
  • model, building / Building the model
  • model, training / Model training
• autoencoders
  • about / Introduction to autoencoders
  • examples / Examples of autoencoders
  • architecture / Autoencoder architectures
  • hyperparameters / Autoencoder architectures
  • convolutional autoencoder (CAE) / Convolutional autoencoder
  • denoising autoencoders / Denoising autoencoders
  • application / Applications of autoencoders
  • for image colorization / Image colorization
  • for image enhancement / More applications

B

• backpropagation, of gradients / Exploding and vanishing gradients - recap
• backpropagation algorithm, MLP
  • about / Training our MLP – the backpropagation algorithm
  • working / Training our MLP – the backpropagation algorithm
  • forward propagation / Step 1 – forward propagation
  • weight updation / Step 2 – backpropagation and weight updation
• backpropagation through time (BPTT) / The intuition behind RNNs
• batch normalization
  • about / Data analysis and preprocessing
  • benefits / Data analysis and preprocessing
• bias-variance decomposition / Bias-variance decomposition

C

• Cabin feature / Cabin
• categorical variables / Feature transformations
• CelebA dataset / Face generation
• challenges, machine learning
  • overview / Challenges of learning
  • feature extraction / Feature extraction – feature engineering
  • noise / Noise
  • overfitting / Overfitting
  • machine learning algorithm selection / Selection of a machine learning algorithm
  • prior knowledge / Prior knowledge
  • missing values / Missing values
• character-level language models
  • about / Character-level language models
  • Shakespeare data, using / Language model using Shakespeare data
• cheat sheet / Model training
• checkpoints
  • reference / Saving checkpoints
• CIFAR-10 object detection
  • pre-trained model, using as feature extractor / CIFAR-10 object detection – revisited
  • solution outline / Solution outline
  • packages, exploring / Loading and exploring CIFAR-10
  • packages, loading / Loading and exploring CIFAR-10
  • inception model transfer values, calculating / Inception model transfer values
  • transfer values, analyzing / Analysis of transfer values
  • model, training / Model building and training
  • model, building / Model building and training
• computational graphs
  • about / Computational graphs
  • data types / TensorFlow data types, variables, and placeholders
  • variables / TensorFlow data types, variables, and placeholders, Variables
  • placeholders / TensorFlow data types, variables, and placeholders, Placeholders
  • mathematical operations / Mathematical operations
• conda / Importing data with pandas
• Continuous Bag-of-Words (CBOW) / Building Word2Vec model
• convolutional autoencoder (CAE)
  • about / Convolutional autoencoder
  • MNIST dataset, using / Dataset
  • model, building / Building the model
  • model, training / Model training
• convolutional neural network (CNN)
  • about / What this book covers, Face generation
  • motivation / Motivation
  • applications / Applications of CNNs
  • layers / Different layers of CNNs
  • MNIST digit classification example / CNN basic example – MNIST digit classification
  • URL / Face generation
• convolution operation
  • about / The convolution operation
  • stride / The convolution operation
  • zero-padding / The convolution operation
• convolution step, CNN / Convolution step
• CUDA 8
  • installing / Installing NVIDIA drivers and CUDA 8
  • URL / Installing NVIDIA drivers and CUDA 8
• curse of dimensionality
  • about / The curse of dimensionality
  • avoiding / Avoiding the curse of dimensionality

D

• DataFrame / Importing data with pandas
• data science
  • example / Understanding data science by an example
  • data size / Data size and industry needs
  • industry needs / Data size and industry needs
• data science algorithms
  • design procedure / Design procedure of data science algorithms
• data types / TensorFlow data types, variables, and placeholders
• Deconvolutional Network (DNN)
  • URL / Face generation
  • about / Face generation
• Deep Convolution Neural Network (DCNN) / Semi-supervised learning with Generative Adversarial Networks (GANs)
• deep learning framework
  • need for / The TensorFlow environment
• denoising autoencoders
  • about / Denoising autoencoders
  • architecture / Denoising autoencoders
  • model, building / Building the model
  • model, training / Model training
• derived features
  • about / Derived features
  • name variable / Name
  • Cabin feature / Cabin
  • Ticket feature / Ticket
• design procedure, data science algorithms
  • data, pre-processing / Data pre-processing, Data pre-processing
  • data cleaning / Data cleaning
  • feature selection / Feature selection
  • model selection / Model selection
  • learning process / Learning process
  • model evaluation / Evaluating your model
• digit classification model
  • building / Digit classification – model building and training, Building the model
  • training / Digit classification – model building and training, Model training
  • data analysis / Data analysis
• downsampling step / The pooling step
• dropout / Building an LSTM cell

E

• embedding / Introduction to representation learning, Building the model
• embedding lookup / Skip-gram Word2Vec implementation
• errors
  • about / Different types of errors
  • apparent error / Apparent (training set) error
  • generalization error / Generalization/true error
• exploding, of gradients / Exploding and vanishing gradients - recap

F

• face generation
  • about / Face generation
  • data, obtaining / Getting the data
  • data, exploring / Exploring the Data
  • model, building / Building the model
  • model inputs, defining / Model inputs
  • discriminator, implementing / Discriminator
  • generator, implementing / Generator
  • model losses, calculating / Model losses
  • model optimizer, implementing / Model optimizer
  • model, training / Training the model
• feature detector / The convolution operation
• feature engineering
  • about / Feature engineering
  • types / Types of feature engineering
  • Titanic example / Titanic example revisited, Titanic example revisited – all together
  • missing values / Missing values
  • feature transformations / Feature transformations
  • derived features / Derived features
  • interaction features / Interaction features
• feature map / The convolution operation
• feature transformations
  • quantitative features / Feature transformations
  • qualitative variables / Feature transformations
  • dummy features / Dummy features
  • factorizing / Factorizing
  • scaling / Scaling
  • binning / Binning
• feed-forward neural network (FNN)
  • about / Feed-forward neural network, The intuition behind RNNs
  • example / Feed-forward neural network
  • hidden units / Hidden units and architecture design
  • architecture design / Hidden units and architecture design
• filter / The convolution operation
• fish recognition/detection model implementation
  • overview / Implementing the fish recognition/detection model
  • knowledge base/dataset / Knowledge base/dataset
  • data analysis pre-processing / Data analysis pre-processing
  • model, building / Model building
  • model training / Model training and testing
  • testing / Model training and testing
  • fish recognition / Fish recognition – all together
  • code / Code for fish recognition
• Flickr
  • URL / Semi-supervised learning
  • about / Semi-supervised learning
• forget gate layer / Why does LSTM work?
• fully connected layer, CNN
  • about / Fully connected layer
  • logits layer / Logits layer

G

• GAN network
  • building / Building the GAN network
  • model hyperparameters, defining / Model hyperparameters
  • generator, defining / Defining the generator and discriminator
  • discriminator, defining / Defining the generator and discriminator
  • generator loss, defining / Discriminator and generator losses
  • discriminator loss, defining / Discriminator and generator losses
  • optimizers, defining / Optimizers
  • model, training / Model training
  • model performance, testing / Generator samples from training
  • new images, generating with checkpoints / Sampling from the generator
• Gate / RNNs – sentiment analysis context
• Gated Recurrent Unit (GRU) / Building the model
• generalization error / Generalization/true error
• Generative Adversarial Networks (GANs)
  • about / An intuitive introduction
  • Generator / An intuitive introduction
  • Discriminator / An intuitive introduction
  • implementation / Simple implementation of GANs
  • MNIST dataset, using / Simple implementation of GANs
  • model inputs, defining / Model inputs
  • variable scope, of TensorFlow / Variable scope
  • leaky ReLU / Leaky ReLU
  • generator, defining / Generator
  • discriminator, building / Discriminator
• global average pooling (GAP) / Discriminator
• gradient / Classification and logistic regression
• gradient clipping / Optimizer
• gradient descent / The intuition behind RNNs

H

• Hadamard product / Defining multidimensional arrays using TensorFlow

I

• input gate layer / Why does LSTM work?
• input layer, CNN / Input layer

K

• k-fold cross-validation / Model building
• Keras
  • used, for sentiment analysis / Keras
• kernel / The convolution operation

L

• label smooth / Discriminator and generator losses
• language model
  • implementation / Implementation of the language model
  • mini-batch generation, for training / Mini-batch generation for training
  • building / Building the model
  • architecture / Model architecture
  • inputs, defining / Inputs
  • LSTM cell, building / Building an LSTM cell
  • RNN output layer, creating / RNN output
  • training loss, calculating / Training loss
  • optimizer / Optimizer
  • network, building / Building the network
  • hyperparameters / Model hyperparameters
  • training / Training the model
  • checkpoints, saving / Saving checkpoints
  • text, generating / Generating text
• latent space / Model inputs
• layers, convolutional neural network (CNN)
  • about / Different layers of CNNs
  • input layer / Input layer
  • convolution step / Convolution step
  • non-linearity / Introducing non-linearity
  • pooling step / The pooling step
  • fully connected layer / Fully connected layer
• lazy evaluation / Getting output from TensorFlow
• leaky ReLU / Leaky ReLU
• learning visibility / Learning visibility
• linear discriminant / Linear models for classification
• linear models, for classification / Linear models for classification
• linear regression, with TensorFlow / Linear regression with TensorFlow
• linear regression model
  • about / Linear models for regression, Linear regression model – building and training
  • key factors / Motivation
  • advertising / Advertising – a financial example
  • building / Linear regression model – building and training
  • training / Linear regression model – building and training
• logistic regression
  • about / Linear models for classification
  • classification / Classification and logistic regression
  • model, building / Logistic regression model – building and training
  • model, training / Logistic regression model – building and training
• logistic regression model, in TensorFlow
  • utilizing / Utilizing logistic regression in TensorFlow
  • placeholders, using / Why use placeholders?
  • model weights and bias, setting / Set model weights and bias
  • running / Logistic regression model
  • training / Training
  • cost function, defining / Cost function
• logits layer / Logits layer
• long-term dependencies
  • problem / The problem of long-term dependencies
• Long Short Term Networks (LSTM)
  • about / The problem of long-term dependencies
  • reference / LSTM networks
  • architecture / LSTM networks
  • information, processing / Why does LSTM work?

M

• machine learning
  • challenges / Getting to learn
  • types / Different learning types
  • error types / Different types of errors
• macOS X
  • TensorFlow CPU installation / TensorFlow CPU installation for macOS X
• mathematical operations / Mathematical operations
• Matplotlib
  • URL / Data analysis and visualization
• maximum likelihood method
  • about / Building Word2Vec model
  • reference / Building Word2Vec model
• max pool / The pooling step
• missing values, feature engineering
  • sample with missing values, removing / Removing any sample with missing values in it
  • inputting / Missing value inputting
  • average value, assigning / Assigning an average value
  • predicting, with regression model / Using a regression or another simple model to predict the values of missing variables
• MNIST dataset analysis
  • about / MNIST dataset analysis
  • MNIST data / The MNIST data
• MNIST digit classification example
  • implementing / CNN basic example – MNIST digit classification
  • model, building / Building the model
  • cost function, defining / Cost function
  • performance measures, defining / Performance measures
  • model, training / Model training
• model coefficients / Simple regression model
• multi-layer perceptron (MLP)
  • about / Capacity of a single neuron, The need for multilayer networks
  • example / The need for multilayer networks
  • training / Training our MLP – the backpropagation algorithm
  • backpropagation algorithm / Training our MLP – the backpropagation algorithm
• multilayer networks
  • need for / The need for multilayer networks
  • Input layer / The need for multilayer networks
  • Hidden layer / The need for multilayer networks
  • Output layer / The need for multilayer networks
• multinomial logistic regression / Building the model

N

• name variable / Name
• Natural Language Processing (NLP) / Introduction to representation learning
• negative sampling
  • reference / Building Word2Vec model
  • about / Building Word2Vec model
• neural network
  • about / Capacity of a single neuron
  • biological motivation and connections / Biological motivation and connections
• neuron / Biological motivation and connections
• node / Biological motivation and connections
• noise-contrastive estimation (NCE)
  • about / Building Word2Vec model
  • reference / Building Word2Vec model
• non-linearity, CNN / Introducing non-linearity
• NVIDIA CUDA Deep Neural Network library (cuDNN) / Installing NVIDIA drivers and CUDA 8
• NVIDIA drivers
  • installing / Installing NVIDIA drivers and CUDA 8

O

• object detection / Object detection
• object detection example
  • about / CIFAR-10 – modeling, building, and training
  • packages, importing / Used packages
  • CIFAR-10 dataset, loading / Loading the CIFAR-10 dataset
  • data analysis / Data analysis and preprocessing
  • preprocessing / Data analysis and preprocessing
  • network, building / Building the network
  • model, training / Model training
  • model, testing / Testing the model
• one-hot encoding / Skip-gram Word2Vec implementation
• overfitting / Understanding data science by an example, Bias-variance decomposition

P

• pandas
  • about / Importing data with pandas
  • reference link / Importing data with pandas
• part-of-speech (POS) tagging / A practical example of the skip-gram architecture
• placeholders / TensorFlow data types, variables, and placeholders, Placeholders
• pooling step, CNN / The pooling step
• Principal Component Analysis (PCA) / Dimensionality reduction, Analysis of transfer values

R

• Rectified linear unit (ReLU) / ReLU
• recurrent neural network (RNN)
  • intuition / The intuition behind RNNs
  • architecture / Recurrent neural networks architectures
  • about / General sentiment analysis architecture
  • for sentiment analysis / RNNs – sentiment analysis context
  • gradients, exploding / Exploding and vanishing gradients - recap
  • gradients, vanishing / Exploding and vanishing gradients - recap
• recurrent neural network (RNN), examples
  • character-level language models / Examples of RNNs
  • vanishing gradient problem / The vanishing gradient problem
  • long-term dependencies problem / The problem of long-term dependencies
• reinforcement learning / Reinforcement learning
• representation learning / Introduction to representation learning
• rule of thumb
  • breaking / Breaking the rule of thumb

S

• semi-supervised learning / Semi-supervised learning, Semi-supervised learning with Generative Adversarial Networks (GANs)
• semi-supervised learning, with GANs
  • implementing / Semi-supervised learning with Generative Adversarial Networks (GANs)
  • building / Intuition
  • data analysis / Data analysis and preprocessing
  • data, preprocessing / Data analysis and preprocessing
  • model, building / Building the model
  • model inputs, defining / Model inputs
  • generator, implementing / Generator
  • discriminator, implementing / Discriminator
  • model losses, defining / Model losses
  • model optimizer, defining / Model optimizer
  • model, training / Model training
• sentiment analysis
  • architecture / General sentiment analysis architecture
  • with recurrent neural network (RNN) / RNNs – sentiment analysis context
  • model, implementation / Sentiment analysis – model implementation
  • Keras, using / Keras
  • data analysis / Data analysis and preprocessing
  • data, preprocessing / Data analysis and preprocessing
  • model, building / Building the model
  • model, training / Model training and results analysis
  • results, analyzing / Model training and results analysis
• session / Getting output from TensorFlow
• sigmoid activation function / Sigmoid
• sigmoid function / Classification and logistic regression
• simple linear regression model
  • about / Simple regression model
  • model coefficients / Learning model coefficients
  • model coefficients, interpreting / Interpreting model coefficients
  • using, for prediction / Using the model for prediction
• skip-gram architecture
  • practical example / A practical example of the skip-gram architecture
• skip-gram Word2Vec
  • implementation / Skip-gram Word2Vec implementation
  • data analysis / Data analysis and pre-processing
  • pre-processing / Data analysis and pre-processing
  • model, building / Building the model
  • training / Training
• stacked LSTM / Building the model, Stacked LSTMs
• Stochastic Gradient Descent (SGD) / Model building and training
• stride / The convolution operation
• subcategories, feature engineering
  • feature selection / Feature selection
  • dimensionality reduction / Dimensionality reduction
  • feature construction / Feature construction
• subsampling / The pooling step, Data analysis and pre-processing
• supervised learning
  • about / Supervised learning, Semi-supervised learning
  • classification / Supervised learning
  • regression / Supervised learning
• SVHN dataset
  • URL / Data analysis and preprocessing

T

• Tanh activation function / Tanh
• TensorBoard
  • about / TensorBoard – visualizing learning
  • using / TensorBoard – visualizing learning
• TensorFlow
  • about / Data size and industry needs, The TensorFlow environment
  • installation / TensorFlow installation, Installing TensorFlow
  • GPU installation, for Ubuntu 16.04 / TensorFlow GPU installation for Ubuntu 16.04
  • NVIDIA drivers, installing for GPU mode installation / Installing NVIDIA drivers and CUDA 8
  • CUDA 8, installing for GPU mode installation / Installing NVIDIA drivers and CUDA 8
  • CPU installation, for Ubuntu 16.04 / TensorFlow CPU installation for Ubuntu 16.04
  • CPU installation, for macOS X / TensorFlow CPU installation for macOS X
  • GPU/CPU installation, for Windows / TensorFlow GPU/CPU installation for Windows
  • output, obtaining / Getting output from TensorFlow
  • used, for defining multidimensional arrays / Defining multidimensional arrays using TensorFlow
  • variable scope / Variable scope
• TensorFlow terminologies
  • about / TensorFlow terminologies – recap
  • tensors / Why tensors?
  • variables / Variables
  • placeholders / Placeholders
  • operations / Operations
• testing phase / Understanding data science by an example
• tf.contrib.rnn.MultiRNNCell function
  • URL / Building an LSTM cell
• tf.nn.dynamic_rnn
  • URL / Building the network
• Ticket feature / Ticket
• Titanic example
  • about / Titanic example – model building and training
  • data handling / Data handling and visualization
  • data visualization / Data handling and visualization
  • data analysis, supervised machine learning / Data analysis – supervised machine learning
• tokenizer / Data analysis and preprocessing
• tokens / General sentiment analysis architecture
• transfer learning (TL)
  • about / What this book covers, Transfer learning
  • intuition / The intuition behind TL
  • usage scenario / The intuition behind TL
  • versus traditional machine learning / Differences between traditional machine learning and TL
  • conditions / Differences between traditional machine learning and TL
• types, machine learning
  • about / Different learning types
  • supervised learning / Supervised learning
  • unsupervised learning / Unsupervised learning
  • semi-supervised learning / Semi-supervised learning
  • reinforcement learning / Reinforcement learning

U

• Ubuntu 16.04
  • TensorFlow GPU installation / TensorFlow GPU installation for Ubuntu 16.04
  • TensorFlow CPU installation / TensorFlow CPU installation for Ubuntu 16.04
• underfitting / Bias-variance decomposition
• unit / Biological motivation and connections
• unseen data / Understanding data science by an example
• unsupervised learning
  • about / Unsupervised learning, Semi-supervised learning
  • clustering / Unsupervised learning
  • association / Unsupervised learning

V

• vanishing, of gradients / Exploding and vanishing gradients - recap
• vanishing gradient problem / The vanishing gradient problem
• variables / TensorFlow data types, variables, and placeholders, Variables
• variable scope, TensorFlow
  • about / Variable scope
  • reference / Variable scope

W

• weight matrix / Skip-gram Word2Vec implementation
• Windows
  • TensorFlow GPU/CPU installation / TensorFlow GPU/CPU installation for Windows
• Word2Vec
  • about / Word2Vec
  • model, building / Building Word2Vec model

Y

• yield
  • URL / Mini-batch generation for training
  • about / Mini-batch generation for training

Z

• zero-padding / The convolution operation