Exercises

Use the following exercises to improve your understanding of RL and the PPO trainer.

1. Convert one of the Unity examples to use just visual observations. Hint, use the GridWorld example as a guide, and remember that the agent may need its own camera.
2. Alter the CNN configuration of an agent using visual observations in three different ways. You can add more layers, take them away, or alter the kernel filter. Run the training sessions and compare the differences with TensorBoard.
3. Convert the GridWorld sample to use vector observations and recurrent networks with memory. Hint, you can borrow several pieces of code from the Hallway example.

4. Revisit the Ball3D example and set it up to use multiple asynchronous agent training.
5. Set up the crawler example and run it with multiple asynchronous agent training.

If you encounter problems running through these samples, be sure to check online. These samples will likely be well worn, with many other people tweaking or enhancing them further.