ML-Agents external brains

Until now, all of our experiments have been with the heuristics brain using an internal RL algorithm called Q-learning. Now that we realize that C# scripts will only go so far, we can look at using ML-Agent external brains developed with Python. The preferred method for Unity is to run a Jupyter Python notebook externally to control the Unity training environment. This requires us to build a special Unity environment. Go through the following steps to learn how to configure the 3D ball environment for external training:

1. Open the Unity editor and load the ML-Agents demo unityenvironment project. If you still have it open from the last chapter, then that will work as well.
2. Open the 3DBall.scene in the ML-Agents/Examples/3DBall folder.
3. Locate the Brain3DBrain object in the Hierarchy window and select it. In the Inspector window set the Brain Type to External.
4. From the menu, select Edit | Project Settings | Player. From the Inspector window, set the properties as shown...