My Journey with Unity ML-Agents

Hey everyone!

I recently dove into the fascinating world of Unity ML-Agents, and I just have to share my experience. It’s been a wild ride, but incredibly rewarding.

For those who don’t know, Unity ML-Agents is a toolkit that allows you to use machine learning to create intelligent behaviors in Unity. It’s super cool and pretty beginner-friendly, even if you don’t have a deep background in AI.

Here is the preview of what Ml agents is capable of doing

Getting Started I decided to start with a simple project: teaching an AI to navigate a maze. Sounds straightforward, right? Well, it’s a bit more complex than it sounds, but that’s what makes it fun!

Setting Up the Environment First things first, I set up my environment in Unity. I created a basic maze with walls and a goal area. Then, I added a little cube to act as my AI agent. The Unity ML-Agents toolkit comes with a handy package that you can import into your project, which makes the setup process much easier.

Defining the Agent Next, I needed to define the agent. This involves writing a script that tells the agent what to do and how to learn. The agent script extends from the Agent class provided by ML-Agents.

Initialization: Set up the agent and the environment.

Observations: Define what information the agent can observe from the environment. In my case, it was the positions of the walls, the goal, and the agent itself.

Actions: Define what actions the agent can take, such as moving forward, backward, left, or right.

Rewards: Set up a reward system to encourage the agent to reach the goal. I gave positive rewards for getting closer to the goal and negative rewards for hitting walls.

Training the Agent

Here’s a my script -> Script

Now comes the fun part: training the agent. Unity ML-Agents uses a reinforcement learning approach, which means the agent learns by trial and error. It takes actions, observes the results, and adjusts its behavior based on the rewards it receives.

To train the agent, I used the provided training scripts. You run these scripts in a Python environment, and they communicate with Unity to train your agent. It took a few tries to get everything running smoothly, but once I did, it was amazing to watch the agent learn. At first, it wandered around aimlessly, bumping into walls and getting frustrated (at least, it seemed that way!). But gradually, it started to understand the layout of the maze.

Tweaking and Improving Training an AI isn’t always a linear process. Sometimes, I had to tweak the environment, adjust the reward system, or change the agent’s observations to help it learn better. For example, I realized that my initial reward system was too harsh, and the agent was getting discouraged (again, just in a manner of speaking). By making the rewards more balanced, the agent started making better progress.

The Moment of Success After several hours of training and tweaking, the magic moment happened: my little cube navigated the maze and reached the goal all by itself! It was such a rewarding experience to see all the hard work pay off. The agent had learned to navigate the maze efficiently, avoiding walls and finding the shortest path to the goal.

What I Learned This project taught me a lot about machine learning and AI. Here are a few key takeaways:

Patience is crucial: Training an AI takes time. Don’t get discouraged if it doesn’t work perfectly on the first try.

Experiment and tweak: Sometimes, small changes in the environment or reward system can make a big difference in how well the agent learns.

Learning is iterative: Both for the AI and for you. Each training run teaches you something new. Final Thoughts

Working with Unity ML-Agents was an incredible experience. It’s amazing to see how far AI technology has come and to be able to experiment with it in such a hands-on way. If you’re interested in AI and game development, I highly recommend giving Unity ML-Agents a try. It’s challenging, fun, and incredibly rewarding.

Thanks for reading! If you have any questions or want to share your own experiences with Unity ML-Agents, feel free to leave a comment. Happy coding!

Cheers, Rishabh Verma