In the ever-evolving landscape of autonomous vehicles, innovation takes center stage. In a recent project, I embarked on a distinctive journey to develop a self-driving car by leveraging unconventional methods. Rather than starting from scratch, I opted for a mod for a car simulator game, allowing me to focus on fine-tuning the neural network's capabilities and enhancing real-time decision-making. In this blog post, I will delve into the core components of the project, highlighting the integration of simulation-based training with real-world data and the use of cutting-edge technologies like UNET, YOLO, and SORT.
The Neural Network Architecture
At the heart of this project lies a neural network equipped with 5 Convolutional layers and 5 fully connected layers. This sophisticated architecture was meticulously crafted to analyze each frame of the simulation, utilizing a downsized 640x160 screen and real-time car speed data. The network's output plays a crucial role in steering angles and speed-related decisions, including acceleration, braking, and maintaining the current speed. The model underwent rigorous training through 7 hours of manual driving using a keyboard (hence the janky steering) in a simulated city environment, ensuring its adaptability to various driving scenarios.
Enhanced Features and Models
During the testing phase, I explored advanced features and models to augment the self-driving capabilities. The integration of UNET enabled a comprehensive analysis of the road in the simulation. Leveraging a labeled dataset from actual roads, I implemented data augmentation techniques to simulate diverse road conditions. Additionally, I incorporated YOLO for car and traffic light detection, enhancing the car's perception of its environment. To ensure effective car tracking, the SORT algorithm was employed, providing a robust solution for real-time object tracking.
Intriguing Experiments with Facebook's "Segment Everything" Approach
As part of the continuous exploration of cutting-edge technologies, I experimented with Facebook's "segment everything" approach. This innovative methodology aimed to evaluate potential advantages in self-driving car development. By segmenting various elements in the simulation, including roads, obstacles, and other objects, the system gained a deeper understanding of the environment. This experiment showcased the project's adaptability and commitment to staying at the forefront of technological advancements.
A Pragmatic Blend of Neural Networks and Computer Vision
This project represents a pragmatic exploration of neural networks and computer vision, seamlessly combining simulation-based training with real-world data. The synergy between the carefully designed neural network architecture and the integration of advanced models showcases the project's commitment to achieving a comprehensive understanding of autonomous systems.
The progress made in this project is a testament to the effectiveness of a hands-on approach to self-driving car development. As the journey into the future of transportation continues, stay tuned for more updates and innovations. The project not only demonstrates the practical application of cutting-edge technologies but also serves as an inspiration for those eager to push the boundaries of what's possible in the realm of autonomous systems. The future of self-driving cars is undoubtedly exciting, and this project is a step forward in unlocking its full potential.
