RoboCup is one of the major global AI events, gathering hundreds of teams from the world’s best universities to compete in various tasks ranging from soccer to home assistance and rescue. The commonality of these three seemingly dissimilar tasks is that in order to perform well, the robot needs to excel at the all major AI tasks: perception, control, navigation, strategy and planning. In this work, we focus on the first of these by presenting what is—to our knowledge—the first fully neural vision system for the Nao robot soccer. This is a challenging task, mainly due to the limited computational capabilities of the Nao robot. In this paper, we propose two novel neural network architectures for semantic segmentation and object detection that ensure low-cost inference, while improving accuracy by exploiting the properties of the environment. These models use synthetic transfer learning to be able to learn from a low number of hand-labeled images. The experiments show that our models outperform state-of-the-art methods such as Tiny YOLO at a fraction of the cost.