Eletoyia Advances to the ADVANCE Phase of the AI-BOOST AI Challenge with VINE-TALK

Eletoyia Deep Tech Chain Ltd has been selected to advance from the Spark phase to the ADVANCE phase of the AI-BOOST AI Challenge Competition with VINE-TALK, an AI-powered solution designed to make autonomous agricultural robots easier to operate through natural language.

The project addresses Challenge 1, led by Consorzio Intellimech / JOiINT LAB, which focuses on developing a generative AI-based natural-language mission generator for autonomous robots in agriculture.

Making agricultural robotics more accessible

Today, programming an autonomous robot to perform a new task often requires specialist robotics expertise. This can make it difficult for farmers and field technicians to adapt robotic systems to changing needs without relying on engineers.

VINE-TALK aims to address this barrier by allowing operators to instruct a robot using spoken or written language. For example, an operator could ask a robot to check the grapes in a particular vineyard row without having to write robot code.

The system interprets the instruction, identifies relevant objects and locations using visual information, and generates a mission using only actions supported by the robot. Before execution, the proposed mission is tested in a simulator and subjected to safety checks. If an instruction is unclear, the system can request clarification rather than guess. Human oversight and an emergency stop are also integral to the approach.

From vineyards to solar energy applications

VINE-TALK is being developed around a vineyard robotics use case, with the aim of demonstrating how natural-language interaction can make autonomous field operations more flexible and accessible.

The approach is also designed to extend beyond agriculture. Building on Eletoyia's experience in developing and operating field robotics, including its patented autonomous solar-panel cleaning robot, the project will explore how the same mission-generation approach could be applied to solar-plant inspection.

This cross-domain application will help demonstrate the potential to adapt natural-language robotic control to different platforms and operational environments.

The next phase

During the five-month ADVANCE phase, the project team plans to integrate VINE-TALK with the challenge's robotic platform, develop and evaluate its speech and grape-detection capabilities, test mission generation in simulation and controlled environments, and progress towards outdoor validation in a vineyard.

The evaluation will focus on instruction-recognition accuracy, mission execution success, response time, robustness to different ways of phrasing instructions, and safety validation before execution. The project's targets include at least 90% instruction-recognition accuracy, at least 85% mission execution success, and a response time of under five seconds from speech to a ready plan. These targets will be assessed during development and testing.

Advancing to the ADVANCE phase marks an important milestone for VINE-TALK as Eletoyia works towards making autonomous robotics more accessible through natural-language interaction.

Eletoyia looks forward to working with the AI-BOOST team and JOiINT LAB throughout the next phase of the competition.

Read more about Eletoyia's research and development work.