Articles
Semantic explanation and navigation for greenhouse robotics systems
Article number
1433_12
Pages
95 – 102
Language
English
Abstract
Robots moving around in an open, semi-structured greenhouse environment need to avoid collisions with humans and other objects.
In this paper, we focus on the challenge of dealing with unexpected but foreseeable hazardous situations for a moving robot.
To deal with such situations, we developed the Semantic Explanation & Navigation System (SENS). SENS uses the recently developed standard common greenhouse ontology (CGO) that represents knowledge and relations about the context of the robot’s situation and possible actions.
SENS uses visual object recognition to detect an object and its location, reasoning to maintain a CGO-based knowledge graph & infer more details about the situation, and an operator dashboard providing explainability.
The operator interprets the explanation and gives instructions to the robot on how to continue operation.
Using a tag- and beacon-based indoor positioning system, the robot is aware of the exact position of tagged objects, also outside of the field of view.
We tested several unexpected situations including detection of harmless tools, dangerous tools, standing and lying humans.
In each situation the robot derived the object type, the situation’s severity, a message class (info, warning, alarm) and a next action proposal (bypass or turn-around). These experiments indicate that understanding of the robot’s status and context increased.
Next steps include more challenging scenarios with multiple autonomous systems and improving instructions to the robot using structured natural language commands through SENS.
In this paper, we focus on the challenge of dealing with unexpected but foreseeable hazardous situations for a moving robot.
To deal with such situations, we developed the Semantic Explanation & Navigation System (SENS). SENS uses the recently developed standard common greenhouse ontology (CGO) that represents knowledge and relations about the context of the robot’s situation and possible actions.
SENS uses visual object recognition to detect an object and its location, reasoning to maintain a CGO-based knowledge graph & infer more details about the situation, and an operator dashboard providing explainability.
The operator interprets the explanation and gives instructions to the robot on how to continue operation.
Using a tag- and beacon-based indoor positioning system, the robot is aware of the exact position of tagged objects, also outside of the field of view.
We tested several unexpected situations including detection of harmless tools, dangerous tools, standing and lying humans.
In each situation the robot derived the object type, the situation’s severity, a message class (info, warning, alarm) and a next action proposal (bypass or turn-around). These experiments indicate that understanding of the robot’s status and context increased.
Next steps include more challenging scenarios with multiple autonomous systems and improving instructions to the robot using structured natural language commands through SENS.
Authors
J.G. Fernández, P.B.U.L. de Heer, J. van Oort, J.P.C. Verhoosel, M.R.A. van Vliet
Keywords
human-robot interaction, knowledge modelling, reasoning, hybrid AI, artificial intelligence, computer vision, symbolic AI, robot-robot interaction
Online Articles (41)