Vehicle teleoperation is a capability that enables a human operator at a control station to control and operate a vehicle in a remote location through commands via input/control devices, receiving feedback from displays.
The operator and the vehicle are separated by a barrier (environment, distance, time, etc.) and exchange information via a communication link. These vehicles generally are equipped with sensors, actuators, and often some level of autonomy.
Vehicle teleoperation is used when necessary to operate in a hazardous or difficult reach environment and when the primary task is exploration, observation, or reconnaissance. It is also often used to reduce mission costs and to avoid loss of life.
Vehicle teleoperation applications include reconnaissance, surveillance, target acquisition (RSTA), exploration, remote mining, inspection, facility security, and entertainment.
Vehicle teleoperation has several characteristics that distinguish it from the simple remote control (i.e., line-of-sight radio control) and other types of teleoperation (e.g., telemanipulation).
- First, vehicle teleoperation demands reliable navigation. Since vehicles are often deployed in unfamiliar or unstructured environments, navigation problems may lead to system failure or loss.
- Second, vehicle teleoperation requires efficient motion command generation. In many cases, task performance is directly correlated to how well a vehicle moves.
- Finally, vehicle teleoperation calls for accurate interpretation of sensor data. Considering that most applications focus on exploration, reconnaissance, or observation, it is clear that misinterpreting or misjudging the remote environment will cause difficulties.
Vehicle teleoperation is concerned with three basic challenges: figuring out where the vehicle is and what state it is in, determining where it should go, and moving it there while avoiding obstacles and collisions.
Besides, there are various task-specific challenges such as observation, environment mapping, and deploying payload. These challenges can be difficult to solve, particularly if the vehicle operates in a hazardous environment with a limited (low bandwidth and/or high-delay) communication link. The difficulty increases when adding additional factors such as operator variation (training, skill, etc.), multiple vehicles, and moving (or malicious) hazards.
Conventionally, three system models are used to operate vehicles: direct control, supervisory control, and fully autonomous control.
1. Direct Control
The direct control model has been in wide use since the early 1900s. Traditionally, It is the most common method for performing vehicle. The operator directly operates the remote vehicle using hand-controllers (e.g., 3-axis joysticks) while monitoring video (from vehicle or worksite cameras) on one or more displays.
A great many vehicles, including aircraft (RPVs, UAVs), ground transport (UGVs), and submersibles (ROVs), have all been operated using direct control. In fact, for many of these vehicles, this approach continues to be state-of-the-art.
Direct control is used because it is cheap and easy to implement, at least compared to the other system models. With direct control, the human closes the control loop. Although the robot (or interface) may assist in either (or both) the sensing or effector paths, the primary responsibility for perceiving the remote environment, making decisions, and executing the plan rests with the human.
It is well known that direct control can be problematic. Because all control decisions depend on the human, system performance is directly linked to human capabilities. Many factors, including sensorimotor limits, knowledge, skill, training, etc., all play a role in how the system functions. Other factors, such as control station design and communication bandwidth, may also significantly influence a direct control system’s efficacy.
2. Supervisory Control
Supervisory control, originally called meta-control, was developed in the 1960s to characterize operators functioning in discontinuous control loops. The supervisory control concept appeared as part of research on how earth-based operators might teleoperate lunar vehicles. The term supervisory control is derived from the analogy between a supervisor’s interaction with subordinate staff.
To effect supervisory control, the operator divides a problem into a sequence of sub-tasks, which the robot then executes on its own. To date, the majority of research in supervisory control has focused on process control and telemanipulation. With supervisory control, the human interacts with the robot through the user interface.
Given a goal to achieve, the robot performs either a minor or major fraction of control, depending on its autonomy level. The more competent a robot is, the longer it will operate autonomously (i.e., exclusive of the human). Once the human has given control to the robot, he supervises task execution by monitoring interface displays. These displays generally show sensor data processed by one or more robot perception modules.
In the supervisory control system model, the human is not restricted to only to a supervisory role. Instead, he may intermittently control the robot by closing a command loop (sensor-perception-display-control-cognition-actuator), or he may control some variables while leaving the others to the robot. The former approach is known as “traded control” and the latter as “shared control.” With either approach, the human may choose to interact with the robot at different levels. For example, suppose the robot operates with a hierarchical controller. In that case, the human may close the control loop at a high, symbolic level (e.g., the top layer of robot perception-cognition or at a lower level (i.e., closer to the actuators).
Although supervisory control has been studied since the 1960s, there continue to be numerous open issues. First, various investigators have questioned whether removing the operator from active participation in the control loop makes detecting and diagnosing abnormalities more difficult. Additionally, it is unclear how to model a robot’s competency (its ability to act “correctly” in various situations) to facilitate operator training and task partitioning. Finally, there are uncertainties related to the sharing/trading of control and context switching, mainly when an operator controls multiple vehicles.
3. Fully Autonomous Control
Fully autonomous control describes robot systems that rely on humans to set high-level goals or to specify high-level strategy, but then execute independently of the human. Fully autonomous control is somewhat of a misnomer because it rarely, if ever, is fully automatic.
With this system model, the human gives high-level goals or tasks to the robot, which independently achieves them. Planning may be performed before execution, interleaved with execution, or continuously during execution. Since the days of the Stanford cart and SRI’s Shakey, fully autonomous control has been what research in “autonomous mobile robots” has aspired to achieve.
In a sense, fully autonomous control is just supervisory control taken to its extreme: the human gives a high-level, abstract goal, which the robot then achieves by itself. The difference between fully autonomous control and supervisory control is the nature of the goal.
In almost all supervisory control systems, goals are limited (e.g., “drive to point X without hitting anything”), and the majority of task planning is performed by the human. With fully autonomous control, goals tend to be more abstract (e.g., “cut the hay in this field”), and the robot has more responsibility for deciding how the task will be performed.
With fully autonomous control, as with supervisory control, the human interacts with the robot through the user interface. Once the human has specified a goal to be achieved, the robot operates independently (i.e., it closes all control loops). As the robot operates, the primary function of the human is to monitor execution via interface displays.
Limitations
Although all three of these system models have been used successfully, they all suffer from limitations. In direct control, tasks can only be performed while the human remains in the loop. Additionally, performance is limited by the operator’s capabilities and workload.
With supervisory and fully autonomous control, failure will occur whenever the human does not, or is unable to, recognize that the robot is ill-suited for the situation. In particular, whenever the human commands the robot to achieve a goal, the robot is forced to make decisions by itself. Consequently, the robot will always fail whenever any part of its autonomy (e.g., planning) is inadequate or inappropriate for performing the task.
Another limitation is that none of the models can accommodate a wide range of users. Direct control, for example, generally demands expert users because the difficulty and risk are great. The exceptions are when the vehicle is operating in a benign environment or if substantial safeguards exist. With the other models, operator variation has less impact because the robot can execute autonomously. However, adjusting the autonomy level to reflect the user’s needs or compensate for the user’s shortcomings is difficult.
An additional problem is that these models rely on humans’ ability to understand the remote environment and process. For the system to function, the human must accurately maintain situational awareness, ascertain what the robot is doing, and recognize when it has problems. Thus, if the user interface is poor or the human builds an incorrect mental model of what is happening, system damage or loss is likely to result.
