In highly constrained settings, e.g., a tentaclelike medical robot maneuvering through narrow cavities in the body for minimally invasive surgery, it may be difficult or impossible for a robot with a generic kinematic design to reach all desirable targets while avoiding obstacles. We introduce a design optimization method to compute kinematic design parameters that enable a single robot to reach as many desirable goal regions as possible while avoiding obstacles in an environment. Our method appropriately integrates sampling based motion planning in configuration space into stochastic optimization in design space so that, over time, our evaluation of a design's ability to reach goals increases in accuracy and our selected designs approach global optimality. We prove the asymptotic optimality of our method and demonstrate performance in simulation for (i) a serial manipulator and (ii) a concentric tube robot, a tentacle-like medical robot that can bend around anatomical obstacles to safely reach clinically- relevant goal regions.

In this work we present a novel, inductance-based system to measure and control the motion of bellows-driven continuum joints in soft robots. The sensing system relies on coils of wire wrapped around the minor diameters of each bellows on the joint. As the bellows extend, these coils of wire become more distant, decreasing their mutual inductance. Measuring this change in mutual inductance allows us to measure the motion of the joint. By dividing the sensing of the joint into two sections and measuring the motion of each section independently, we are able to measure the overall deformation of the joint with a piece-wise constant-curvature approximation. This technique allows us to measure lateral displacements that would be otherwise unobservable. When measuring bending, the inductance sensors measured the joint orientation with an RMS error of 1.1 °. The inductance sensors were also successfully used as feedback to control the orientation of the joint. The sensors proposed and tested in this work provided accurate motion feedback that would be difficult to achieve robustly with other sensors. This sensing system enables the creation of robust, self-sensing soft robots based on bellows-driven continuum joints.

This paper describes a framework of algorithms for the active localization and tracking of flexible needles and targets during image-guided percutaneous interventions. The needle and target configurations are tracked by Bayesian filters employing models of the needle and target motions and measurements of the current system state obtained from an intra-operative imaging system which is controlled by an entropy-minimizing active localization algorithm. Versions of the system were built using particle and unscented Kalman filters and their performance was measured using both simulations and hardware experiments with real magnetic resonance imaging data of needle insertions into gel phantoms. Performance of the localization algorithms is given in terms of accuracy of the predictions and computational efficiency is discussed.

In this paper we present a system to enhance the performance of feature correspondence based alignment algorithms for laser scan data. We show how this system can be utilized as a new approach for evaluation of mapping algorithms. Assuming a certain a priori knowledge, our system augments the sensor data with hypotheses ('Virtual Scans') about ideal models of objects in the robot's environment. These hypotheses are generated by analysis of the current aligned map estimated by an underlying iterative alignment algorithm. The augmented data is used to improve the alignment process. Feedback between data alignment and data analysis confirms, modifies, or discards the Virtual Scans in each iteration. Experiments with a simulated scenario and real world data from a rescue robot scenario show the applicability and advantages of the approach. By replacing the estimated 'Virtual Scans' with ground truth maps our system can provide a flexible way for evaluating different mapping algorithms in different settings.

It is particularly challenging to devise techniques for underpinning the behaviour of autonomous vehicles in surveillance missions as these vehicles operate in uncertain and unpredictable environments where they must cope with little stability and tight deadlines in spite of their restricted resources. State-of-the-art techniques typically use probabilistic algorithms that suffer a high computational cost in complex real-world scenarios. To overcome these limitations, we propose a approach that combines the probabilistic reasoning based on the target motion model offered by Monte Carlo simulation with long-term strategic capabilities provided by automated task planning. We demonstrate our approach by focusing on one particular surveillance mission, search-and-tracking, and by using two different vehicles, a fixed-wing UAV deployed in simulation and the "Parrot AR.Drone2.0" quadcopter deployed in a physical environment. Our experimental results show that our unique way of integrating probabilistic and deterministic reasoning pays off when we tackle realistic missions.

The correct grasp of objects is a key aspect for the right fulfillment of a given task. Obtaining a good grasp requires algorithms to automatically determine proper contact points on the object as well as proper hand configurations, especially when dexterous manipulation is desired, and the quantification of a good grasp requires the definition of suitable grasp quality measures. This article reviews the quality measures proposed in the literature to evaluate grasp quality. The quality measures are classified into two groups according to the main aspect they evaluate: location of contact points on the object and hand configuration. The approaches that combine different measures from the two previous groups to obtain a global quality measure are also reviewed, as well as some measures related to human hand studies and grasp performance. Several examples are presented to illustrate and compare the performance of the reviewed measures.

We consider the problem of assigning software processes (or tasks) to hardware processors in distributed robotics environments. We introduce the notion of a , which supports the adaptation of software to specific hardware configurations. Task variants facilitate the trade-off of functional quality versus the requisite capacity and type of target execution processors. We formalise the problem of assigning task variants to processors as a mathematical model that incorporates typical constraints found in robotics applications; the model is a constrained form of a multi-objective, multi-dimensional, multiple-choice knapsack problem. We propose and evaluate three different solution methods to the problem: constraint programming, a constructive greedy heuristic and a local search metaheuristic. Furthermore, we demonstrate the use of task variants in a real instance of a distributed interactive multi-agent navigation system, showing that our best solution method (constraint programming) improves the system's quality of service, as compared to the local search metaheuristic, the greedy heuristic and a randomised solution, by an average of 16, 31 and 56% respectively.

We present a framework for representing scenarios with complex object interactions, where a robot cannot directly interact with the object it wishes to control and must instead influence it via intermediate objects. For instance, a robot learning to drive a car can only change the car's pose indirectly via the steering wheel, and must represent and reason about the relationship between its own grippers and the steering wheel, and the relationship between the steering wheel and the car. We formalize these interactions as chains and graphs of Markov decision processes (MDPs) and show how such models can be learned from data. We also consider how they can be controlled given known or learned dynamics. We show that our complex model can be collapsed into a single MDP and solved to find an optimal policy for the combined system. Since the resulting MDP may be very large, we also introduce a planning algorithm that efficiently produces a potentially suboptimal policy. We apply these models to two systems in which a robot uses learning from demonstration to achieve indirect control: playing a computer game using a joystick, and using a hot water dispenser to heat a cup of water.

Recognising intent in collaborative human robot tasks can improve team performance and human perception of robots. Intent can differ from the observed outcome in the presence of mistakes which are likely in physically dynamic tasks. We created a dataset of 1227 throws of a ball at a target from 10 participants and observed that 47% of throws were mistakes with 16% completely missing the target. Our research leverages facial images capturing the person's reaction to the outcome of a throw to predict when the resulting throw is a mistake and then we determine the actual intent of the throw. The approach we propose for outcome prediction performs 38% better than the two-stream architecture used previously for this task on front-on videos. In addition, we propose a 1D-CNN model which is used in conjunction with priors learned from the frequency of mistakes to provide an end-to-end pipeline for outcome and intent recognition in this throwing task.

To avoid collisions, Micro Air Vehicles (MAVs) flying in teams require estimates of their relative locations, preferably with minimal mass and processing burden. We present a relative localization method where MAVs need only to communicate with each other using their wireless transceiver. The MAVs exchange on-board states (velocity, height, orientation) while the signal strength indicates range. Fusing these quantities provides a relative location estimate. We used this for collision avoidance in tight areas, testing with up to three AR.Drones in a area and with two miniature drones ( ) in a area. The MAVs could localize each other and fly several minutes without collisions. In our implementation, MAVs communicated using Bluetooth antennas. The results were robust to the high noise and disturbances in signal strength. They could improve further by using transceivers with more accurate signal strength readings.

The brain is perhaps the most advanced and robust computation system known. We are creating a method to study how information is processed and encoded in living cultured neuronal networks by interfacing them to a computer-generated animal, the Neurally-Controlled Animat, within a virtual world. Cortical neurons from rats are dissociated and cultured on a surface containing a grid of electrodes (multi-electrode arrays, or MEAs) capable of both recording and stimulating neural activity. Distributed patterns of neural activity are used to control the behavior of the Animat in a simulated environment. The computer acts as its sensory system providing electrical feedback to the network about the Animat's movement within its environment. Changes in the Animat's behavior due to interaction with its surroundings are studied in concert with the biological processes (e.g., neural plasticity) that produced those changes, to understand how information is processed and encoded within a living neural network. Thus, we have created a hybrid real-time processing engine and control system that consists of living, electronic, and simulated components. Eventually this approach may be applied to controlling robotic devices, or lead to better real-time silicon-based information processing and control algorithms that are fault tolerant and can repair themselves.

One of the key issues in space exploration is that of deciding what space tasks are best done with humans, with robots, or a suitable combination of each. In general, human and robot skills are complementary. Humans provide as yet unmatched capabilities to perceive, think, and act when faced with anomalies and unforeseen events, but there can be huge potential risks to human safety in getting these benefits. Robots provide complementary skills in being able to work in extremely risky environments, but their ability to perceive, think, and act by themselves is currently not error-free, although these capabilities are continually improving with the emergence of new technologies. Substantial past experience validates these generally qualitative notions. However, there is a need for more rigorously systematic evaluation of human and robot roles, in order to optimize the design and performance of human-robot system architectures using well-defined performance evaluation metrics. This article summarizes a new analytical method to conduct such quantitative evaluations. While the article focuses on evaluating human-robot systems, the method is generally applicable to a much broader class of systems whose performance needs to be evaluated.

We often specify tasks for a robot using temporal language that can include different levels of abstraction. For example, the command contains spatial abstraction, given that "floor" consists of individual rooms that can also be referred to in isolation ("kitchen", for example). There is also a temporal ordering of events, defined by the word "before". Previous works have used syntactically co-safe Linear Temporal Logic (sc-LTL) to interpret temporal language (such as "before"), and Abstract Markov Decision Processes (AMDPs) to interpret hierarchical abstractions (such as "kitchen" and "second floor"), separately. To handle both types of commands at once, we introduce the Abstract Product Markov Decision Process (AP-MDP), a novel approach capable of representing non-Markovian reward functions at different levels of abstractions. The AP-MDP framework translates LTL into its corresponding automata, creates a product Markov Decision Process (MDP) of the LTL specification and the environment MDP, and decomposes the problem into subproblems to enable efficient planning with abstractions. AP-MDP performs faster than a non-hierarchical method of solving LTL problems in over of path planning tasks, and this number only increases as the size of the environment domain increases. In a cleanup world domain, AP-MDP performs faster in over of tasks. We also present a neural sequence-to-sequence model trained to translate language commands into LTL expression, and a new corpus of non-Markovian language commands spanning different levels of abstraction. We test our framework with the collected language commands on two drones, demonstrating that our approach enables robots to efficiently solve temporal commands at different levels of abstraction in both indoor and outdoor environments.

This paper presents a scalable procedure for time-constrained planning of a class of uncertain nonlinear multi-robot systems. In particular, we consider robotic agents operating in a workspace which contains regions of interest (RoI), in which atomic propositions for each robot are assigned. The main goal is to design decentralized and robust control laws so that each robot meets an individual high-level specification given as a metric interval temporal logic (MITL), while using only local information based on a limited sensing radius. Furthermore, the robots need to fulfill certain desired transient constraints such as collision avoidance between them. The controllers, which guarantee the transition between regions, consist of two terms: a nominal control input, which is computed online and is the solution of a decentralized finite-horizon optimal control problem (DFHOCP); and an additive state feedback law which is computed offline and guarantees that the real trajectories of the system will belong to a hyper-tube centered along the nominal trajectory. The controllers serve as actions for the individual weighted transition system (WTS) of each robot, and the time duration required for the transition between regions is modeled by a weight. The DFHOCP is solved at every sampling time by each robot and then necessary information is exchanged between neighboring robots. The proposed approach is scalable since it does not require a product computation among the WTS of the robots. The proposed framework is experimentally tested and the results show that the proposed framework is promising for solving real-life robotic as well as industrial applications.

Assistive robot arms enable people with disabilities to conduct everyday tasks on their own. These arms are dexterous and ; however, the interfaces people must use to control their robots are . Consider teleoperating a 7-DoF robot arm with a 2-DoF joystick. The robot is helping you eat dinner, and currently you want to cut a piece of tofu. Today's robots assume a pre-defined mapping between joystick inputs and robot actions: in one mode the joystick controls the robot's motion in the - plane, in another mode the joystick controls the robot's - motion, and so on. But this mapping misses out on the task you are trying to perform! Ideally, one joystick axis should control how the robot stabs the tofu, and the other axis should control different cutting motions. Our insight is that we can achieve intuitive, user-friendly control of assistive robots by the robot's high-dimensional actions into low-dimensional and human-controllable . We divide this process into three parts. First, we explore models for learning latent actions from offline task demonstrations, and formalize the properties that latent actions should satisfy. Next, we combine learned latent actions with autonomous robot assistance to help the user reach and maintain their high-level goals. Finally, we learn a personalized alignment model between joystick inputs and latent actions. We evaluate our resulting approach in four user studies where non-disabled participants reach marshmallows, cook apple pie, cut tofu, and assemble dessert. We then test our approach with two disabled adults who leverage assistive devices on a daily basis.

We propose a new meta-module design for two important classes of modular robots. The new meta-modules are three-dimensional, robust and compact, improving on the previously proposed ones. One of them applies to so-called modular robot units, such as M-TRAN, SuperBot, SMORES, UBot, PolyBot and CKBot, while the other one applies to so-called modular robot units, which include Molecubes and Roombots. The new meta-modules use the rotational degrees of freedom of these two types of robot units in order to expand and contract, as to double or halve their length in each of the two directions of its three dimensions, therefore simulating the capabilities of Crystalline and Telecube robots. Furthermore, in the edge-hinged case we prove that the novel meta-module can also perform the scrunch, relax and transfer moves that are necessary in any tunneling-based reconfiguration algorithm for expanding/contracting modular robots such as Crystalline and Telecube. This implies that the use of meta-meta-modules is unnecessary, and that currently existing efficient reconfiguration algorithms can be applied to a much larger set of modular robots than initially intended. We also prove that the size of the new meta-modules is optimal and cannot be further reduced.

This paper presents a novel system for autonomous, vision-based drone racing combining learned data abstraction, nonlinear filtering, and time-optimal trajectory planning. The system has successfully been deployed at the first autonomous drone racing world championship: the . Contrary to traditional drone racing systems, which only detect the next gate, our approach makes use of any visible gate and takes advantage of multiple, simultaneous gate detections to compensate for drift in the state estimate and build a global map of the gates. The global map and drift-compensated state estimate allow the drone to navigate through the race course even when the gates are not immediately visible and further enable to plan a near time-optimal path through the race course in real time based on approximate drone dynamics. The proposed system has been demonstrated to successfully guide the drone through tight race courses reaching speeds up to and ranked second at the .

The Robotics Technology Branch at the NASA Johnson Space Center is developing robotic systems to assist astronauts in space. One such system, Robonaut, is a humanoid robot with the dexterity approaching that of a suited astronaut. Robonaut currently has two dexterous arms and hands, a three degree-of-freedom articulating waist, and a two degree-of-freedom neck used as a camera and sensor platform. In contrast to other space manipulator systems, Robonaut is designed to work within existing corridors and use the same tools as space walking astronauts. Robonaut is envisioned as working with astronauts, both autonomously and by teleoperation, performing a variety of tasks including, routine maintenance, setting up and breaking down worksites, assisting crew members while outside of spacecraft, and serving in a rapid response capacity.

Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception. A multitude of ideas and methods for how to accomplish this have already appeared in the past, their broader utility perhaps impeded by insufficient computational power or costly hardware. The history of these ideas, perhaps selective due to our perspectives, is presented with the goal of organizing the past literature and highlighting the seminal contributions. We argue that those contributions are as relevant today as they were decades ago and, with the state of modern computational tools, are poised to find new life in the robotic perception systems of the next decade.

The paper briefly outlines DLR's experience with real space robot missions (ROTEX and ETS VII). It then discusses forthcoming projects, e.g., free-flying systems in low or geostationary orbit and robot systems around the space station ISS, where the telerobotic system MARCO might represent a common baseline. Finally it describes our efforts in developing a new generation of "mechatronic" ultra-light weight arms with multifingered hands. The third arm generation is operable now (approaching present-day technical limits). In a similar way DLR's four-fingered hand II was a big step towards higher reliability and yet better performance. Artificial robonauts for space are a central goal now for the Europeans as well as for NASA, and the first verification tests of DLR's joint components are supposed to fly already end of 93 on the space station.

The exploration of new environments is a crucial challenge for mobile robots. This task becomes even more complex with the added requirement of ensuring safety. Here, safety refers to the robot staying in regions where the values of certain environmental conditions (such as terrain steepness or radiation levels) are within a predefined threshold. We consider two types of safe exploration problems. First, the robot has a map of its workspace, but the values of the environmental features relevant to safety are unknown beforehand and must be explored. Second, both the map and the environmental features are unknown, and the robot must build a map whilst remaining safe. Our proposed framework uses a Gaussian process to predict the value of the environmental features in unvisited regions. We then build a Markov decision process that integrates the Gaussian process predictions with the transition probabilities of the environmental model. The Markov decision process is then incorporated into an exploration algorithm that decides which new region of the environment to explore based on information value, predicted safety, and distance from the current position of the robot. We empirically evaluate the effectiveness of our framework through simulations and its application on a physical robot in an underground environment.