The ultimate goal of the observation mechanism is to be able to know at all (or most) of the time what is the current manipulation process and what is the visual relationship between the hand and the object. The fact that the observer will have to move in order to keep track of the manipulation process, makes one think of the stabilizability principle for general DEDS as a model for the tracking technique that has to be performed by the observer's camera.
In real-world applications, many manipulation tasks are performed by
robots, including, but not limited to, lifting, pushing, pulling, grasping, squeezing, screwing and unscrewing of machine parts. Modeling all the possible
tasks
and also the possible order in which they are to be performed is possible to
do within a DEDS state model. The different hand/object visual relationships
for different tasks can be modeled as the set of states . Movements of
the hand and object, either as 2-D or 3-D motion vectors, and the positions
of the hand within the image frame of the observer's camera can be thought
of as the events set
that causes state transitions within the
manipulation process. Assuming, for the time being, that we have no direct
control over the manipulation process itself, we can define the set of
admissible control inputs
as the possible tracking actions that can be
performed by the hand holding the camera, which actually can alter the visual
configuration of the manipulation process (with respect to the observer's
camera). Further, we can define a set of ``good'' states, where the visual
configuration of the manipulation process enables the camera to keep
track and to know the movements in the system. Thus, it can be seen that
the problem of observing the robot reduces to the problem of forming
an output stabilizing observer (an observer that can always return to a set of ``good'' visual states)
for the system under consideration.
It should be noted that a DEDS representation for a manipulation task is by no means unique, in fact, the degree of efficiency depends on the designer who builds the model for the task, testing the optimality of a visual manipulation models is an issue that remains to be addressed. Automating the process of building a model was discussed in the previous section. As the observer identifies the current state of a manipulation task in a non ambiguous manner, it can then start using a practical and efficient way to determine the next state within a predefined set, and consequently perform necessary tracking actions to stabilize the observation process with respect to the set of good states. That is, the current state of the system tells the observer what to look for in the next step.