ABSTRACT: imitationof the human, by a robot, requires tremendous

ABSTRACT: Motion control is an important function for any human-robot interface which gives an opportunity to theusers to remotely operate the humanoid robot. The need for creating such humanoid robot arises from the diverse socioeconomic interests such as day to day activities of differently abled to lending a hand to the human race in nearlyinaccessible areas such as mines, areas exposed to radiation, war zones, etc. The recent developments in the field ofmotion controlled humanoid robot include the gaming interface Xbox 360 released by Microsoft for the purpose ofextreme involvement of the player in the gameplay, another noticeable area being depth image processing and skeletaltracking where the concept of motion tracking and motion mimic technology is being used has widely developed. TheHumanoid robot takes its inputs from the depth sensor in the form of image feed from the depth camera and outline thegestures tracked by it. The time constraints on various jobs make them grossly dependent on real time dataprocessing and execution. . Then the system communicates wirelessly with the self-designed Semi- Humanoid whichin-turn, imitates the operator with maximum accuracy.In this paper we propose the application of the digital world of motion tracking to the real world applications ofhumanoid robots with the help of the Kinect sensor. Humanoid robots seem to be the best alternative for human labor.Communication at work place environment greatly depends on body movements apart from direct audio signals.While verbal signals give brief idea of the details of the work to be done, it is the physical demonstration that tends tohave maximum impact. Much work has progressively been done in the fields of gesture recognition andimplementation. However, with advent of comparatively cheaper and easy to use depth imaging techniquessuch as Microsoft Kinect, the accuracy of gesture recognition can be increased to greater percentage. Perfect imitationof the human, by a robot, requires tremendous calculations and harmonious work of many different aspects of thesystem. A common form of mobile robot today is semi-autonomous, where the robot acts partially on its own, but thereis always a human in the control loop through a link i.e. Telerobotic. In this technique, there are no sensors on the bot,those it may use to take a decision. One of the possible methods of controlling such a bot is by amalgamation of depthimaging and skeletal tracking of the human operator. For extending the area of application of the bot, it has to bewirelessly controlled and equipped with fine maneuvering techniques. The signal transmission demands many-to-manycommunication with optimum use of bandwidth and data protection. If achieved with maximum accuracy thistechnology stands to be of great applications at outer space exploration, remote surgeries, hostile industrial andmilitary conditions, security purpose and of course, gaming and recreational activities.The approach to the problem was made on a calculated basis by first simulating on software and then Implementingon hardware. Broadly, the approach can be divided into four categories dealing with different works. Successiveamalgamations of the acquired results lead to the final goal. First step is to model the Humanoid. It is followeddesigning of the control architecture, which is the suite of control required for behavioral synchronization betweenhuman and the BOT. Next step caters to the acquisition and processing of image feed from the Kinect sensor. Thus, thecontrolling parameters are determined here. Final step involves wirelessly transmitting the inferred values of thecontrolling parameters to the Bot. the microcontrollers (mcu) used in the robot decodes these parameters and thusredirects the robot to react accordingly. The approach has been discussed thoroughly through the following paragraphs.A. Hardware Modelling of the HumanoidThe software design was readied with 6 Degrees of Freedom. The model was divided into various links and every linkwas modeled individually in the SolidWorks software. All the links were mated and ADAMS was used to test themodel under real physics by simultaneously solving equations for kinematics, statics, quasi – statics and dynamics formulti-body system. Specifically for simulation purpose, the virtual model is designed based on principles of kinematicmodeling by Denavit-Hartenberg methodology for robotic manipulators 4. There are total 18 Degrees of Freedom(DOF). Each arm has 5 DOF and each leg has 4 DOF. (Fig.1)Fig.1. Adam’s plant model of humanoid for simulation.The destination coordinates are fed into the ADAMS plant which has been configured to take input in the form ofangular velocities and output the current joint angles. The ADAMS plant is executed in MATLAB (Simulink)environment with a function file to control it. The stability of the humanoid was inspired from previous works donein this regard 5. There are two separate methods to control the motion of a humanoid i.e. DC (Direct Control)and CC (Command Control).Direct Control performs the conversion of 3D joint coordinates of the user to servoangles whereas the Command Control converts information to a single command for the robot’s understanding. Theinformation that is sent is the destination positions for the tool frame and the calculation of joint angles is left forthe model. For gain of maneuvering options in terms of speed and simplicity for most cases, the biped locomotion wasvoted against in favor of wheeled maneuvering system. Next step was to construct the hardware model. The materialfor construction was chosen to be Aluminum and 6 Servo Motors of the rating 60g/17kg/0.14sec were used. For thebase of the BOT, four motors, two each of the rating 12V, 250 rpm, 60Kgcm and 12V, 250 rpm, 30Kgcm were used.To regulate the speed of the driving motors, PWM was executed with Motor Drivers. The gross weight of the BOT is10kg (approx.) including the weight of the BECs (Battery Eliminator Circuits) and Power Supply. LiPo Batteries ofthe rating 11.1V, 6000 mAh 25~50C (Qty-2) and 11.1V, 850 mAh 20C (Qty-4) have been used for power supply. Theservos operate in the range 4.8V~6.0V and thus BECs were used to convert 11.1V to 6.0V.B. Control ArchitectureAs shown above (fig.3), for convenience this has been described in two parts, AVR Controller and ZIGBEECommunication. The AVR microcontroller that has been used is ATMEGA 32 which is an 8-bit MCU. This controlsthe servo motors and the driving motors. There are in-total 10 motors to be controlled (6 Servo + 4Shunt). Four MCUshave been used, out of which three control the servos and the remaining one controls the driving motors. Each MCUhas been interfaced with an LCD to continuously display its status. ZIGBEE is the standard that defines a set ofcommunication protocols for low-data-rate, very low power applications. We are using XBee RF modules forcommunication. These modules interface to a host device through a logic-level asynchronous serial port. The linkbetween XBee and ATMEGA 32 takes place via UART (Universal Asynchronous Receiver/Transmitter). The dataformat and transmission speeds are configurable. The AVR CPU connects to AVR UART via six registers i.e.Depth Imaging is the central point of this publication as it can be considered a giant leap in terms of boostingaccuracy while operator imitation. The sensor being used is Kinect which is a motion sensing input device byMicrosoft for XBOX 360 video game console. It is a horizontal bar connected to a small base with a motorized pivotand is designed to be positioned lengthwise. The device features an “RGB camera, depth sensor and multi-arraymicrophone running proprietary software”, which provide full-body 3D motion capture, facial recognition and voicerecognition capabilities. The depth sensor consists of an infrared laser projector combined with a monochrome CMOSsensor, which captures video data in 3D under any ambient li ght conditions. The sensing range of the depth sensoris adjustable, and the Kinect software is capable of automatically calibrating the sensor based on the physicalenvironment, accommodating for the presence of the user.Fig.3. Kinect cameraThird party software and open source drivers were implemented to gain compatibility with MATLAB. At the heart ofKinect, lies a time of flight camera that measures the distance of any given point from the sensor using the time takenby near-IR light to reflect from the object. In addition to it, an IR grid is projected across the scene to obtaindeformation information of the grid to model surface curvature. Cues from RGB and depth stream from the sensorare used to fit a stick skeleton model to the human body. The reference frame for the Kinect sensor is a predefinedlocation approximately at the center of the view. The horizontal right hand side of observation denotes the x axis andtransverse axis denotes y axis. The z axis is the linear distance between the Kinect and user, the farther the use thehigher the z value. We monitor the real time positions and orientations of 15 different data points in the user’s body,namely: Head, Neck, Left Shoulder, Left Elbow, Left Hand, Right Shoulder, Right Elbow, Right Hand, Torso, LeftHip, Left Knee, Left Foot, Right Hip, Right Knee and Right Foot. The sensor generates a 2D Matrix of 225x7dimension with the above mentioned data points for a single person. The seven columns are User ID, TrackingConfidence, x, y, z (coordinates in mm) and the next X, Y (the pixel value of the corresponding data point). Usingthe raw data points from Kinect, vectors are constructed and using those, the angles between different links and The data points are then used to calculate and derive values ranging from link length to angular velocity asdiscussed in the later part.As mentioned earlier, the technology defined by the Zig Bee specification is intended to be simpler and lessexpensive than other WPANs, such as Bluetooth. Zig Bee is targeted at radio-frequency (RF) applications thatrequire a low data rate, long battery life, and secure networking. One of the Xbee pair is connected to the PC usingXbee USB adapter board and the other one is interfaced to the Microcontroller. The data enters the module’s UARTfrom the serial port of the PC through the DI pin as asynchronous serial signal. The signal should idle be highwhen no data is being transmitted. Each data byte consists of a start bit (low), 8 data bits (least significant beingfirst) and a stop bit.Fig. 5. Diagrammatic analogy of communication methodNow that the information to be sent and method of communication is ready, the next step is finding out the memoryaddressing technique so that the data transmitted should be in synchronization with the data received. In order toaddress the 10 actuated joints with a single duplex channel of 8 bit word length we had to involve an addressingstrategy. The number of bits required to address 10 individual motors would ideally be 4 bits and to transmit 8 bits ofdata to each of the motors makes the word length of 12 bits which is not available. So instead of delivering the word ina single transmission the information is broken in 2 nibbles of 4 bit length and 2 consecutive transmissions are requiredto successfully transmit the 8 bit of data. With this split we had to assign 1 bit to distinguish between the higher and thelower nibble which again created a problem as the word length of a transmission is limited to 8 bits only. Twoseparate 4-bit address for each motor are used to send the higher and lower nibble of data from the source. Thisscheme allows us to address a 40 maximum of 256 different addressees and transmit information of 8 bits to eachone in 2 consecutive transmission. The analogy, for better understanding can be presented as in fig.6 shown aboveAfter the reception of the control signal at the remotely located manipulators the microcontrollers controlling theservos decodes the servo angles from the control signal and implements it. Controlling a servo using a microcontrollerrequires no external driver like H-bridge only a control signal needs to be generated for the servo to position it in anyfixed angle. The control signal is 50Hz (i.e. the period is 20ms) and the width of positive pulse controls the angle.This implementation of the servo angle completes 1 cycle of the process as it ends with the beginning of the Kinectsensor inputting the joint information to the control system. Thus, in this way the behavioral synchronization betweenan operator and the Humanoid robot is achieved.The dynamic process of behavioral synchronization through a humanoid begins with derivation of real timeinformation about the joints of the user be it prismatic (linear) or revolute (rotational). With degree of freedom as highas 10 the task becomes extremely tedious and requires tremendous processing power. One of the major problems inmonitoring multiple joints information is the definition of the reference frame with respect to which informationabout the other frames can be derived. Even though the reference for individual joints may vary it is convenient toprocess all the joints with a single reference point. The Kinect Sensor provides this information to the centralcontrol block using a combination of infrared imaging and RGB camera. The infrared laser creates a depth profile ofthe area in display and the RGB camera returns RGB matrix. The sensor generates a 2D matrix of 225 x 7 dimensionwith the above mentioned data points for a single person and a maximum tracking capability of 15 people. The sevencolumns are User ID, Tracking Confidence, x, y, z (coordinates in mm) and the next X, Y (the pixel value of thecorresponding data point). Using this information we calculate the joint angles in different planes between differentlinks as well as the link lengths between data points. Having raw joint information is not enough for further operations,because of that on the next step the vector between joints should be constructed. As an example below a description ofconstruction of vector between joints of right hand has been given: Connecting the right shoulder, right elbow and rightelbow and palm determines the angle for right shoulder.Using the x, y and z coordinates derived from Kinect sensor and using the formula for distance calculation the Linklengths are found.Length = {(xrs?xre)2+(yrs?yre)2+(zrs?zre)2}1/2After calculating the joint angles and link lengths, the next step in the behavioral synchronization would be to realizethese real time parameters in the desired form. This step includes conversion of the parameters to actual servo angles. Itis necessary for tuning purpose between the real angle values of joints and robot servo values. Another reason for thenecessity of this step is because the real human joint angle values range from 0~360°, when the robot servo motor canaccept values from 0~255. Now that the destination coordinates of the individual tool frames for the humanoid robothave been derived, two simultaneous steps follow, embedding the gesture movements into the virtual (software) andreal (hardware) model. Since each arm has 3DOF, the tool frame coordinates – x, y and z depends on 3 angles ?1, ?2and ?3 which can be expressed as:Here, ‘T’ is the transformation matrix that relates the coordinates with the joint angles.These angular velocities are then converted as discussed earlier and then transmitted to the Robot. The servo-motorsattached at designated joints thus move accordingly and operator imitation is achieved.IV. CONCLUSIONThe implemented architecture suggests simple, low cost and robust solution for controlling a semi-humanoid withhuman gestures. The complete setup requires the synchronization of software and hardware. The above mentionedproblems were faced for setting it up. This project can serve as a development platform for interaction between humanand BOT via gestures and with the quality of resources of products, more sophistication can be inculcated. Presentlybiped motion and holding mechanism in the hands are being worked upon. The legged motion is expected to enable therobot to maneuver over uneven terrains easily than the wheels. The picking/holding mechanism would increase itsapplicability in the industries by manifolds. Thus the greater goal of attaining better man-machine teams for dynamicenvironments can be realized with perfection.ACKNOWLEDGMENTWe sincerely thank our faculty advisor Professor Himani goyal for her continuous guidance and support over the periodof our research.