Citation: Peng, X.; Li, J.; Li, B.; Wu, J.
Constrained Image-Based Visual
Servoing of Robot Manipulator with
Third-Order Sliding-Mode Observer.
Machines 2022, 10, 465. https://
doi.org/10.3390/machines10060465
Academic Editors: Xiaochun
Cheng and Daming Shi
Received: 21 April 2022
Accepted: 9 June 2022
Published: 11 June 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Constrained Image-Based Visual Servoing of Robot
Manipulator with Third-Order Sliding-Mode Observer
Xiuyan Peng, Jiashuai Li, Bing Li * and Jiawei Wu
College of Intelligent System Science and Engineering, Harbin Engineering University, Nantong Street,
Harbin 150001, China; pengxiuyan@hrbeu.edu.cn (X.P.); heu_lijiashuai0810@163.com (J.L.);
mail_wujiawei1213@163.com (J.W.)
* Correspondence: libing265@hrbeu.edu.cn
Abstract:
A new image-based robot visual servo control strategy based on a third-order sliding-mode
observer (TOSM) model predictive control is proposed in this study. This new control strategy solves
the problem of robot visual servo control with system constraints and time-varying disturbances when
the camera and model of the robot manipulator are uncertain and the joint velocity is unknown. In the
proposed method, the joint velocity and system centralized uncertainties are estimated simultaneously
based on a third-order sliding-mode observer, and the image-based visual servoing problem is
transformed into a nonlinear optimization problem based on a model predictive control method
considering both visibility constraints and actuator constraints, which minimizes the predicted
trajectory cost function to generate the control signal for each cycle. Simulations were carried out to
verify the effectiveness of the proposed control scheme.
Keywords:
visual servoing; third-order sliding-mode observer; model predictive control; robot
manipulator
1. Introduction
To make robots more intelligent and flexible, they are often equipped with vision
sensors as tools for interacting with the outside world. In recent decades, robotic visual
servoing has been fully researched and widely used in various areas, such as robotic produc-
tion logistic [
1
], robotic service [
2
,
3
], robotic navigation and exploration [
4
,
5
]. According
to the different feedback information from the camera, the visual servoing system can
be divided into image-based visual servoing [
6
,
7
], position-based visual servoing [
8
,
9
],
and hybrid visual servoing [
10
,
11
]. Refs. [
6
,
7
] adopted the image-based visual servoing
method, which takes the deviation between the current and desired image features as
the control deviation. The robustness of different position-based visual servoing systems
was discussed in [
8
,
9
], where the control deviations are the deviations of relative pose
between the end-effector and the target in the Cartesian coordinate system. The hybrid
visual servoing method combines the above two visual servoing methods, and the control
deviation is composed of two-dimensional (2D) and three-dimensional (3D) deviations.
Refs. [
10
,
11
] discussed the applications of the hybrid visual servoing method in mobile
robots and parallel robots, respectively. Among these, image-based visual servoing has
been the most widely studied and is discussed in this study.
In the image-based visual servo system, the hand-eye mapping relationship is rep-
resented by the dynamic relationship between the velocity of the feature points and the
joints of the manipulator, which is usually expressed as a Jacobian matrix. Because the
Jacobian matrix contains the robot kinematics model and internal and external parame-
ters of the camera, the traditional Jacobian matrix requires a tedious calibration process,
and accurate calibration results are difficult to obtain in practical application scenarios.
Uncalibrated visual servoing directly defines the error in the image space according to
the image features of the manipulator end effector and target and estimates the Jacobian
Machines 2022, 10, 465. https://doi.org/10.3390/machines10060465 https://www.mdpi.com/journal/machines
