Article
Temperature Control for Automated Tape Laying with Infrared
Heaters Based on Reinforcement Learning
Martin Römer *, Johannes Bergers, Felix Gabriel and Klaus Dröder
Citation: Römer, M.; Bergers, J.;
Gabriel, F.; Dröder, K. Temperature
Control for Automated Tape Laying
with Infrared Heaters Based on
Reinforcement Learning. Machines
2022, 10, 164. https://doi.org/
10.3390/machines10030164
Academic Editors: Kelvin K.L. Wong,
Dhanjoo N. Ghista, Andrew W.H. Ip
and Wenjun (Chris) Zhang
Received: 13 January 2022
Accepted: 18 February 2022
Published: 22 February 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Institute of Machine Tools and Production Technology, Technische Universität Braunschweig, Langer Kamp 19b,
38106 Braunschweig, Germany; j.bergers@tu-braunschweig.de (J.B.); f.gabriel@tu-braunschweig.de (F.G.);
k.droeder@tu-braunschweig.de (K.D.)
* Correspondence: martin.roemer@tu-braunschweig.de
Abstract:
The use of fiber-reinforced lightweight materials in the field of electromobility offers
great opportunities to increase the range of electric vehicles and also enhance the functionality of
the components themselves. In order to meet the demand for a high number of variants, flexible
production technologies are required which can quickly adapt to different component variants and
thereby avoid long setup times of the required production equipment. By applying the formflexible
process of automated tape laying (ATL), it is possible to build lightweight components in a variant-
flexible way. Unidirectional (UD) tapes are often used to build up lightweight structures according to
a predefined load path. However, the UD tape which is used to build the components is particularly
sensitive to temperature fluctuations due to its low thickness. Temperature fluctuations within
the production sites as well as the warming of the tape layer and the deposit surface over longer
process times have an impact on the heat flow which is infused to the tape and make an adaptive
control of the tape heating indispensable. At present, several model-based control strategies are
available. However, these strategies require a comprehensive understanding of the ATL system and
its environment and are therefore difficult to design. With the possibility of model-free reinforcement
learning, it is possible to build a temperature control system that learns the common dependencies of
both the process being used and its operating environment, without the need to rely on a complete
understanding of the physical interrelationships. In this paper, a reinforcement learning approach
based on the deep deterministic policy gradient (DDPG) algorithm is presented, with the aim to
control the temperature of an ATL endeffector based on infrared emitters. The algorithm was
adapted to the thermal inertia of the system and trained in a real process environment. With only
a small amount of training data, the trained DDPG agent was able to reliably maintain the ATL
process temperatures within a specified tolerance range. By applying this technique, UD tape can
be deposited at a consistent process temperature over longer process times without the need for a
cooling system. Reducing process complexity can help to increase the prevalence of lightweight
components and thus contribute to lower energy consumption of electric vehicles.
Keywords: tapelaying; temperature control; automation; machine learning; infrared heating
1. Introduction
With the raising demand for improving the range of electric vehicles, the vehicle
weight is increasing due to the higher number of battery cells that are installed in the
vehicles. At the same time, this additional weight also increases energy consumption, which
in turn has a negative impact on the vehicles’ range. Therefore, lightweight construction
of vehicle structures is becoming more and more important. Fiber-reinforced plastic
composites offer an opportunity to reduce the weight of structural components and at the
same time maintain or even increase their structural stability. In order to satisfy the demand
for an increasing number of variants, a process chain is being developed in the project
“Großserienfähige Variantenfertigung von Kunststoff-Metall-Hybridbauteilen” (English:
High-volume variant production of plastic–metal hybrid components, HyFiVe) with the
Machines 2022, 10, 164. https://doi.org/10.3390/machines10030164 https://www.mdpi.com/journal/machines
