Article
Development of an Artificial Intelligence System for the
Automatic Evaluation of Cervical Vertebral Maturation Status
Jing Zhou
1
, Hong Zhou
1
, Lingling Pu
1
, Yanzi Gao
1
, Ziwei Tang
1
, Yi Yang
1
, Meng You
2
, Zheng Yang
3
,
Wenli Lai
1
and Hu Long
1,
*
Citation: Zhou, J.; Zhou, H.; Pu, L.;
Gao, Y.; Tang, Z.; Yang, Y.; You, M.;
Yang, Z.; Lai, W.; Long, H.
Development of an Artificial
Intelligence System for the Automatic
Evaluation of Cervical Vertebral
Maturation Status. Diagnostics 2021,
11, 2200. https://doi.org/10.3390/
diagnostics11122200
Academic Editor: Keun Ho Ryu
Received: 21 October 2021
Accepted: 21 November 2021
Published: 25 November 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1
State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases,
Department of Orthodontics, West China Hospital of Stomatology, Sichuan University,
Chengdu 610041, China; nkuzhoujing@163.com (J.Z.); zh17736839697@163.com (H.Z.);
dangdapll@163.com (L.P.); yanmei100@163.com (Y.G.); tziwei2016@163.com (Z.T.);
yy18583233717@163.com (Y.Y.); wenlilai@scu.edu.cn (W.L.)
2
State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases,
Department of Oral Radiology, West China Hospital of Stomatology, Sichuan University,
Chengdu 610041, China; youmeng@scu.edu.cn
3
State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases,
Department of General Dentistry, West China Hospital of Stomatology, Sichuan University,
Chengdu 610041, China; hxkqyx@163.com
* Correspondence: hlong@scu.edu.cn
Abstract:
Background: Cervical vertebral maturation (CVM) is widely used to evaluate growth
potential in the field of orthodontics. This study is aimed to develop an artificial intelligence (AI)
system to automatically determine the CVM status and evaluate the AI performance. Methods: A
total of 1080 cephalometric radiographs, with the age of patients ranging from 6 to 22 years old, were
included in the dataset (980 in training dataset and 100 in testing dataset). Two reference points
and thirteen anatomical points were labelled and the cervical vertebral maturation staging (CS) was
assessed by human examiners as gold standard. A convolutional neural network (CNN) model was
built to train on 980 images and to test on 100 images. Statistical analysis was conducted to detect
labelling differences between AI and human examiners, AI performance was also evaluated. Results:
The mean labelling error between human examiners was 0.48
±
0.12 mm. The mean labelling error
between AI and human examiners was 0.36
±
0.09 mm. In general, the agreement between AI results
and the gold standard was good, with the intraclass correlation coefficient (ICC) value being up
to 98%. Moreover, the accuracy of CVM staging was 71%. In terms of F1 score, CS6 stage (85%)
ranked the highest accuracy. Conclusions: In this study, AI showed a good agreement with human
examiners, being a useful and reliable tool in assessing the cervical vertebral maturation.
Keywords:
artificial intelligence; cervical vertebral maturation; skeletal age; deep learning; convolutional
neural network; orthodontics
1. Introduction
Dental malocclusion, with a prevalence of 20–83% among both adolescents and adults,
manifests as misaligned teeth resulting in poor masticatory function and esthetic prob-
lems [
1
–
4
]. Of particular, among adolescents, early interventions could eliminate or in-
tercept the development of malocclusion [
5
], e.g., mandibular advancement therapy for
adolescents with mandibular retrusion [
6
]. To ensure the success of early interventions,
meticulous and correct assessment of growth potential and the timing of growth spurt is
very important. Traditionally, skeletal age was used to assess the growth potential, among
which hand-wrist bone age and cervical vertebral maturation (CVM) staging were widely
used by dental practitioners [
7
,
8
]. Since hand-wrist radiographs require additional radio-
graphic examinations, both orthodontists and patients are reluctant to use this method in
Diagnostics 2021, 11, 2200. https://doi.org/10.3390/diagnostics11122200 https://www.mdpi.com/journal/diagnostics
