Applying artificial intelligence for the application of bridges deterioration detection system

ABSTRACT


INTRODUCTION
Bridges are a very important part of transport infrastructure, they are very expensive to construct and maintain, there are many factors leading to bridge deterioration.There have been bridge collapsing incidents leading to serious consequences in terms of lives.By replicating personal self and conscience processes, bridge degradation detection systems (BDDs) deal with real-time sensing, description, and assessment of the security and efficacy evolving of components.This system is comprised of multiple of sensors, data collecting devices, flow information, a network enabling data processing, fault diagnosis and efficiency prognosis, a graphical user interface, and a software [1]- [5].
Recently, many researchers have dedicated themselves to expanding computer technology, deep learning, and computer vision for its excellent advantages in artificial intelligence (AI) [6]- [10], computer vision [11] and internet of things (IoT) technologies [12]- [15].They tend to use artificial intelligence to discover and perform theorems, speculations, and schemes, and their joinings will make it possible for the accumulation of much more information at a cheap rate, actually results in innovative approaches to condition -based determination and prediction by resolving numerous complicated problems in standard sensing, project addressed, safety assessment, and reliability test.These solutions can assist the system in recognizing a significant inherent progression in the structural behavior in real-world scenarios.Several EU-funded initiatives have previously produced guidelines and suggestions to address the problems and possibilities of bridge structural monitoring and degradation detection [16], [17].In Korea and Japan, many long-span bridges have applied the monitoring system to monitor the bridge status, but the current system is complicated to use.In Vietnam, many big bridges have been applied to monitor such as Can Tho, Bai Chay, Thuan Phuoc, ... to observe the bridge situation but have not combined monitoring all bridges in one center.
Techniques for detecting bridge degradation include both physical and nonphysical measures.The stochastic, non-parametric, and auto-regressive frameworks [18] were utilized to investigate a physical structure and anticipate reactions that used modeling.The vibration-based damage detection techniques [19] enable damage identification by comparing actual rates and conventional forms parameters inside the model.Furthermore, the convolutional neural network (CNN) has gained popularity and find excellent, and CNN is a new type of neural network that employs modern theory.CNN is amongst the most latest advancements in the field of image processing [7], language classification [6], [8], and natural speaker recognition [20], [21].As a result, CNN applications have successfully concentrated on focusing primarily by using directed vibration analysis.
In this paper, we design a monitoring system for parameters and warning of roads on multiple bridges at the same time, each bridge is considered a monitoring node, each monitoring node will be configured as well.As equipment differs depending on bridge type, data at nodes are encapsulated and transmitted, stored in the cloud to monitor status and issue alerts at all bridges through zero technology wire for managing and exploiting the current monitoring system and connecting between monitoring systems in a large area.Sensors and cameras are installed on various parts of the bridge.An alert should be given to the management center for taking precautions whenever there are parameters exceed the threshold value of the system.The following are the key aspects of this work: 1) first, we use convolution neural networks to recognize and predict the state on the bridge from the sensors; 2) we employed IoT technology to monitor on website or remote app of bridges using Google Firebase cloud computing facility to help the management center to monitor multiple bridges at the same time through the internet in different conditions and network quality to ensure data collection center is not interrupted at any time, information quality is always ensured; 3) finally, the connection between the application and Firebase will be made.So users can monitor the parameters to know the current bridge status through sensors parameters.
The remainder of the paper is carried out as follows.The suggested design of the bridge deterioration detection system, which has the primary purpose of damage identification, is shown in section 2. We developed an embedded system utilizing a Raspberry Pi application to perform the detect object procedure, with the acceleration and strain gauge data are given in a line chart and numeric value in section 3, and the testing analysis discussed in section 4. Finally, section 5 concludes the overall observations.

THE ARCHITECTURE OF THE PROPOSED BRIDGE DETERIORATION DETECTION SYSTEM
When the parameters outperform the system's sensors threshold value, the bridge degradation detection system's main job is always to identify the damage index.Data gathering, artificial intelligence, an alert system, and monitoring are only a few of the modules that carried out the key task.Figure 1 depicts the monitoring and alerting structure for bridges, which is explained as follows: -Strain gauge and accelerometer sensors: strain gauge and accelerometer of the bridge can be measured by the values of sensors.When these parameters exceed the sensor's threshold value it happens to affect the bridge.The acceleration signals are distinguished in the region and a strain gauge sensor is used to detect the bending.As a result, the sensors generate accelerate and strain gauge data, whereby the signal is transferred to the collecting data module.-Data collection process: The data acquisition module collects the strain gauge and accelerometer signals.
The MCP3204 is a 12-bit analog to digital converter that is handled by the Raspberry embedded processors in this module.-CNNs were used to detect the bridge's unusual condition: The strain gauge and accelerometer data are sent into the inputs of a CNN algorithm to estimate the bridge damage index.The identification of the training/testing processes in the analysis output is at the heart of this CNN algorithm.Before passing the data to the CNNs algorithm, all of the input data should be standardized.The abnormal situation of the bridge is identified by the classification output via bridge-mounted sensors.The modules that are mentioned above are strengthened by designing an interface using Qt Designer software for local monitoring on HDMI monitors and designing a website.Moreover, local monitoring and alerting take place at the bridge, while remote mode allows users to view data from anywhere through the internet.Figure 2 depicts the architectural layout.
The architecture of the BDDs is shown in Figure 2 that includes local and remote monitoring and alerting mode.In bridges, the strain gauge and accelerometer are installed at specific points as in Figure 1.The data acquisition module gathers information from sensors.Moreover, bridge operators either monitor locally TELKOMNIKA Telecommun Comput El Control  Applying artificial intelligence for the application of … (Xuan-Kien Dang) 151 at the bridge's HDMI display or remotely via a computer linked to the internet using the Google Firebase cloud database.The two main functions of the system are as below: -Monitoring function: The monitor system shows the strain gauge and accelerometer data from data collecting.It helps in monitoring the status of data collection from the sensors.The acceleration and strain gauge data are shown in the module as both a line graph and a data field at the bridge.-Alert function: Whenever the identification outputs are in unusual states, the alert system informs the user that the bridge's status is not safe and sounds an alarm.

The image pre-processing
The first step to address is the pre-processing block including three image processing stages.The beginning action is to read the RGB picture and transform it to grayscale.As a result, the three-dimensional A grayscale image is transformed to a feature vector in the second phase, which comprises all of the image's representative features.Here, the vector can have n dimensions, but we only need one to save processing intervals.The last phase converts the dataset into the characteristics space, where zero represents the mean value and adjusts it to the processing library's required specifications.

Training process
In this study, first, we used the SSD300 network SSD version, so the input photo was resized to 300x 300x3 pixels.Then the image is filled into the SSD network with each image having an 8732-default box, then take out the image with the confidence of 200 bounding boxes out of 8732.Next, use the fast non-maximum suppression algorithm to get the bounding box and low confidence is eliminated.Permanently, we use the threshold to get the necessary information of the image as shown in Figure 3.We have the loss function [7], which is comparable to the approach used in other studies: The ( 1) related to the default boxes: where ;    = {1,0} is a category P indication for matching the i-th default box to the j-th ground truth box.In (4) gives the created scale of the basic boxes [21] for each convolution layer: We selected   is 0.25 and   is 0.95 (s k is 0.1, 0.2, 0.375, 0.55, 0.725.0.9 means 30, 60, 112.5, 165, 217.5, 270 pixels input image (875x656x3)).The algorithm is planned in such a way that the total number of tensors that must be loaded into memory is kept to a minimum.In most situations, it looks through all possible computing orders Σ(G) and chooses the smallest one.
Where: R(i, π, G) is the list of intermediate tensors that are connected to any of πi. . .πn nodes, |A| represents the size of the tensor A, and size(πi) is the total amount of memory needed for internal storage during operation i.Where R(i, π, G) is a listing of transitional convolution layers related to any of the πi. . .πn nodes, | A | is the size of the tensor A, and size(πi) is the total of memory required for storage capacity throughout process i.
As illustrated in Figure 4, we utilize the Google Colab GPU to train the model [10] using input data of 480 pictures for training and 160 images for testing.Moreover, we require an incredibly powerful setup (GPU/CPU) for embedded systems employing Raspberry Pi apps running on smartphones to create these models for practical systems, such that this hardware performs: To detect and recognize, extract selected features of maps including apply convolution filters, the output is labeled with the item, and the confidence level is shown in percentages.The hardware of a BBDs-V1 system consists of accelerometer sensors and strain gauge sensors that are analog signals converted to digital signals through the MCP3204 12-bit analog-to-digital converter (ADC) IC which communicates with the Raspberry Pi by SPI protocol as shown in Figure 5.These sensor signals are processed using a CNN, the Raspberry gives an alert by a buzzer and a touch screen in a station.In addition, the alarm signal is also transmitted to the cloud through Firebase Google for monitoring on the website.The algorithm is performed according to the following steps: Step 1: Collect sensor value pictures from the Raspberry embedded computer, save 1 image with 480 saved photos in succession after 5s, and use the image as an input to the CNN pre -process in Figure 3.

Experimental
Parameters are set through a data set consisting of 2 cases, Normal and Danger.Normal status includes 315 images and Danger status with 165 images taken from experiments.In this study, the signal from the analog sensor signals is collected by the MCP3204 12-bit analog-to-digital converter (ADC) and sends to the Google Firebase database via the Raspberry Pi control board.Firebase provides access to real-time databases.In Firebase, information is stored as JavaScript object notation (JSON) and synchronized in real-time.Later we designed a website (Figures 6 and 7) to monitor the status of the bridge using web HTML technology developed on the front-end Angular architecture and provided with data through the connection between firebase and the website.When the system decides that no unsafe (abnormal) states exist, a timer will then be activated, and the clock will start.During the active timer duration, switch modes and countdown timer functions will be stopped after 5 seconds to allow for the next phase.However, if unsafe conditions arise in the system during the countdown (no physical effect on the system is required), the timer is reset, the system returns to its normal state, and the system supervisor can operate and record the status using the system function keys.The test recognized a hazardous situation in the experiment, and the outcome was as shown in Figure 7.By using the fasst non-maximum suppression method, the output dependability is maximum; the system provided better results in terms of quality, processing speed, and runtime.

CONCLUSION
The CNN algorithm was investigated and used for the bridge degradation detection system in this study (BBDs).The designed system uses internet of things (IoT) and artificial intelligence (AI) technologies to monitor remote bridges on websites and apps via the Google Firebase cloud database based on Raspberry Pi 4 hardware.Experimental results showed that the proposed system based on CNN can work stably and concur in actual conditions.The system's ability to function reliably and in accordance with the specified aims was demonstrated in the first phase of successful laboratory testing.

Figure 3 .
Figure 3. Structure of SSD for BDDs

Figure 4 .
Figure 4. Images input data collection

Step 2 :Figure 5 .
Figure 5.The hardware of a BBDs: (a) BBDs tested on HCM City University of Transport and (b) the structure of BBDs that have been established

Figure 6 .
Figure 6.The website to monitor the status of the bridge: (a) image inputs and (b) testing-no alarm stage

Figure 7 .
Figure 7.The website to monitor the status of the bridge: (a) image input and (b) testing -alarm stage (dangerous states) is the smooth  loss between the anticipated and ground-truth box parameters.  is the certainty loss over multiple classifications (c) as in(3).

Table 1 .
Distribution of frames, formed from the sensors signals into the dataset