基于视觉的内陆水道斜接闸门模型更新和评估Matlab代码.rar

# Vision-based model updating and evaluation of miter gates on inland waterways This GitHub repository is my code contribution to the paper: Vision-based model updating and evaluation of miter gates on inland waterways. The paper can be found here: https://www.sciencedirect.com/science/article/pii/S0141029623000883 We also released a conference paper at the International Workshop on Structural Health Monitoring (IWSHM) 2023 that focuses more on the vision-based displacements measurement part. A link to the conference paper will be posted here once it becomes available. If you use any part of this code, ideas, or figures below, please be sure to cite both papers!! I was the sole developer of several MATLAB scripts for both of these papers including: * Visualization script that displays the displacement and displacement errors as a heat map on an image of the structure (disp_heatmap.m) * Light normalization for all specified images in a folder (light_normalization.m) * Creates a video from images in a specified folder (createVideo.m) I was also the primary developer of this script with minor edits from my co-author Shuo Wang: * Animation script that displays how the displacements change over time along the height of the gate (miterGatePlotAnimation.m) Additionally, I heavily expanded upon and added new features to this script. * Robust KLT-based optical flow method for displacement measurement (motion_compensate_homography_klt_video.m) The original 407 line script was written by co-author Shaik Althaf V. Shajihan and can be found on his Github here. Shuo Wang also contributed to this original script. The original script performed feature-based multi-ROI (Region of Interest) tracking for two images with adjustments for camera motion via homography and produced a quiver plot between both images. Additionally, a user could manually select ROI feature points and then later save and load those feature points to reduce additional manual effort. Afterwards, I heavily expanded the script to include new features such as: * 51 user options with associated functionality to increase flexibility for a variety of image sets * Feature-based multi-ROI tracking for a series of images in a folder rather than just two images * Flexible plotting functions to allow the user to plot the displacement in a variety of ways including over time, per point, and per ROI * Ability to load specific static and nonstatic points of interest to track from an Excel file of pixel locations * Ability to load and save ROIs rather than specific feature points to allow for increased flexibility for feature selection for the same ROIs * Ability to undistort images for improved tracking * Light normalization for ROIs for improved tracking * Averaged displacements across time and for an ROI in order to better see trends and reduce the effect of outliers * Quiver plot videos rather than a single image * Increased modularity and improved readability by refactoring the code With these new features and expansions, the script was significantly enhanced for improved flexibility and robustness in a variety of field conditions often experienced by large civil infrastructure. Throughout this deep expansion process, both Shuo Wang and Shaik Althaf V. Shajihan made minor edits. Very detailed descriptions of what each script does and how it works can be found in the following sections. For references for this README, please see the papers and their references. Note: None of the data or results are given due to security reasons. If you would like a copy of either paper, please feel free to email me at: caseyjr2@illinois.edu ## Robust KLT-based optical flow method for displacement measurements The main input to this script is a folder of images of the structure. The images were taken at regular intervals using a feature of the camera used. The photos taken were analyzed using computer vision techniques to track the movement of the gates. The computer vision method implemented uses a Kanade-Lucas-Tomasi (KLT) feature tracker to track object movement by measuring pixel displacements. This script also utilizes several techniques to make the pixels easier to track such as light normalization and image homography to compensate for camera motion/drift. Additionally, this script was developed to be very robust for the variety of image sets collected and allow the user flexibility to extract the information desired without additional computational cost. The figure below shows the general steps of the script, where red ovals represent user input. <img src="readme_imgs/klt_method.JPG" alt="klt method" width="600"/> Firstly, the images are undistorted to correct any camera lens distortion. Camera lens distortion makes lines that are supposed to be straight appear curved, which can cause errors in the pixel tracking. To calculate the parameters of the camera lens distortion for our specific cameras, several experiments involving taking photos of a checkerboard pattern at different angles were run and the Camera Calibrator App in MATLAB was used to estimate camera intrinsics, extrinsics, and lens distortion parameters in order to remove the effects of lens distortion from an image. (Note that to better approximate the lens distortion, the images of the checkerboard pattern need to be captured at a distance roughly equal to the distance from the camera to the objects of interest, i.e., the miter gate, and as a general rule, the checkerboard should fill at least 20% of the captured image. Therefore, for this project, a very large checkerboard pattern is needed, which is hard to achieve. Thus, the conducted experiment can only be an approximation for undistortion of the miter gate images.) Next, the static and non-static regions of interest (ROI's) along with scale factors for each must be defined. Static ROI’s are regions in the image sets that should not be moving such as a back wall or kerb. These regions will be used to calculate homography later to more accurately account for camera vibration. On the other hand, non-static ROI’s are regions that the user is interested in tracking that are not known as static. Tracking these specific non-static ROI’s rather than performing full optical flow saves a significant amount of computational time and energy since not all of the pixels will be tracked. For both types of ROI's, the user can define them by either: 1.) Selecting the ROI's manually, 2.) Loading ROI's they have already manually selected, or 3.) Loading a .csv file with pixel coordinates for each point they want to investigate. For option 3, these pixel coordinates are then converted into square ROI's with a user specified half length. For the paper, static ROI's were chosen manually and non-static ROI's were chosen via option 3. Along with defining these ROI's, one must also input a scaling factor for each, so that the pixel displacements can later be converted to physical displacement in millimeters. If the user loads previously manually selected ROI's, then the corresponding scaling factors are loaded along with them. Otherwise, three options were implemented for scaling: 1.) The user draws a line for a known distance near the ROI and inputs the value, 2.) The user inputs the camera distance to the ROI, 3.) A .csv file with 3D global coordinates corresponding to the ROI's is loaded and used to automatically calculate the scaling factors. The figure below shows an example of selected ROI’s, where the green boxes show static ROI’s and the red boxes show non-static or "test" ROI’s. In this case, the back wall is used as the static ROI’s and the weld access holes and bolts near the miters are used for the non-static ROI’s. <img src="readme_imgs/rois_img.jpg" alt="rois img" width="600"/> Afterwards, local light normalization is implemented to allow features to be more easily distinct and therefore easier to track. It also helps eliminate shadows and sudden changes in li