Algorithms-for-Real-Time-Endoscopy-Image-Processing-Pipeline-in资源-CSDN文库

需积分: 5 5 浏览量 2023-11-08 17:07:56 上传评论收藏 1.18MB PDF 举报

【Algorithms for Real-Time Endoscopy Image Processing Pipeline】在临床决策支持系统中，实时内窥镜图像处理管道的算法设计是关键。这些算法必须考虑到内窥镜图像的特点以及实时实现所需的计算效率。以下是对该主题的详细阐述：一、降噪算法降噪算法基于图像细节层次的检测。根据图像的不同细节级别，应用不同的噪声滤波器。这种方法旨在在保持图像信息的同时，有效减少噪声，提高图像质量。二、对比度增强算法非线性对比度增强算法被用来突出血管与背景之间的对比度，而不会过度强调噪声。这是非线性增强算法的一个主要缺点，因为它可能导致噪声的放大。通过精心设计，该算法能够在增强图像特征的同时，控制噪声的引入。三、定制色彩校正算法色彩校正算法考虑了用户的偏好，确保每个颜色坐标的最大误差不超过0.5%。这保证了图像颜色的准确性和一致性，对于医学图像尤其重要，因为颜色可能是诊断的关键因素。四、“马赛克”合成算法用于创建低细节图像的全景图，平均拼接误差小于0.75像素。这使得医生可以查看更广阔的视野，有助于发现可能被常规视图遗漏的病灶或异常。五、软件实现与性能这些算法的软件实现能够以约30帧/秒的速度处理4K内窥镜视频，满足实时处理的需求。这种速度对于临床应用至关重要，因为医生需要即时反馈以进行决策。六、临床决策支持系统（CDSS） CDSS是医疗视频系统发展的现代趋势，它整合了自动图像分析结果和医生的观察结果。这些系统利用内窥镜图像处理的输出，提供辅助诊断信息，帮助医生做出更精确的判断。例如，通过自动检测肿瘤、息肉或其他异常结构，CDSS可以提醒医生注意潜在问题，提高诊断的准确性。内窥镜图像处理涉及到多个步骤，包括降噪、对比度增强、色彩校正和图像合成，所有这些都对临床决策支持系统的效能有直接影响。高效、实时的图像处理算法能够改善医疗成像的质量，从而提升医疗诊断的效率和准确性。

资源推荐

资源详情

资源评论

DOI: 10.4018/IJERTCS.2019100103



Volume 10 • Issue 4 • October-December 2019







Alexandr A. Pozdeev, Saint Petersburg Electrotechnical University “LETI,” St. Petersburg, Russia

Nataliia A. Obukhova, Saint Petersburg Electrotechnical University “LETI,” St. Petersburg, Russia

Alexandr A. Motyko, Saint Petersburg Electrotechnical University “LETI,” St. Petersburg, Russia



A set of algorithms, taking in account endoscopic image features and computational cost for real-time

realization is proposed. A noise reduction algorithm is based on determining the level of detail in an

image fragment. For fragments with a different level of detail, different noise reduction filters are

used. The enhancement algorithm is based on nonlinear contrast enhancement which highlights the

contrast of vessels relative to the background without significant noise stressing, which is one of the

main disadvantages of nonlinear enhancement algorithms. The custom color correction algorithm

takes into account user preferences and provides a mean error less than 0.5% for each color coordinate.

The “mosaic” synthesis algorithm gets panoramic images of low detail images with a mean stitching

error less than 0.75 pix. The software realization of algorithms allows processing 4K endoscopic

video with a speed of about 30 fps.



Color Correction, Contrast Enhancement, Denoising, Highlights Removing, Infrared Channel, Inpainting,

Panoramic Images



Clinical decision support system (CDSS) represents a modern trend in the development of medical

video systems. Systems of this type integrate automatic image analysis results with the results obtained

by the physician, as well as use the information from the system database. This integration allows

for higher sensitivity and specificity of the diagnosis compared to cases of diagnoses being made

independently by a physician, or the system (Liedlgruber, & Uhl, 2011).

Thus, visual analysis conducted by the physician is an important part of CDSS. This demands

for high image quality for the physician to conduct analysis and, as a consequence, enables to ensure

high efficiency of visual analysis which requires to:

• Develop the algorithms to improve the quality of medical images, providing for a high level of

image ergonomics;

Volume 10 • Issue 4 • October-December 2019

• Develop the algorithms for the synthesis of medical images with new properties and with

increased diagnostic value.

The problem definition in this research is to develop the algorithms for image improvement and

visualization for real–time pipeline realization for CDSS in endoscopy (gastroscopy).

In this case the main requirements for the algorithms are the following:

1. High effectiveness of the algorithm, taking in account the features of endoscopic images. The

main features of endoscopic images which make popular algorithms of digital processing low

effective for application in pipeline are: uneven contrast (there are very dark and very bright

areas in the same image), large areas of image with very low amount of detail;

2. Each algorithm must be designed as a step of the endoscopic pipeline, which implies high

processing speed. The computational cost of algorithms must give possibility of real – time

pipeline realization, also the result of one step must be improved by the next step.

Additionally, the designed algorithms and software have to be adapted to real time processing

of endoscopic video data with high resolution (up to FHD, 4K) and frame rate (up to 30 – 60 fps).

The main pipeline steps can be divided in two groups. The first group must provide the obtainment

of quality images from a sensor. The algorithms of the second group are aimed at removing artifacts

caused by difficult observation conditions. Such conditions include the lack of light, complex shape

of objects under observation, organ muscular breathing, movement of the sensor, which leads to the

following artifacts: high presence of noise, nonlinear brightness and contrast characteristics, blur and

low sharpness. The critical procedures of this group are: noise reduction and contrast enhancement.

These tasks are especially difficult in endoscopy because of the fact the endoscopic images suffer

from various types of degradation (Campiho & Kamel, 2014). Moreover, these algorithms are the

most computationally expensive. Thus, the first part of our research includes the development of real-

time algorithms of the second group, such as noise removing, brightness and contrast enhancement,

taking into account the main features of endoscopic images.

The following new real-time algorithms for the endoscopic pipeline were proposed:

• The algorithm of noise removing, which is based on experimental results of modern noise

reduction algorithms assessment. The algorithm determines the level of high frequency in the

image fragment (level of detail in the fragment). The median/KNN filtering and NLM were used

for low detailed fragments and high detailed fragments respectively. The algorithm enables one

to obtain high quality images with reasonable computational cost;

• The algorithm of image enhancement. For image sharpening and contrast enhancement it

is proposed to consider the local features of the image on the basis of the found functional

relationship between the correction strength and the normalized brightness variance in the

image fragment. The algorithm carries out contrast enhancement without significant stressing

of the noise component, which is one of main disadvantages of modern non-linear enhancement

algorithms, especially in low detail images;

• The algorithm of special custom color correction, which enables the physician to adjust the

picture according his or her own color preference.

The second part of this research is devoted to the proposed algorithms of endoscopic image

visualization, which enable to produce images with high diagnostic value and clearer for interpretation

for the physician. New algorithms of visualization were designed next, taking into account main

features of endoscopic images:

低细节片段采

用中位数/KNN

滤波，高细节

片段采用NLM

滤波。

Volume 10 • Issue 4 • October-December 2019

• The algorithm of endoscopic mosaic images synthesis provides an image with wide viewing

angle of the organ under supervision which is convenient for analysis and navigation during

the physician’s inspection. This algorithm takes into account low detail level of the initial data;

• The algorithm of combined image synthesis, which increases diagnostic value of images by

including information obtained in white light and the IR light.

This paper is an extended version of the publications (Obukhova, Motyko, & Pozdeev, 2017) and

(Obukhova, Motyko, & Pozdeev, 2018). The paper is organized as follows. The analysis of modern

methods and algorithms of noise removing, contrast enhancement and color correction as steps of

endoscopic real-time pipeline is given in second section. In third, based on the assessment of the

effective noise removing technique for endoscopy image processing pipeline was proposed. Fourth

section is devoted to a new local non-linear algorithm of contrast and brightness enhancement. The

algorithm of special custom color correction is given in fifth section. The proposed algorithms

of endoscopic picture visualization such as: the algorithm of special custom color correction and

the algorithm of combined image synthesis are described in fifth and seventh sections. The main

results of experimental research for proposed algorithms and performance test are described in the

eighth part of the paper. The results of algorithms application in a real-time endoscopic pipeline are

summarized in the Conclusion.



There is a significant number of works devoted to endoscopy images processing, for example, an

extremely comprehensive review with more than 300 works in the field of endoscopy image analysis

and processing was made in work of Münzer, Schoeffmann and Böszörmenyi (2018).

Upon the analysis of such works in the field of endoscopy image processing, subsequent conclusion

could be reached. Algorithms proposed in the works devoted to particular task of enhancement, for

example, contrast enhancement (Sheraizin, & Sheraizin, 2006) or distortion correction (Asari, Kumar,

& Radhakrishnan, 1999; Barreto, Swaminathan, & Roquette, 2007). It is important to take note that

the algorithms described in these works were implemented to solve separate tasks, without taking

into account the pipeline steps.

The crucial point for quality representation and processing of endoscopic images is noise reduction

due to the fact that the noise is one of the most common problems in medical image processing.

Even a high-resolution medical video is bound to have some noise in it. As for the endoscopic video

it is a constant problem, because the lack of light forces one to use high gain which leads to high

level of noise. Oulhaj, Amine, Rziza and Aboutajdine (2012) made a comparison of noise reduction

algorithms for medical images and concluded that NLM produces the best results but requires high

computational costs. Therefore, there appears to be no algorithm which satisfies such requirements

as high quality and low computational cost.

Contrast is one of the key characteristics of an image. Images with low contrast usually display

little visual information, whereas high-contrast images maximize the available visual information. The

conditions under which an image is captured greatly affect its contrast. In endoscopic images improved

contrast is crucial for a correct diagnosis, but, on the other hand, the conditions for obtaining medical

images are very difficult. Therefore, because of the fact that medical images suffer from degradation,

it is necessary to eliminate such degradation before images are demonstrated to a physician.

Nowadays traditional image enhancement is based on global linear transformation (Damiani,

et al., 2007). (Imtiaz, Khan, & Wahid, 2013) propose to acquire a set of grayscale images using

the FIСE technology (Miyake et al., 2005), then select the image with the maximum entropy

and restore its color to increase the contrast. In other works (Imtiaz, & Wahid, 2014) the

adaptive sigmoid function is utilized to convert the intensity component values of the image.

Such enhancement is not effective for medical images, due to the fact that overall conditions of

Volume 10 • Issue 4 • October-December 2019

medical image obtainment have nonlinear contrast and brightness changes. Therefore, there are

very dark and very bright areas in one image, which demands for special correction techniques.

In (Xia, Chen, & Peters, 2018) it was attempted to solve this problem by splitting the image

into the base and detailed layer, as well as using different methods of processing for fragments

with good illumination, low illumination and extremely low illumination. This algorithm is

parallelizable; however, it is computationally costly, which makes it difficult to utilize in real-

time systems in conjunction with other algorithms. The other problem of modern contrast

enhancement techniques is noise stressing, especially in the part of image with low amount of

detail. Therefore, it is required to design nonlinear local brightness and contrast enhancement

algorithms, as well as algorithms for medicine image adaptation.

It is required to place particular emphasis on color correction of endoscopic images, and especially

on custom color correction. The monitor and video processor can be adjusted to match the user’s

own color preferences. The important point for endoscopic color imaging is to provide the user with

the specific image that he or she desires to see. The problem of color correction is well-studied

(Chen, et al., 2012), however the approach of conducting custom color correction with regard to the

modern trend of observer-dependent color imaging workflow is rarely taken (Sarkar, 2011). In order

to complete the abovementioned task, a special algorithm based on recalculating color space of the

sensor according to customer preferences given in the form of customer color space by using a linear

transformation matrix was implemented.

The main problem of endoscopic information is the narrow field of view (Nassimizadeh, et

al., 2015). In order to overcome the problem, it was proposed to utilize the panoramic mosaics

representation of organs with wide cavity. The main advantage of the proposed algorithm in comparison

with the existing approaches (detailed review of methods and algorithms for endoscopic images

stitching is given in (Reeff, 2011; Bergen, Wittenberg, 2014)) is connected with taking into account

the main feature of endoscopic image, which is the low quantity of detail. The reliability degree for

key points matched pairs is introduced which enables one to eliminate threshold for matched pairs

filtering and using the maximum quantity of key points pairs.





Modern algorithms of noise reduction can be divided into two groups. The algorithms of the

first group work directly with the image signal. These algorithms include smoothing filters,

morphological filters, median and rank filters, adaptive median filter, k - nearest neighbors filter

(KNN), and non-local mean filter (NLM). The algorithms of the second group are based on the

decomposition of the image signal into a basis, followed by the processing of the decomposition

transformants, e.g. a 3D block matching algorithm.

For the purposes of this research, the most effective algorithms of the first group, namely KNN,

NLM and a rather simple median filter were utilized.

NLM is the modern algorithm for image denoising. Unlike “local mean” filters, which take the

mean value of a group of pixels surrounding a target pixel to smooth the image, non-local means

filtering takes a mean of all pixels in the image, weighted by how similar these pixels are to the target

pixel. This results in much greater post-filtering clarity, and less loss of detail in the image compared

with local mean algorithms (Buades, Coll, & Morel, 2005).

The NLM (Buades, Coll, & Morel, 2005) has the following background. The main idea is to

define the filtered pixel value by averaging the pixels with the similar neighborhoods. The basic idea

of KNN (Wenchao, & Qi, 2012) is similar to NLM, but instead of all the pixels of the image, only

k-nearest neighbors is taken into consideration for each pixel.

Volume 10 • Issue 4 • October-December 2019

The low detailed images (or large areas of images) are a typical case in endoscopic observations

of the organs with smooth tissues. Consequently, it is crucial for denoising algorithms to cope with

this kind of flat surfaces.

However, a physician often takes the most interest in the vascular patterns, various wrinkles and

other high detailed parts of images. It is understood that in such fragments the algorithm for noise

reduction should perform with utmost effectiveness.

Thus, in research of the algorithm selection for endoscopic pipeline two criteria must be taken

into account: a reasonable compromise between effectiveness and processing speed, and preserving

features of endoscopic images.

In this research (Obukhova, Motyko, & Pozdeev, 2018) it was discovered that on low detailed

fragments, which dominate on endoscopic images, the computationally costly algorithms do not

provide well expressed effects in comparison with relatively simple approaches. The difference is

approximately 2dB. It is necessary to stress that endoscopic images have rather big area of fragments

with low level of detail about 70-80% of the full image square.

Based on this position and taking into account both requirements (quality and low computational

cost) we propose the following algorithm:

1. Image processing by the operator of a multiscale morphological gradient in order to emphasize

the high-frequency component;

2. Splitting the image into blocks with N*M size and determining for each block the detail level

estimation D k l

( )

according to Equation (1):

D k l D k l D

M min

, , /

( )

(1)

D k l x i y j

k l

, ( , )

( )

= + +

= =

∑∑

1 1

D D k l k KK l LL

min

moda=

( )

{

}

= =, .. , ..1 1

where Λ x y,

( )

– the pixel brightness in the image obtained from the initial one L x y,

( )

as a result

of spatial differentiation with operator multiscale morphological differentiation; x y

k l

, - the block

upper left corner coordinates.

3. Threshold processing. The detail estimations obtained for all blocks is random value. Analysis

and study of its histograms suggests that it has Rayleigh distribution. Let

be the variance of

D k l

( )

values calculated for all image blocks, p

be the quantile of Rayleigh distribution. In

this case, one is able to calculate the threshold value according to Equation (2):

λ σ= −

( )

ln p

(2)

4. If

D k l

( )

< λ then median or KNN filter is used for block processing else NLM filter.

图像处理采用多尺度形态学梯度算子，以强调高频分量

剩余20页未读，继续阅读

评论收藏

内容反馈

FOR_BH

粉丝: 0
资源: 44

Algorithms-for-Real-Time-Endoscopy-Image-Processing-Pipeline-in

Real-Time Image and Video Processing From Research to Reality

图像处理算法

image+processing

Algorithm-algorithms-make-it-real.zip

Algorithm-Problem-Solving-with-Algorithms-and-Data-Structures-using-Python.zip

problem-solving-with-algorithms-and-data-structure-using-python 中文版

Machine-Learning-Algorithms-master 配套数据集及源代码

On-Manifold Preintegration for Real-Time Visual-Inertial Odometry.pdf

python-algorithms-mastering-basic-algorithms-in-the-python-language.9781430232377.53502

Algorithms for Image Processing and Computer Vision, 2nd Edition

Algorithms-for-Sensor-Based-Robotics-源码.rar

Algorithms for Non-negative Matrix 论文概要

Python-for-Algorithms--Data-Structures--and-Interviews, 关于算法和数据结构的Udemy课程文件.zip

Algorithm-Data-Structures-and-Algorithms-in-Java-2nd-Edition-by-Robert-Lafore.zip

Image-Fusion-Algorithms-and-Applications

Algorithm-algorithms-in-python.zip

Computer Vision Algorithms in Image Algebra

Data-Structures-and-Algorithms-with-Python-Undergraduate-Topics-in-Computer-Science.pdf.pdf

matlab路由选择算法仿真代码-Routing-Algorithms-For-Enenrgy-Efficiency-In-UWSNs:UWS

neo4j-graph-algorithms-algo-3.5.4.0.jar

Algorithm-Algorithms-and-Data-Structures-in-Ruby.zip

data-algorithms-book, 数据算法书的MapReduce Spark Java和 Scala.zip

10Algorithms-08.pdf

(eBook)(math) P.Embree - C Algorithms for real-time dsp

Algorithm-Algorithms-Study-Group.zip

Practical Machine Learning and Image Processing

最新资源