毕业论文-基于iSCSI的重复数据删除系统的设计与实现.doc资源-CSDN文库

59 浏览量 2023-06-30 05:04:16 上传评论收藏 981KB DOC 举报

资源推荐

资源详情

资源评论

SelectionParagraphFormatLineSpacingLinesToPointsSelectionParagraphFormatLineSpacingLinesToPointselectionParagraaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaphFormatLineSpacingLinesToPointsSelectionParagraphFormatLineSpacingLinesTSelectionParbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbagraphFoLineSpacingLinesToPointsSelectionParagraphFormatLineSpacingLinesToPointse11111111111111111111111111111111lectionParagraphFormatLineSpacingLinesToPointsSelectionParagraphFormatLineSpacingLinesToPoctionParagraphFormatLineSpaci2222222222222222222222ngLinesToPoints2SelectionParagraphFormatLineSpacingLinesToPointsSelectionParagraphFormatLineSpacingLinesToPointselectionParagraphFccccccccccccccccccccccccccccccccccccccccccccccccccccccccormatLineSpacingLinesToPointsSelectionParagraphFormatLineSpacingLinesToPoctionParagraSelec

摘要

信息化的快速发展致使数据量与日俱增，简单的存储这些数据对企业而言并不是

最佳的解决方案——存储需要投入成本，大量的文件最终将会加重企业数据备份以及

灾难恢复系统的负担。企业与其不断的扩充磁盘容量来应对数据量的增加，还不如转

向数据删除技术，以存储更少的数据。近年来新兴的重复数据删除技术就是减少存储

空间的有效方式之一。

通过对重复数据删除技术的深入研究，提出了一种基于 iSCSI 平台的重复数据删

除存储系统。该系统实现了 LBA 映射、指纹计算、指纹检索和指纹索引表管理等功

能。通过 LBA 映射表的组织和管理，实现了重复数据删除前后数据块逻辑地址的转

化和对应关系；指纹计算模块中采用基于散列的 SHA-1 算法，实现了将 4KB 数据块

转化为 160 位摘要值的过程；指纹检索和指纹索引表的管理采用三级索引结构，实现

了指纹的精确定位和快速查找。为了弥补重复数据删除带来的系统性能的损失，针对

重复数据删除功能中指纹检索性能瓶颈进行了优化，提出了基于布鲁姆过滤的指纹检

索算法，大量的指纹检索请求被过滤掉，从而提高检索效率。

对系统性能、重复数据删除压缩比和检索过滤算法的效果进行了相关测试。分别

测试了标准 iSCSI 和加入重复数据删除模块后的 iSCSI 系统的性能，结果表明，加入

重复数据删除之后，虽然系统性能有所下降，但是下降的幅度还是预期的范围之内；

对重复数据删除压缩比进行了测试，测试结果表明压缩效果的好坏与应用环境密切相

关，当应用于那些信息重复度较高的环境如备份存储系统、归档存储系统等时，具有

较好的压缩效果；最后对检索过滤算法进行了测试，测试出的过滤率和误判率都可以

达到预期效果。

关键词：重复数据删除，指纹检索优化，存储性能

Abstract

Resulted in the rapid development of information technology increasing the amount of

data, simple storage of these data to enterprises is not the best solution - storage needs input

costs, a large number of documents that will ultimately increase the enterprise data backup

and disaster recovery burden. Compared to expand disk capacity to respond to the increase

in the amount of data, companies might as well turn to remove the technical data to store

less data.In recent years, new data deduplication technology is one of effective way to

reduce storage space.

Data de-duplication technology by further research, a platform based on iSCSI

deduplication storage systems. This system has LBA mapping, fingerprint calculation,

fingerprints and fingerprint search index table management. LBA mapping table by the

organization and management, and data de-duplication before data blocks the conversion of

logical address and correspondence; fingerprint calculation module based on SHA-1 hash

algorithm, implemented into the 4KB block 160 Summary value of the process; fingerprints

and fingerprint index table to retrieve the management of all three index structure is used to

achieve precise positioning and fast fingerprint search. To make up for deduplication

performance caused the loss of data deduplication feature for fingerprint retrieval

performance bottlenecks, for a special algorithm optimization, proposed fingerprint

retrieval based on Bloom filter filtering algorithm to filter out a large number of fingerprint

retrieval request, thereby enhancing the efficiency of retrieval.

On system performance, data deduplication, compression ratio and the effect of

filtering algorithms to retrieve the relevant tests. ISCSI and standard were tested by adding

data deduplication module of the iSCSI system performance, results show that adding data

deduplication, the system performance has declined; on data deduplication compression

ratio were tested, the test results show that good compression bad environment is closely

related with the application, when applied to repeat that information environment such as a

摘要 .........................................................................................................I

ABSTRACT ...................................................................................................II

目录 .......................................................................................................IV

1 绪论 ..................................................................................................1

1.1 课题背景 .............................................................................................1

1.2 课题研究目的及意义 .........................................................................2

1.3 国内外发展现状 .................................................................................2

1.4 课题的主要研究工作 .........................................................................4

1.5 课题的来源 .........................................................................................5

2 系统关键技术概述 ...............................................................................6

2.1 ISCSI 平台简介.....................................................................................6

2.2 重复数据简介 .....................................................................................7

2.3 重复数据删除的基本原理 .................................................................8

2.4 数据处理粒度分析 .............................................................................9

2.5 BLOOM FILTER 算法 ..........................................................................10

2.6 本章小结 ...........................................................................................13

3 重复数据删除方案设计......................................................................14

3.1 系统功能需求 ...................................................................................14

3.2 系统总体设计 ...................................................................................14

3.3 LBA 映射表........................................................................................16

3.4 指纹计算模块 ...................................................................................16

3.5 指纹管理和检索模块 .......................................................................17

3.6 基于 BLOOM FILTER 算法的指纹检索优化......................................19

3.7 本章小结 ...........................................................................................20

4 重复数据删除系统实现......................................................................21

4.1 LBA 映射表实现 ................................................................................21

4.2 指纹计算模块实现 ...........................................................................22

4.3 指纹索引表的建立与指纹检索 .......................................................22

4.4 BLOOM FILTER 过滤算法的实现.......................................................23

4.5 处理流程分析 ...................................................................................24

4.6 本章小结 ...........................................................................................27

5 系统测试与分析 .................................................................................28

5.1 测试环境介绍 ...................................................................................28

5.2 测试结果及分析 ...............................................................................28

5.3 本章小结 ...........................................................................................32

6 总结与展望.........................................................................................33

6.1 总结...................................................................................................33

6.2 未来展望 ...........................................................................................33

致谢.......................................................................................................35

参考文献...................................................................................................36

剩余42页未读，继续阅读

评论收藏

内容反馈

matlab大师

粉丝: 2506
资源: 8万+

毕业论文-基于iSCSI的重复数据删除系统的设计与实现.doc

毕业论文基于iscsi的重复数据删除系统的设计与实现.doc

基于iscsi的重复数据删除系统的与实现.doc

具有密钥更新加密的重复数据删除存储系统的设计与实现

论文研究-基于iSCSI OSD存储系统的设计与分析.pdf

【批量下载】ceph-iscsi-tools-2.1-3.el7cp.noarch等.zip

iscsi-initiator-utils-6.2.0.871-0.10.el5.x86_64.rpm

论文研究-基于iSCSI的网络存储系统性能分析 .pdf

论文研究-iSCSI协议一致性测试研究与实现.pdf

论文研究-一种基于重复数据删除的备份系统 .pdf

基于对象的重复数据删除备份系统的有效RAID数据布局

一种基于Winnowing分块的重复数据删除备份与恢复系统设计与实现 (2012年)

基于重复数据删除技术的SQL Server数据库备份系统.pdf

iscsi-initiator-utils-6.2.0.868-0.7.el5.i386.rpm

iscsi-initiator-utils-6.2.0.874-22.el7_9.x86_64.rpm

Linux下基于iSCSI存储系统的实现与性能评测.pdf

iscsi-initiator-utils-iscsiuio-6.2.0.874-10.el7.x86_64.rpm

iscsi_initiator安装包

iscsi-initiator-utils-6.2.0.874-19.el7.x86_64.rpm

iscsi-initiator-utils-6.2.0.873-27.el6_9.i686.rpm

linux中搭建ISCSI共享存储

网络游戏-实现基于iSCSI的网络存储协议栈的方法.zip

iscsi-initiator-utils-iscsiuio-6.2.0.874-21.el7_9.x86_64.rpm

北邮大三上2022年《算法设计与分析》期末试题-A卷

计算机毕业设计答辩PPT模板（11套）.zip

2019city.zip

1000套计算机毕业设计带源码

山东大学计算机图形学2010试卷A(含答案).doc

最新资源