Contributingstorageusingthetransparentfilesystem资源-CSDN文库

需积分: 10 49 浏览量 2011-10-04 03:37:40 上传评论收藏 694KB PDF 举报

透明文件系统（Transparent File System，简称TFS）是旨在解决贡献存储问题的一种创新技术，尤其在个人计算机上未被充分利用的资源方面。该系统由詹姆斯·西帕尔（James Cipar）、马克·D·科纳（Mark D. Corner）和埃默里·D·伯杰（Emery D. Berger）在马萨诸塞大学阿默斯特分校共同研发。TFS的核心目标是在不影响主要用户正常使用的情况下，提供大量的不可靠存储空间，即当前所有可用的空间，用于背景任务处理。 ### TFS与贡献式存储贡献式应用允许用户将其个人电脑上未使用的资源贡献到共享池中，如SETI@home、Folding@home和Freenet等，它们提供了数据处理和内容分发等多样化的服务。然而，尽管有多个研究项目提出了支持对等存储系统的贡献式应用程序，但其实际应用却相对有限。这主要是因为大量贡献本地存储可能会干扰主机用户的正常使用。为克服这一障碍，TFS通过将所有可用的存储空间提供给背景任务，同时确保不会影响普通文件访问操作的性能，实现了存储贡献与用户使用之间的平衡。实验证明，与传统的用户空间存储机制相比，TFS使对等贡献存储系统提供的存储量增加了40%，且在性能上提升了两倍。 ### TFS的工作原理与特点 TFS设计的关键在于其能够高效管理文件碎片化、老化策略以及对等网络中的文件复制。它在文件系统层面运作，对用户来说是透明的，这意味着用户无需了解其内部工作细节即可享受其带来的性能提升。TFS通过智能地分配和管理存储空间，使得即使在大量存储贡献的情况下，也能保持良好的读写性能，避免了传统文件系统常见的性能瓶颈问题。 ### 对复制策略的影响 TFS还分析了其对对等存储系统中文件复制策略的影响。结果显示，TFS并没有显著增加用于文件复制所需资源，这意味着即使在大量贡献存储的场景下，系统仍然能够有效管理和复制文件，而不会造成额外的负担。 ### 结论与未来展望 TFS作为一种高性能、高效率的透明文件系统，为贡献式存储领域带来了革命性的改变。它不仅解决了用户贡献存储与日常使用之间的矛盾，还大大提高了对等网络存储系统的整体性能和可靠性。未来，随着更多贡献式应用的普及和技术的不断进步，TFS有望成为贡献式存储解决方案的标准配置，进一步推动分布式计算和数据存储技术的发展。透明文件系统（TFS）是一项具有深远意义的技术革新，它为个人用户参与贡献存储开辟了一条新的路径，同时也为科研机构和商业组织提供了更为灵活、高效的存储解决方案。随着TFS的广泛应用，我们有理由期待一个更加开放、高效和可持续的数据存储未来。

资源推荐

资源详情

资源评论

Contributing Storage Using the Transparent

File System

JAMES CIPAR, MARK D. CORNER, and EMERY D. BERGER

University of Massachusetts Amherst

Contributory applications allow users to donate unused resources on their personal computers to

a shared pool. Applications such as SETI@home, Folding@home, and Freenet are now in wide use

and provide a variety of services, including data processing and content distribution. However,

while several research projects have proposed contributory applications that support peer-to-peer

storage systems, their adoption has been comparatively limited. We believe that a key barrier to

the adoption of contributory storage systems is that contributing a large quantity of local storage

interferes with the principal user of the machine.

To overcome this barrier, we introduce the Transparent File System (TFS). TFS provides back-

ground tasks with large amounts of unreliable storage—all of the currently available space—

without impacting the performance of ordinary ﬁle access operations. We show that TFS allows

a peer-to-peer contributory storage system to provide 40% more storage at twice the performance

when compared to a user-space storage mechanism. We analyze the impact of TFS on replication

in peer-to-peer storage systems and show that TFS does not appreciably increase the resources

needed for ﬁle replication.

Categories and Subject Descriptors: D.4.3 [Operating Systems]: File Systems Management—File

organization

General Terms: Performance

Additional Key Words and Phrases: Contributory systems, fragmentation, aging, peer-to-peer

ACM Reference Format:

Cipar, J., Corner, M. D., and Berger, E. D. 2007. Contributing storage using the transparent ﬁle

system. ACM Trans. Stor. 3, 3, Article 12 (October 2007), 26 pages. DOI = 10.1145/1288783.1288787

http://doi.acm.org/10.1145/1288783.1288787

This material is based upon work supported by the National Science Foundation under CAREER

Awards CNS-0447877 and CNS-0347339, and DUE 0416863. Any opinions, ﬁndings, and conclu-

sions or recommendations expressed in this material are those of the author(s) and do not neces-

sarily reﬂect the views of the NSF.

Authors’ addresses: J. Cipar, M. D. Corner (corresponding author), E. D. Berger, Computer Science

Building, University of Massachusetts Amherst, 140 Governor’s Drive, Amherst, MA 01003; email:

mcorner@cs.umass.edu.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is

granted without fee provided that copies are not made or distributed for proﬁt or direct commercial

advantage and that copies show this notice on the ﬁrst page or initial screen of a display along

with the full citation. Copyrights for components of this work owned by others than ACM must be

honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers,

to redistribute to lists, or to use any component of this work in other works requires prior speciﬁc

permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn

Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or permission@acm.org.



2007 ACM 1553-3077/2007/10-ART12 $5.00 DOI = 10.1145/1288783.1288787 http://doi.acm.org/

10.1145/1288783.1288787

ACM Transactions on Storage, Vol. 3, No. 3, Article 12, Publication date: October 2007.

12:2

•

J. Cipar et al.

1. INTRODUCTION

Contributory applications allow users to donate unused resources from their

personal computers to a shared pool. These applications harvest idle resources

such as CPU cycles, memory, network bandwidth, and local storage to serve a

common distributed system. These applications are distinct from other peer-to-

peer systems because the resources being contributed are not directly consumed

by the contributor. For instance, in Freenet [Clarke et al. 2001], all users con-

tribute storage and any user may make use of the storage, but there is no rela-

tionship between user data and contributed storage. Contributory applications

in wide use include computing efforts like Folding@home [Larson et al. 2002]

and anonymous publishing and content distribution such as Freenet [Clarke

et al. 2001]. The research community has also developed a number of con-

tributory applications, including distributed backup and archival storage

[Rowstron and Druschel 2001b], serverless network ﬁle systems [Adya et al.

2002], and distributed web caching [Freedman et al. 2004]. However, the adop-

tion of storage-based contributory applications has been limited compared to

those that are CPU-based.

Two major barriers impede broader participation in contributory storage

systems. First, existing contributory storage systems degrade normal applica-

tion performance. While transparency—the effect that system performance is

as if no contributory application is running—has been the goal of other OS

mechanisms for network bandwidth [Venkataramani et al. 2002], main mem-

ory [Cipar et al. 2006], and disk scheduling [Lumb et al. 2002], previous work

on contributory storage systems has ignored its local performance impact. In

particular, as more storage is allocated, the performance of the user’s ﬁle system

operations quickly degrades [McKusick et al. 1984].

Second, despite the fact that end-user hard drives are often half

empty [Douceur and Bolosky 1999; Huang et al. 2005], users are generally

reluctant to relinquish their free space. Though disk capacity has been steadily

increasing for many years, users view storage space as a limited resource. For

example, three of the Freenet FAQs express the implicit desire to donate less

disk space [Freenet 2007]. Even when users are given the choice to limit the

amount of storage contribution, this option requires the user to decide a priori

what is a reasonable contribution. Users may also try to donate as little as pos-

sible while still taking advantage of the services provided by the contributory

application, thus limiting its overall effectiveness.

Contributions. This article presents the Transparent File System (TFS), a

ﬁle system that can contribute 100% of the idle space on a disk while impos-

ing a negligible performance penalty on the local user. TFS operates by storing

ﬁles in the free space of the ﬁle system so that they are invisible to ordinary

ﬁles. In essence, normal ﬁle allocation proceeds as if the system were not con-

tributing any space at all. We show in Section 5 that TFS imposes nearly no

overhead on the local user. TFS achieves this both by minimizing interference

with the ﬁle system’s block allocation policy and by sacriﬁcing persistence for

contributed space: Normal ﬁles may overwrite contributed space at any time.

TFS takes several steps that limit this unreliability, but because contributory

ACM Transactions on Storage, Vol. 3, No. 3, Article 12, Publication date: October 2007.

Contributing Storage Using the Transparent File System

•

12:3

applications are already designed to work with unreliable machines, they be-

have appropriately in the face of unreliable ﬁles. Furthermore, we show that

TFS does not appreciably impact the bandwidth needed for replication. Users

typically create little data in the course of a day [Bolosky et al. 2000], thus

the erasure of contributed storage is negligible when compared to the rate of

machine failures.

TFS is especially useful for replicated storage systems executing across rela-

tively stable machines with plentiful bandwidth, as in a university or corporate

network. This environment is the same one targeted by distributed storage sys-

tems such as FARSITE [Adya et al. 2002]. As others have shown previously,

for high-failure modes such as wide-area Internet-based systems, the key lim-

itation is the bandwidth between nodes, not the total storage. The bandwidth

needed to replicate data after failures essentially limits the amount of storage

the network can use [Blake and Rodrigues 2003]. In a stable network, TFS

offers substantially more storage than dynamic, user-space techniques for con-

tributing storage.

Organization. In Section 2, we ﬁrst provide a detailed explanation of the

interference caused by contributory applications, and discuss current alterna-

tives for contributing storage. Second, we present the design of TFS in Section 3,

focusing on providing transparency to normal ﬁle access. We describe a fully

operating implementation of TFS. We then explain in Section 4 the interac-

tion between machine reliability, contributed space, and the amount of storage

used by a contributory storage system. Finally, we demonstrate in Section 5

that the performance of our TFS prototype is on par with the ﬁle system it was

derived from, and up to twice as fast as user-space techniques for contributing

storage.

2. INTERFERENCE FROM CONTRIBUTING STORAGE

All contributory applications we are aware of are conﬁgured to contribute a

small, ﬁxed amount of storage; the contribution is small so as not to interfere

with normal machine use. This low level of contribution has little impact on ﬁle

system performance and ﬁles will generally only be deleted by the contributory

system, not because the user needs storage space. However, such small, ﬁxed-

size contributions limit contribution to small-scale storage systems.

Instead of using static limits, one could use a dynamic system that monitors

the amount of storage used by local applications. The contributory storage sys-

tem could then use a signiﬁcantly greater portion of the disk, while yielding

space to the local user as needed. Possible approaches include the watermark-

ing schemes found in Elastic Quotas [Leonard et al. 2002] and FS

[Huang

et al. 2005]. A contributory storage system could use these approaches as fol-

lows: Whenever the current allocation exceeds the maximum watermark set

by the dynamic contribution system, it could delete contributory ﬁles until the

contribution level falls below a lower watermark.

However, if the watermarks are set to comprise all free space on the disk,

the ﬁle system is forced to delete ﬁles synchronously from contributed storage

when writing new ﬁles to disk. In this case, the performance of the disk would be

ACM Transactions on Storage, Vol. 3, No. 3, Article 12, Publication date: October 2007.

12:4

•

J. Cipar et al.

severely degraded, similar to the synchronous cleaning problem in LFS [Seltzer

et al. 1993]. For this reason, Elastic Quotas and FS

use more conservative

watermarks (e.g., at most 85%), allowing the system to delete ﬁles lazily as

needed.

Choosing a proper watermark leaves the system designer with a tradeoff

between the amount of storage contributed and local performance. At one end

of the spectrum, the system can contribute little space, limiting its usefulness.

At the other, local performance suffers.

To see why local performance suffers, consider the following: As a disk ﬁlls,

the ﬁle system’s block allocation algorithm becomes unable to make ideal al-

location decisions, causing fragmentation of the free space and allocated ﬁles.

This fragmentation increases the seek time when reading and writing ﬁles, and

has a noticeable effect on the performance of disk-bound processes. In an FFS

ﬁle system, throughput can drop by as much as 77% in a ﬁle system that is

only 75% full versus an empty ﬁle system [Smith and Seltzer 1997]; the more

storage one contributes, the worse the problem becomes. The only way to avoid

this is to maintain enough free space on the disk to allow the allocation algo-

rithm to work properly, but this limits contribution to only a small portion of the

disk.

Though some ﬁle systems provide utilities to defragment their disk layout,

these utilities are ineffective when there is insufﬁcient free space on the ﬁle

system. For instance, the defragmentation utility provided with older versions

of Microsoft Windows will not even attempt to defragment a disk if more than

85% is in use. On modern Windows systems, the defragmentation utility will

run when the disk is more than 85% full, but will give a warning that there

is not enough free space to defragment properly [Microsoft 2007]. When one

wants to contribute all of the free space on the disk, it will be impossible to

meet these requirements of the defragmentation utility.

Any user-space scheme to manage contributed storage will be plagued by

the performance problems introduced by the contributed storage; ﬁlling the

disk with data inevitably slows access. Given a standard ﬁle system interface,

it is simply not possible to order the allocation of data on-disk to preserve perfor-

mance for normal ﬁles. As Section 3 shows, by incorporating the mechanisms for

contribution into the ﬁle system itself, TFS maintains ﬁle system performance

even at high levels of contribution.

Figure 1 depicts this effect. This ﬁgure shows the time taken to run the copy

phase of the Andrew benchmark on ﬁle systems with different amounts of space

being consumed. The full details of the benchmark are presented in Section 5.

As the amount of consumed space increases, the time it takes to complete the

benchmark increases. We assume that the user is utilizing 50% of the disk

space for noncontributory applications, which corresponds to results from a

survey of desktop ﬁle system contents [Douceur and Bolosky 1999]. The ﬁgure

shows that contributing more than 20% of the disk space will noticeably affect

the ﬁle system’s performance, even if the contributed storage would have been

completely idle. As a preview of TFS performance, note that when contributing

35% of the disk, TFS is twice as fast as Ext2 for copying ﬁles.

ACM Transactions on Storage, Vol. 3, No. 3, Article 12, Publication date: October 2007.

剩余25页未读，继续阅读

评论收藏

内容反馈

chocolatezwj

粉丝: 0
资源: 5

Contributing storage using the transparent file system

最新资源