## What does `s3_cache.py` do?
### In general
`s3_cache.py` maintains a cache, stored in an Amazon S3 (Simple Storage Service) bucket, of a given directory whose contents are considered non-critical and are completely & solely determined by (and should be able to be regenerated from) a single given file.
The SHA-256 hash of the single file is used as the key for the cache. The directory is stored as a gzipped tarball.
All the tarballs are stored in S3's Reduced Redundancy Storage (RRS) storage class, since this is cheaper and the data is non-critical.
`s3_cache.py` itself never deletes cache entries; deletion should either be done manually or using automatic S3 lifecycle rules on the bucket.
Similar to git, `s3_cache.py` makes the assumption that [SHA-256 will effectively never have a collision](http://stackoverflow.com/questions/4014090/is-it-safe-to-ignore-the-possibility-of-sha-collisions-in-practice).
### For Bootstrap specifically
`s3_cache.py` is used to cache the npm packages that our Grunt tasks depend on and the RubyGems that Jekyll depends on. (Jekyll is needed to compile our docs to HTML so that we can run them thru an HTML5 validator.)
For npm, the `node_modules` directory is cached based on our `npm-shrinkwrap.json` file.
For RubyGems, the `gemdir` of the current RVM-selected Ruby is cached based on the `pseudo_Gemfile.lock` file generated by our Travis build script.
`pseudo_Gemfile.lock` contains the versions of Ruby and Jekyll that we're using (read our `.travis.yml` for details).
## Why is `s3_cache.py` necessary?
`s3_cache.py` is used to speed up Bootstrap's Travis builds. Installing npm packages and RubyGems used to take up a significant fraction of our total build times. Also, at the time that `s3_cache.py` was written, npm was occasionally unreliable.
Travis does offer built-in caching on their paid plans, but this do-it-ourselves S3 solution is significantly cheaper since we only need caching and not Travis' other paid features.
## Configuration
`s3_cache.py` is configured via `S3Cachefile.json`, which has the following format:
```json
{
"cache-name-here": {
"key": "path/to/file/to/SHA-256/hash/and/use/that/as/the/cache.key",
"cache": "path/to/directory/to/be/cached",
"generate": "shell-command --to run --to regenerate --the-cache $from scratch"
},
...
}
```
`s3_cache.py` will SHA-256 hash the contents of the `key` file and try to fetch a tarball from S3 using the hash as the filename.
If it's unable to fetch the tarball (either because it doesn't exist or there was a network error), it will run the `generate` command. If it was able to fetch the tarball, it will extract it to the `cache` directory.
If it had to `generate` the cache, it will later create a tarball of the `cache` directory and try to upload the tarball to S3 using the SHA-256 hash of the `key` file as the tarball's filename.
## AWS Setup
### Overview
1. Create an Amazon Web Services (AWS) account.
2. Create an Identity & Access Management (IAM) user, and note their credentials.
3. Create an S3 bucket.
4. Set permissions on the bucket to grant the user read+write access.
5. Set the user credentials as secure Travis environment variables.
### In detail
1. Create an AWS account.
2. Login to the [AWS Management Console](https://console.aws.amazon.com).
3. Go to the IAM Management Console.
4. Create a new user (named e.g. `travis-ci`) and generate an access key for them. Note both the Access Key ID and the Secret Access Key.
5. Note the user's ARN (Amazon Resource Name), which can be found in the "Summary" tab of the user browser. This will be of the form: `arn:aws:iam::XXXXXXXXXXXXXX:user/the-username-goes-here`
6. Note the user's access key, which can be found in the "Security Credentials" tab of the user browser.
7. Go to the S3 Management Console.
8. Create a new bucket. For a non-publicly-accessible bucket (like Bootstrap uses), it's recommended that the bucket name be random to increase security. On most *nix machines, you can easily generate a random UUID to use as the bucket name using Python:
```bash
python -c "import uuid; print(uuid.uuid4())"
```
9. Determine and note what your bucket's ARN is. The ARN for an S3 bucket is of the form: `arn:aws:s3:::the-bucket-name-goes-here`
10. In the bucket's Properties pane, in the "Permissions" section, click the "Edit bucket policy" button.
11. Input and submit an IAM Policy that grants the user at least read+write rights to the bucket. AWS has a policy generator and some examples to help with crafting the policy. Here's the policy that Bootstrap uses, with the sensitive bits censored:
```json
{
"Version": "2012-10-17",
"Id": "PolicyTravisReadWriteNoAdmin",
"Statement": [
{
"Sid": "StmtXXXXXXXXXXXXXX",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::XXXXXXXXXXXXXX:user/travis-ci"
},
"Action": [
"s3:AbortMultipartUpload",
"s3:GetObjectVersion",
"s3:ListBucket",
"s3:DeleteObject",
"s3:DeleteObjectVersion",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
"arn:aws:s3:::XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/*"
]
}
]
}
```
12. If you want deletion from the cache to be done automatically based on age (like Bootstrap does): In the bucket's Properties pane, in the "Lifecycle" section, add a rule to expire/delete files based on creation date.
13. Install the [`travis` RubyGem](https://github.com/travis-ci/travis): `gem install travis`
14. Encrypt the environment variables:
```bash
travis encrypt --repo twbs/bootstrap "AWS_ACCESS_KEY_ID=XXX"
travis encrypt --repo twbs/bootstrap "AWS_SECRET_ACCESS_KEY=XXX"
travis encrypt --repo twbs/bootstrap "TWBS_S3_BUCKET=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
```
14. Add the resulting secure environment variables to `.travis.yml`.
## Usage
Read `s3_cache.py`'s source code and Bootstrap's `.travis.yml` for how to invoke and make use of `s3_cache.py`.
没有合适的资源?快使用搜索试试~ 我知道了~
asp.net + mvc + bootstrap-3.3.0(sqlserver2008)
共568个文件
html:114个
dll:73个
less:70个
4星 · 超过85%的资源 需积分: 16 51 下载量 157 浏览量
2015-12-31
13:55:41
上传
评论 3
收藏 16.02MB RAR 举报
温馨提示
项目采用asp.net + mvc + bootstrap-3.3.0(sqlserver2008)技术,实现从数据库读取数据,增加和修改及删除记录,并实现分页,数据库自动创建。。方便初学者学习。
资源推荐
资源详情
资源评论
收起资源包目录
asp.net + mvc + bootstrap-3.3.0(sqlserver2008) (568个子文件)
_._ 3B
Global.asax 107B
DesignTimeResolveAssemblyReferences.cache 167KB
MvcApplication1.csprojResolveAssemblyReference.cache 113KB
DesignTimeResolveAssemblyReferencesInput.cache 51KB
MvcApplication1.csproj.GenerateResource.Cache 1KB
CNAME 17B
Web.config 4KB
Web.config 3KB
packages.config 2KB
Web.Release.config 1KB
Web.Debug.config 1KB
repositories.config 134B
AccountController.cs 5KB
Configuration.cs 3KB
201512280246332_InitialCreate.cs 2KB
AssemblyInfo.cs 1KB
AccountInitializer.cs 1KB
201512280246332_InitialCreate.Designer.cs 855B
201512280605024_AddTestTable.Designer.cs 852B
AccountContext.cs 835B
Global.asax.cs 721B
SysUser.cs 644B
201512280605024_AddTestTable.cs 641B
RouteConfig.cs 610B
WebApiConfig.cs 484B
MVCDemoController.cs 441B
SysUserRole.cs 409B
SysRole.cs 368B
AdminController.cs 345B
FilterConfig.cs 281B
Test .cs 246B
_LayoutAdmin.cshtml 2KB
Index.cshtml 2KB
_Layout.cshtml 2KB
Logins.cshtml 2KB
Index.cshtml 1KB
Details.cshtml 1KB
Login.cshtml 1KB
Edit.cshtml 1021B
Create.cshtml 948B
Delete.cshtml 886B
HelperTest.cshtml 403B
Register.cshtml 223B
Index.cshtml 105B
_ViewStart.cshtml 55B
MvcApplication1.csproj 41KB
bootstrap.css 134KB
bootstrap.css 134KB
bootstrap.css 134KB
bootstrap.min.css 111KB
bootstrap.min.css 111KB
docs.css 29KB
bootstrap-theme.css 21KB
bootstrap-theme.css 21KB
bootstrap-theme.css 21KB
docs.min.css 21KB
bootstrap-theme.min.css 19KB
bootstrap-theme.min.css 19KB
qunit.css 4KB
pygments-manni.css 3KB
PagedList.css 3KB
PagedList.css 3KB
non-responsive.css 3KB
carousel.css 2KB
cover.css 2KB
blog.css 2KB
justified-nav.css 2KB
dashboard.css 2KB
dashboard.css 2KB
jumbotron-narrow.css 1KB
offcanvas.css 990B
signin.css 793B
signin.css 793B
sticky-footer-navbar.css 678B
sticky-footer.css 602B
tooltip-viewport.css 385B
grid.css 362B
theme.css 255B
jumbotron.css 127B
starter-template.css 96B
navbar.css 90B
navbar-static-top.css 78B
navbar-fixed-top.css 52B
.csslintrc 456B
EntityFramework.dll 4.96MB
EntityFramework.dll 4.75MB
EntityFramework.dll 4.75MB
EntityFramework.SqlServer.dll 607KB
EntityFramework.SqlServer.dll 588KB
EntityFramework.SqlServer.dll 588KB
System.Web.Mvc.dll 495KB
System.Web.Mvc.dll 495KB
Newtonsoft.Json.dll 366KB
Newtonsoft.Json.dll 366KB
System.Web.Http.dll 316KB
System.Web.Http.dll 316KB
System.Web.Razor.dll 259KB
System.Web.Razor.dll 259KB
System.Web.WebPages.dll 200KB
共 568 条
- 1
- 2
- 3
- 4
- 5
- 6
资源评论
- 双鱼星星2016-10-12还不错,就是缺少数据库文件
aa_aaaaaaa
- 粉丝: 0
- 资源: 1
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功