[![Build Status](https://dev.azure.com/great-expectations/great_expectations/_apis/build/status/great_expectations?branchName=develop&stageName=required)](https://dev.azure.com/great-expectations/great_expectations/_build/latest?definitionId=1&branchName=develop)
![Coverage](https://img.shields.io/azure-devops/coverage/great-expectations/great_expectations/1/main)
[![Documentation Status](https://readthedocs.org/projects/great-expectations/badge/?version=latest)](http://great-expectations.readthedocs.io/en/latest/?badge=latest)
<!-- <<<Super-quickstart links go here>>> -->
<img align="right" src="./generic_dickens_protagonist.png">
Great Expectations
================================================================================
*Always know what to expect from your data.*
Introduction
--------------------------------------------------------------------------------
Great Expectations helps data teams eliminate pipeline debt, through data testing, documentation, and profiling.
Software developers have long known that testing and documentation are essential for managing complex codebases. Great Expectations brings the same confidence, integrity, and acceleration to data science and data engineering teams.
See [Down with Pipeline Debt!](https://medium.com/@expectgreatdata/down-with-pipeline-debt-introducing-great-expectations-862ddc46782a) for an introduction to the philosophy of pipeline testing.
<!--
--------------------------------------------------
<<<A bunch of logos go here for social proof>>>
--------------------------------------------------
-->
Key features
--------------------------------------------------
### Expectations
Expectations are assertions for data. They are the workhorse abstraction in Great Expectations, covering all kinds of common data issues, including:
- `expect_column_values_to_not_be_null`
- `expect_column_values_to_match_regex`
- `expect_column_values_to_be_unique`
- `expect_column_values_to_match_strftime_format`
- `expect_table_row_count_to_be_between`
- `expect_column_median_to_be_between`
- ...and [many more](https://docs.greatexpectations.io/en/latest/reference/glossary_of_expectations.html)
Expectations are <!--[declarative, flexible and extensible]()--> declarative, flexible and extensible.
<!--To test out Expectations on your own data, check out the [<<step-1 tutorial>>]().-->
<!--
<<animated gif showing typing an Expectation in a notebook cell, running it, and getting an informative result>>
-->
### Batteries-included data validation
Expectations are a great start, but it takes more to get to production-ready data validation. Where are Expectations stored? How do they get updated? How do you securely connect to production data systems? How do you notify team members and triage when data validation fails?
Great Expectations supports all of these use cases out of the box. Instead of building these components for yourself over weeks or months, you will be able to add production-ready validation to your pipeline in a day. This “Expectations on rails” framework plays nice with other data engineering tools, respects your existing name spaces, and is designed for extensibility.
<!--
Check out [The Era of DIY Data Validation is Over]() for more details.
-->
<!--
<<animated gif showing slack message, plus click through to validation results, a la: https://docs.google.com/presentation/d/1ZqFXsoOyW2KIkMBNij3c7KOM0RhajhAHKesdCL_BKHw/edit#slide=id.g6b0ff79464_0_183>>
-->
![ooooo ahhhh](./readme_assets/terminal.gif)
### Tests are docs and docs are tests
```diff
! This feature is in beta
```
Many data teams struggle to maintain up-to-date data documentation. Great Expectations solves this problem by rendering Expectations directly into clean, human-readable documentation.
Since docs are rendered from tests, and tests are run against new data as it arrives, your documentation is guaranteed to never go stale. Additional renderers allow Great Expectations to generate other type of "documentation", including <!--[slack notifications](), [data dictionaries](), [customized notebooks]()--> slack notifications, data dictionaries, customized notebooks, etc.
<!--
<<Pic, similar to slide 32: https://docs.google.com/presentation/d/1ZqFXsoOyW2KIkMBNij3c7KOM0RhajhAHKesdCL_BKHw/edit#slide=id.g6af8c4cd70_0_38>>
<<Pic, showing an Expectation that renders a graph>>
Check out [Down with Documentation Rot!]() for more details.
-->
![Your tests are your docs and your docs are your tests](./readme_assets/test-are-docs.jpg)
### Automated data profiling
```diff
- This feature is experimental
```
Wouldn't it be great if your tests could write themselves? Run your data through one of Great Expectations' data profilers and it will automatically generate Expectations and data documentation. Profiling provides the double benefit of helping you explore data faster, and capturing knowledge for future documentation and testing.
<!--
<<<pretty pics of profiled data>>>
<<<esp. multi-batch profiling>>>
-->
![ooooo ahhhh](./readme_assets/datadocs.gif)
Automated profiling doesn't replace domain expertise—you will almost certainly tune and augment your auto-generated Expectations over time—but it's a great way to jump start the process of capturing and sharing domain knowledge across your team.
<!--
<<<Note: this feature is still in early beta. Expect changes.>>>
Visit our gallery of expectations and documentation generated via automatic data profiling [here]().
You can also test out profiling on your own data [here]().
-->
### Pluggable and extensible
Every component of the framework is designed to be extensible: Expectations, storage, profilers, renderers for documentation, actions taken after validation, etc. This design choice gives a lot of creative freedom to developers working with Great Expectations.
Recent extensions include:
* [Renderers for data dictionaries](https://greatexpectations.io/blog/20200731_data_dictionary_plugin/)
* [BigQuery and GCS integration](https://github.com/great-expectations/great_expectations/pull/841)
* [Notifications to MatterMost](https://github.com/great-expectations/great_expectations/issues/902)
We're very excited to see what other plugins the data community comes up with!
Quick start
-------------------------------------------------------------
To see Great Expectations in action on your own data:
You can install it using pip
```
pip install great_expectations
```
or conda
```
conda install -c conda-forge great-expectations
```
and then run
```
great_expectations init
```
(We recommend deploying within a virtual environment. If you’re not familiar with pip, virtual environments, notebooks, or git, you may want to check out the [Supporting Resources](http://docs.greatexpectations.io/en/latest/reference/supporting_resources.html), which will teach you how to get up and running in minutes.)
For full documentation, visit [Great Expectations on readthedocs.io](http://great-expectations.readthedocs.io/en/latest/).
If you need help, hop into our [Slack channel](https://greatexpectations.io/slack)—there are always contributors and other users there.
<!--
-------------------------------------------------------------
<<<More social proof: pics and quotes of power users>>>
-------------------------------------------------------------
-->
Integrations
-------------------------------------------------------------------------------
Great Expectations works with the tools and systems that you're already using with your data, including:
<table>
<thead>
<tr>
<th colspan="2">Integration</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr><td style="text-align: center; height=40px;"><img height="40" src="https://dev.pandas.io/static/img/pandas.svg" /> </td><td style="width: 200px;">Pandas </td><td>Great for in-memory machine learning pipelines!</td></tr>
<tr><td style="text-align: center; height=40px;
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
共1136个文件
py:738个
json:109个
yml:59个
资源分类:Python库 所属语言:Python 资源全名:great_expectations-0.13.31.tar.gz 资源来源:官方 安装方法:https://lanzao.blog.csdn.net/article/details/101784059
资源推荐
资源详情
资源评论
收起资源包目录
Python库 | great_expectations-0.13.31.tar.gz (1136个子文件)
setup.cfg 335B
data_docs_default_styles.css 2KB
data_docs_custom_styles.css 699B
data_docs_custom_styles_template.css 699B
yellow_trip_data_sample_2019-02.csv 954KB
yellow_trip_data_sample_2019-10.csv 945KB
yellow_trip_data_sample_2020-02.csv 945KB
yellow_trip_data_sample_2019-05.csv 945KB
yellow_trip_data_sample_2019-09.csv 944KB
yellow_trip_data_sample_2019-12.csv 944KB
yellow_trip_data_sample_2019-11.csv 944KB
yellow_trip_data_sample_2019-04.csv 944KB
yellow_trip_data_sample_2019-06.csv 944KB
yellow_trip_data_sample_2020-01.csv 944KB
yellow_trip_data_sample_2019-07.csv 944KB
yellow_trip_data_sample_2019-03.csv 944KB
yellow_trip_data_sample_2019-08.csv 944KB
yellow_trip_data_sample_2020-03.csv 943KB
yellow_trip_data_sample_2020-10.csv 941KB
yellow_trip_data_sample_2020-11.csv 941KB
yellow_trip_data_sample_2020-12.csv 941KB
yellow_trip_data_sample_2020-09.csv 940KB
yellow_trip_data_sample_2020-08.csv 939KB
yellow_trip_data_sample_2020-07.csv 939KB
yellow_trip_data_sample_2020-06.csv 937KB
yellow_trip_data_sample_2020-04.csv 934KB
yellow_trip_data_sample_2020-05.csv 933KB
yellow_trip_data_sample_2019-01.csv 932KB
yellow_trip_data_sample_2018-10.csv 912KB
yellow_trip_data_sample_2018-11.csv 912KB
yellow_trip_data_sample_2018-12.csv 912KB
yellow_trip_data_sample_2018-09.csv 912KB
yellow_trip_data_sample_2018-05.csv 912KB
yellow_trip_data_sample_2018-04.csv 911KB
yellow_trip_data_sample_2018-08.csv 911KB
yellow_trip_data_sample_2018-07.csv 911KB
yellow_trip_data_sample_2018-06.csv 911KB
yellow_trip_data_sample_2018-02.csv 911KB
yellow_trip_data_sample_2018-03.csv 911KB
yellow_trip_data_sample_2018-01.csv 910KB
yellow_trip_data_9000_lines_sample_2019-03.csv 849KB
yellow_trip_data_8500_lines_sample_2019-02.csv 811KB
yellow_trip_data_7500_lines_sample_2019-01.csv 699KB
distributional_expectations_data_base.csv 74KB
distributional_expectations_data_test.csv 74KB
Titanic.csv 69KB
elements-by-episode.csv 65KB
fixed_distributional_test_dataset.csv 64KB
alice_columnar_table_single_batch_data.csv 739B
toy_data_complete.csv 100B
toy_data_incomplete.csv 80B
same_column_names.csv 68B
strf_test.csv 63B
unicode.csv 24B
null_file.csv 0B
yellow_tripdata.db 6.82MB
titanic_sql_test_cases.db 164KB
test_cases_for_sql_data_connector.db 124KB
titanic.db 92KB
Titanic.feather 100KB
test_basic_project_upgrade_expected_stdout.fixture 3KB
test_basic_project_upgrade_expected_v012_stdout.fixture 3KB
test_project_upgrade_with_manual_steps_expected_stdout.fixture 3KB
test_project_upgrade_with_manual_steps_expected_v012_stdout.fixture 3KB
test_project_upgrade_with_exception_expected_stdout.fixture 2KB
test_project_upgrade_with_exception_expected_v012_stdout.fixture 2KB
test_v2_to_v3_project_upgrade_expected_stdout.fixture 2KB
test_v2_to_v3_project_upgrade_expected_v012_stdout.fixture 2KB
.ge_store_backend_id 55B
.ge_store_backend_id 55B
glossary_scroller.gif 2.44MB
validation_failed_unexpected_values.gif 794KB
.gitignore 90B
.gitignore 13B
.gitignore 0B
.gitignore 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
.gitkeep 0B
BasicDatasetProfiler.html 359KB
BasicDatasetProfiler.html 133KB
BasicDatasetProfiler.html 133KB
c3b4c5df224fef4b1a056a0f3b93aba5.html 258B
c3b4c5df224fef4b1a056a0f3b93aba5.html 258B
index.html 0B
index.html 0B
favicon.ico 1KB
MANIFEST.in 346B
create_test_cases_for_sql_data_connector.ipynb 12KB
validation_playground.ipynb 8KB
validation_playground.ipynb 8KB
validation_playground.ipynb 8KB
共 1136 条
- 1
- 2
- 3
- 4
- 5
- 6
- 12
资源评论
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 基于QT+C++的智能云监护仪项目,能够实时显示使用者心电、血氧、血压波形及其它各种参数+源码(毕业设计&课程设计&项目开发)
- 基于java开发的app接收硬件端传输的心音信号,具有显示心音波形,发出心音的功能+源码(毕业设计&课程设计&项目开发)
- Python 程序语言设计模式思路-行为型模式:职责链模式:将请求从一个处理者传递到下一个处理者
- 9241703124789646.16健身系统2.apk
- postgresql-16.3-1-windows-x64.exe
- Python 程序语言设计模式思路-结构型模式:装饰器讲解及利用Python装饰器模式实现高效日志记录和性能测试
- 基于YOLOv5和DeepSORT的多目标跟踪仿真与记录
- Python 程序语言设计模式思路-创建型模式:原型模式:通过复制现有对象来创建新对象,面向对象编程
- 卸载软件geek卸载软件geek
- Python 程序语言设计模式思路-创建型模式:单例模式,确保一个类的唯一实例(装饰器)面向对象编程、继承
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功