pandas-profiling-3.6.3.tar.gz

# `pandas-profiling`  [](https://github.com/ydataai/pandas-profiling/actions/workflows/tests.yml) [](https://pypi.python.org/pypi/pandas-profiling/) [](https://pypi.org/project/pandas-profiling/) [](https://codecov.io/gh/ydataai/pandas-profiling) [](https://github.com/ydataai/pandas-profiling/releases) [](https://pypi.org/project/pandas-profiling/) [](https://github.com/python/black) <p align="center"> <a href="https://pandas-profiling.ydata.ai/docs/master/">Documentation</a> | <a href="https://discord.com/invite/mw7xjJ7b7s">Discord</a> | <a href="https://stackoverflow.com/questions/tagged/pandas-profiling">Stack Overflow</a> | <a href="https://pandas-profiling.ydata.ai/docs/master/pages/reference/changelog.html#changelog">Latest changelog</a> </p> <p align="center"> Do you like this project? Show us your love and <a href="https://engage.ydata.ai">give feedback!</a> </p> `pandas-profiling` primary goal is to provide a one-line Exploratory Data Analysis (EDA) experience in a consistent and fast solution. Like pandas `df.describe()` function, that is so handy, pandas-profiling delivers an extended analysis of a DataFrame while alllowing the data analysis to be exported in different formats such as **html** and **json**. The package outputs a simple and digested analysis of a dataset, including **time-series** and **text**. ### Key features - **Type inference**: automatic detection of columns' data types (*Categorical*, *Numerical*, *Date*, etc.) - **Warnings**: A summary of the problems/challenges in the data that you might need to work on (*missing data*, *inaccuracies*, *skewness*, etc.) - **Univariate analysis**: including descriptive statistics (mean, median, mode, etc) and informative visualizations such as distribution histograms - **Multivariate analysis**: including correlations, a detailed analysis of missing data, duplicate rows, and visual support for variables pairwise interaction - **Time-Series**: including different statistical information relative to time dependent data such as auto-correlation and seasonality, along ACF and PACF plots. - **Text analysis**: most common categories (uppercase, lowercase, separator), scripts (Latin, Cyrillic) and blocks (ASCII, Cyrilic) - **File and Image analysis**: file sizes, creation dates, dimensions, indication of truncated images and existence of EXIF metadata - **Compare datasets**: one-line solution to enable a fast and complete report on the comparison of datasets - **Flexible output formats**: all analysis can be exported to an HTML report that can be easily shared with different parties, as JSON for an easy integration in automated systems and as a widget in a Jupyter Notebook. The report contains three additional sections: - **Overview**: mostly global details about the dataset (number of records, number of variables, overall missigness and duplicates, memory footprint) - **Alerts**: a comprehensive and automatic list of potential data quality issues (high correlation, skewness, uniformity, zeros, missing values, constant values, between others) - **Reproduction**: technical details about the analysis (time, version and configuration) > ### 🎁 Latest features > - Looking for how you can do an EDA for Time-Series 🕛 ? Check [this blogpost](https://towardsdatascience.com/how-to-do-an-eda-for-time-series-cbb92b3b1913). > - You want to compare 2 datasets and get a report? Check [this blogpost](https://medium.com/towards-artificial-intelligence/how-to-compare-2-dataset-with-pandas-profiling-2ae3a9d7695e) ## 📝 Use cases Pandas-profiling can be used to deliver a variety of different use-case. The documentation includes guides, tips and tricks for tackling them: | Use case | Description | |----------|----------------------------------------------------------------------------------------------| | [Comparing datasets](https://pandas-profiling.ydata.ai/docs/master/pages/use_cases/comparing_datasets.html ) | Comparing multiple version of the same dataset | | [Profiling a Time-Series dataset](https://pandas-profiling.ydata.ai/docs/master/pages/use_cases/time_series_datasets.html) | Generating a report for a time-series dataset with a single line of code | |[Profiling large datasets](https://pandas-profiling.ydata.ai/docs/master/pages/use_cases/big_data.html ) | Tips on how to prepare data and configure `pandas-profiling` for working with large datasets | | [Handling sensitive data](https://pandas-profiling.ydata.ai/docs/master/pages/use_cases/sensitive_data.html ) | Generating reports which are mindful about sensitive data in the input dataset | | [Dataset metadata and data dictionaries](https://pandas-profiling.ydata.ai/docs/master/pages/use_cases/metadata.html) | Complementing the report with dataset details and column-specific data dictionaries | | [Customizing the report's appearance](https://pandas-profiling.ydata.ai/docs/master/pages/use_cases/custom_report_appearance.html ) | Changing the appearance of the report's page and of the contained visualizations | > ⚡ Looking for a Spark backend to profile large datasets? It's [work in progress](https://github.com/ydataai/pandas-profiling/projects/3). ## ▶️ Quickstart Start by loading your pandas `DataFrame` as you normally would, e.g. by using: ```python import numpy as np import pandas as pd from pandas_profiling import ProfileReport df = pd.DataFrame(np.random.rand(100, 5), columns=["a", "b", "c", "d", "e"]) ``` To generate the standard profiling report, merely run: ```python profile = ProfileReport(df, title="Pandas Profiling Report") ``` ### Using inside Jupyter Notebooks There are two interfaces to consume the report inside a Jupyter notebook: through widgets and through an embedded HTML report. <img alt="Notebook Widgets" src="https://pandas-profiling.ydata.ai/docs/master/assets/widgets.gif" width="800" /> The above is achieved by simply displaying the report as a set of widgets. In a Jupyter Notebook, run: ```python profile.to_widgets() ``` The HTML report can be directly embedded in a cell in a similar fashion: ```python profile.to_notebook_iframe() ``` <img alt="HTML" src="https://pandas-profiling.ydata.ai/docs/master/assets/iframe.gif" width="800" /> ### Exporting the report to a file To generate a HTML report file, save the `ProfileReport` to an object and use the `to_file()` function: ```python profile.to_file("your_report.html") ``` Alternatively, the report's data can be obtained as a JSON file: ```python # As a JSON string json_data = profile.to_json() # As a file profile.to_file("your_report.json") ``` ### Using in the command line For standard formatted CSV files (which can be read directly by pandas without additional settings), the `pandas_profiling` executable can be used in the command line. The example below generates a report named _Example Profiling Report_, using a configuration file called `default.yaml`, in the file `report.html` by processing a `data.csv` dataset. ```sh pandas_profiling --title "Exampl