PyPI 官网下载 | pyoptimus-22.1.0b3.tar.gz

# Optimus [](https://hi-optimus.com) [](https://github.com/hi-primus/optimus/actions/workflows/main.yml) [](https://hub.docker.com/r/hiprimus/optimus) [](https://pypi.org/project/pyoptimus/) [](https://github.com/hi-primus/optimus/releases) [](http://calver.org) [](https://pepy.tech/project/pyoptimus) [](https://pepy.tech/project/pyoptimus/month) [](https://pepy.tech/project/pyoptimus/week) [](https://github.com/bulutyazilim/awesome-datascience) [](https://communityinviter.com/apps/hi-bumblebee/welcome) # Overview Optimus is an opinionated python library to easily load, process, plot and create ML models that run over pandas, Dask, cuDF, dask-cuDF, Vaex or Spark. Some amazing things Optimus can do for you: * Process using a simple API, making it easy to use for newcomers. * More than 100 functions to handle strings, process dates, urls and emails. * Easily plot data from any size. * Out of box functions to explore and fix data quality. * Use the same code to process your data in your laptop or in a remote cluster of GPUs. [See Documentation](https://docs.hi-optimus.com/en/latest/) ## Try Optimus To launch a live notebook server to test optimus using binder or Colab, click on one of the following badges: [](https://mybinder.org/v2/gh/hi-primus/optimus/develop-22.1?filepath=https%3A%2F%2Fraw.githubusercontent.com%2Fhi-primus%2Foptimus%2Fdevelop-22.1%2Fexamples%2F10_min_to_optimus.ipynb) [](https://colab.research.google.com/github/hi-primus/optimus/blob/master/examples/10_min_to_optimus_colab.ipynb) ## Installation (pip): In your terminal just type: ``` pip install pyoptimus ``` By default Optimus install Pandas as the default engine, to install other engines you can use the following commands: | Engine | Command | |-----------|----------------------------------------| | Dask | ```pip install pyoptimus[dask]``` | | cuDF | ```pip install pyoptimus[cudf]``` | | Dask-cuDF | ```pip install pyoptimus[dask-cudf]``` | | Vaex | ```pip install pyoptimus[vaex]``` | | Spark | ```pip install pyoptimus[spark]``` | To install from the repo: ``` pip install git+https://github.com/hi-primus/optimus.git@develop-22.1 ``` To install other engines: ``` pip install git+https://github.com/hi-primus/optimus.git@develop-22.1#egg=pyoptimus[dask] ``` ### Requirements * Python 3.7 or 3.8 ## Examples You can go to [10 minutes to Optimus](https://github.com/hi-primus/optimus/blob/develop-22.1/examples/10_min_to_optimus.ipynb) where you can find the basics to start working in a notebook. Also you can go to the [Examples](https://github.com/hi-primus/optimus/tree/develop-22.1/examples/examples.md) section and find specific notebooks about data cleaning, data munging, profiling, data enrichment and how to create ML and DL models. Here's a handy [Cheat Sheet](https://htmlpreview.github.io/?https://github.com/hi-primus/optimus/blob/develop-22.1/docs/cheatsheet/optimus_cheat_sheet.html) with the most common Optimus' operations. ## Start Optimus Start Optimus using ```"pandas"```, ```"dask"```, ```"cudf"```,```"dask_cudf"```,```"vaex"``` or ```"spark"```. ```python from optimus import Optimus op = Optimus("pandas") ``` ## Loading data Now Optimus can load data in csv, json, parquet, avro and excel formats from a local file or from a URL. ```python #csv df = op.load.csv("../examples/data/foo.csv") #json df = op.load.json("../examples/data/foo.json") # using a url df = op.load.json("https://raw.githubusercontent.com/hi-primus/optimus/develop-22.1/examples/data/foo.json") # parquet df = op.load.parquet("../examples/data/foo.parquet") # ...or anything else df = op.load.file("../examples/data/titanic3.xls") ``` Also, you can load data from Oracle, Redshift, MySQL and Postgres databases. ## Saving Data ```python #csv df.save.csv("data/foo.csv") # json df.save.json("data/foo.json") # parquet df.save.parquet("data/foo.parquet") ``` You can also save data to oracle, redshift, mysql and postgres. ## Create dataframes Also, you can create a dataframe from scratch ```python df = op.create.dataframe({ 'A': ['a', 'b', 'c', 'd'], 'B': [1, 3, 5, 7], 'C': [2, 4, 6, None], 'D': ['1980/04/10', '1980/04/10', '1980/04/10', '1980/04/10'] }) ``` Using `display` you have a beautiful way to show your data with extra information like column number, column data type and marked white spaces. ```python display(df) ```  ## Cleaning and Processing Optimus was created to make data cleaning a breeze. The API was designed to be super easy to newcomers and very familiar for people that comes from Pandas. Optimus expands the standard DataFrame functionality adding `.rows` and `.cols` accessors. For example you can load data from a url, transform and apply some predefined cleaning functions: ```python new_df = df\ .rows.sort("rank", "desc")\ .cols.lower(["names", "function"])\ .cols.date_format("date arrival", "yyyy/MM/dd", "dd-MM-YYYY")\ .cols.years_between("date arrival", "dd-MM-YYYY", output_cols="from arrival")\ .cols.normalize_chars("names")\ .cols.remove_special_chars("names")\ .rows.drop(df["rank"]>8)\ .cols.rename("*", str.lower)\ .cols.trim("*")\ .cols.unnest("japanese name", output_cols="other names")\ .cols.unnest("last position seen", separator=",", output_cols="pos")\ .cols.drop(["last position seen", "japanese name", "date arrival", "cybertronian", "nulltype"]) ``` # Need help? 🛠️ ## Feedback Feedback is what drive Optimus future, so please take a couple of minutes to help shape the Optimus' Roadmap: http://bit.ly/optimus_survey Also if you want to a suggestion or feature request use https://github.com/hi-primus/optimus/issues ## Troubleshooting If you have issues, see our [Troubleshooting Guide](https://github.com/hi-primus/optimus/tree/develop-22.1/troubleshooting.md) # Contributing to Optimus 💡 Contributions go far beyond pull requests and commits. We are very happy to receive any kind of contributions including: * [Documentation](https://docs.hi-optimus.com/en/latest/) updates, enhancements, designs, or bugfixes. * Spelling or grammar fixes. * README.md corrections or redesigns. * Adding unit, or functional [tests](https://github.com/hi-primus/optimus/tree/develop-22.1/tests) * Triaging GitHub issues -- especially determining whether an issue still persists or is reproducible. * [Blogging, speaking about, or creating tutorials](https://hioptimus.com/category/blog/) about Optimus and its many features. * Helping others on our official chats # Backers and Sponsors Become a [backer](https://opencollective.com/optimus#backer) or a [sponsor](https://opencollective.com/optimus#sponsor) and get your image on our README on Github with a link to your site. [](#backers) [![OpenCollective](https:/