一些有用的自定义配置单元udf函数、特殊数组、json、数学、字符串函数。___下载.zip

# hive-third-functions [](https://www.shanruifeng.win) [](https://travis-ci.org/aaronshan/hive-third-functions) [](https://github.com/aaronshan/hive-third-functions/tree/master/README.md) [](https://github.com/aaronshan/hive-third-functions/tree/master/README-zh.md) [](https://github.com/aaronshan/hive-third-functions/releases) [](https://github.com/aaronshan/hive-third-functions) ## Introduction Some useful custom hive udf functions, especial array and json functions. > Note: > hive-third-functions support hive-0.11.0 or higher. ## Build ### 1. install dependency Now, jdo2-api-2.3-ec.jar not available in the maven central repository, so we have to manually install it into our local maven repository. ``` wget http://www.datanucleus.org/downloads/maven2/javax/jdo/jdo2-api/2.3-ec/jdo2-api-2.3-ec.jar -O ~/jdo2-api-2.3-ec.jar mvn install:install-file -DgroupId=javax.jdo -DartifactId=jdo2-api -Dversion=2.3-ec -Dpackaging=jar -Dfile=~/jdo2-api-2.3-ec.jar ``` ### 2. mvn package ``` cd ${project_home} mvn clean package ``` If you want to skip unit tests, please run: ``` cd ${project_home} mvn clean package -DskipTests ``` It will generate hive-third-functions-${version}-shaded.jar in target directory. You can also directly download file from [release page](https://github.com/aaronshan/hive-third-functions/releases). > current latest version is `2.2.1` ## Maven Now, I had already release `hive-third-functions` to maven repositories. To add a dependency on `hive-third-functions` using Maven, use the following: ``` <dependency> <groupId>com.github.aaronshan</groupId> <artifactId>hive-third-functions</artifactId> <version>2.2.1</version> </dependency> ``` ## Functions ### 1. string functions | function| description | |:--|:--| |pinyin(string) -> string | convert chinese to pinyin| |md5(string) -> string | md5 hash| |sha256(string) -> string |sha256 hash| |codepoint(string) -> integer | Returns the Unicode code point of the only character of string.| |hamming_distance(string1, string2) -> bigint | Returns the Hamming distance of string1 and string2.| |levenshtein_distance(string1, string2) -> bigint | Returns the Levenshtein edit distance of string1 and string2.| |normalize(string, form) -> varchar | Transforms string with the specified normalization form. form must be be one of the following keywords: <span id="jump">Normalize Form Description</span> | |strpos(string, substring) -> bigint | Returns the starting position of the first instance of substring in string. Positions start with 1. If not found, 0 is returned.| |split_to_map(string, entryDelimiter, keyValueDelimiter) -> map&lt;varchar, varchar> | Splits string by entryDelimiter and keyValueDelimiter and returns a map. entryDelimiter splits string into key-value pairs. keyValueDelimiter splits each pair into key and value.| |split_to_multimap(string, entryDelimiter, keyValueDelimiter) -> map(varchar, array(varchar)) | Splits string by entryDelimiter and keyValueDelimiter and returns a map containing an array of values for each unique key. entryDelimiter splits string into key-value pairs. keyValueDelimiter splits each pair into key and value. The values for each key will be in the same order as they appeared in string.| [Normalize Form Description](#jump) | Form | Description | |:--|:--| | NFD | Canonical Decomposition | | NFC | Canonical Decomposition, followed by Canonical Composition | | NFKD | Compatibility Decomposition | | NFKC | Compatibility Decomposition, followed by Canonical Composition | ### 2. array functions | function| description | |:--|:--| |array_contains(array&lt;E&gt;, E) -> boolean | whether array contains value or not.| |array_equals(array&lt;E&gt;, array&lt;E&gt;) -> boolean | whether two array equals or not.| |array_intersect(array, array) -> array | returns the two array's intersection, without duplicates.| |array_max(array&lt;E&gt;) -> E | returns the maximum value of input array.| |array_min(array&lt;E&gt;) -> E | returns the minimum value of input array.| |array_join(array, delimiter, null_replacement) -> string | concatenates the elements of the given array using the delimiter and an optional `null_replacement` to replace nulls.| |array_distinct(array) -> array | remove duplicate values from the array.| |array_position(array&lt;E&gt;, E) -> long | returns the position of the first occurrence of the element in array (or 0 if not found).| |array_remove(array&lt;E&gt;, E) -> array | remove all elements that equal element from array.| |array_reverse(array) -> array | reverse the array element.| |array_sort(array) -> array | sorts and returns the array. The elements of array must be orderable.| |array_concat(array, array) -> array | concatenates two arrays.| |array_value_count(array&lt;E&gt;, E) -> long | count array's element number that element value equals given value.| |array_slice(array, start, length) -> array | subsets array starting from index start (or starting from the end if start is negative) with a length of length.| |array_element_at(array&lt;E&gt;, index) -> E | returns element of array at given index. If index < 0, element_at accesses elements from the last to the first.| |array_shuffle(array) -> array | Generate a random permutation of the given array x.| |sequence(start, end) -> array<Long> | Generate a sequence of integers from start to stop.| |sequence(start, end, step) -> array<Long> | Generate a sequence of integers from start to stop, incrementing by step.| |sequence(start_date_string, end_data_string, step) -> array<String> | Generate a sequence of date string from start to stop, incrementing by step.| |array_value_count(array&lt;E&gt;, E) -> long | count array's element number that element value equals given value..| ### 3. map functions | function| description | |:--|:--| |map_build(x&lt;K&gt;, y&lt;V&gt;) -> map&lt;K, V&gt;| returns a map created using the given key/value arrays.| |map_concat(x&lt;K, V&gt;, y&lt;K, V&gt;) -> map&lt;K,V&gt; | returns the union of two maps. If a key is found in both `x` and `y`, that key’s value in the resulting map comes from `y`.| |map_element_at(map&lt;K, V&gt;, key) -> V | returns value for given `key`, or `NULL` if the key is not contained in the map.| |map_equals(x&lt;K, V&gt;, y&lt;K, V&gt;) -> boolean | whether map x equals with map y or not.| ### 4. date functions | function| description | |:--|:--| |day_of_week(date_string \| date) -> int | day of week,if monday,return 1, sunday return 7, error return null.| |day_of_year(date_string \| date) -> int | day of year. The value ranges from 1 to 366.| |zodiac_en(date_string \| date) -> string | convert date to zodiac| |zodiac_cn(date_string \| date) -> string | convert date to zodiac chinese | |type_of_day(date_string \| date) -> string | for chinese. 获取日期的类型(1: 法定节假日, 2: 正常周末, 3: 正常工作日 4:攒假的工作日),错误返回-1. | ### 5. json functions | function| description | |:--|:--| |json_array_get(json, jsonPath) -> array(varchar) |returns the element at the specified index into the `json_array`. The index is zero-based.| |json_array_length(json, jsonPath) -> array(varchar) |returns the array length of `json` (a string containing a JSON array).| |json_array_extract(json, jsonPath) -> array(varchar) |extract json array by given jsonPath.| |json_array_extract_scalar(json, jsonPath) -> array(varchar) |like `json_array_extract`, but returns the