# Indian-Sign-Language-Recognition
Sign Languages are a set of languages that use predefined actions and movements to convey a message. These languages are primarily developed to aid deaf and other verbally challenged people. They use a simultaneous and precise combination of movement of hands, orientation of hands, hand shapes etc. Different regions have different sign languages like American Sign Language, Indian Sign Language etc. We focus on Indian Sign language in this project.
Indian Sign Language (ISL) is a sign language that is predominantly used in South Asian countries. It is sometimes referred to as Indo-Pakistani Sign Language (IPSL). There are many special features present in ISL that distinguish it from other Sign Languages. Features like Number Signs, Family Relationship, use of space etc. are crucial features of ISL. Also, ISL does not have any temporal inflection.
In this project, we aim towards analyzing and recognizing various alphabets from a database of sign images. Database consists of various images with each image clicked in different light condition with different hand orientation. With such a divergent data set, we are able to train our system to good levels and thus obtain good results.
We investigate different machine learning techniques like Support Vector Machines (SVM), Logistic Regression, K-nearest neighbors (KNN) and a neural network technique Convolution Neural Networks (CNN) for detection of sign language.
## Getting Started
### Prerequisites
Before running this project, make sure you have following dependencies -
* [Dataset](https://drive.google.com/folderview?id=0Bw239KLrN7zofmxvSmtsVHlrbkFRY1NwMjh2NFJGX1ZtY0lKOTR0REJnQnBUdVgyVDlMMkk&usp=sharing) (Download the images from this link)
* [Python 2.7](https://www.python.org/downloads/)
* [pip](https://pypi.python.org/pypi/pip)
* [OpenCV](https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_setup/py_setup_in_windows/py_setup_in_windows.html)
Now, using ```pip install``` command, include following dependencies
+ Numpy
+ Pandas
+ Sklearn
+ Scipy
+ Opencv
+ Tensorflow
### Running
To run the project, perform following steps -
1. Separate the test folder from images folder and rename the images folder to train.
2. Make sure you have both of these folders and all python files in the same folder.
3. Run the preprocessing_surf.py file to make the csv file of training data set.
4. classification.py contains the code for svm, knn and many other classifiers.
5. cnn.py contains the code for deep learning as the name suggests.
## Workflow
<p align="center">
<br>
<img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Visualization/flowchart.jpg">
</p>
### Image Preprocessing
#### Segmentation:
The main objective of the segmentation phase is to remove the background and noises, leaving only the Region of Interest (ROI), which is the only useful information in the image. This is achieved via Skin Masking defining the threshold on RGB schema and then converting RGB colour space to grey scale image. Finally Canny Edge technique is employed to identify and detect the presence of sharp discontinuities in an image, thereby detecting the edges of the figure in focus.
<p align="center">
<br>
<img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Processed_images/BGR2HSV.png"> <img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Processed_images/masked.png"> <img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Processed_images/canny%20edge%20detection.png">
<br>
BGR to HSV           Masked           Canny Edge
</p>
<br>
#### Feature Extraction:
The Speeded Up Robust Feature (SURF) technique is used to extract descriptors from the segmented hand gesture images. SURF is a novel feature extraction method which is robust against rotation, scaling, occlusion and variation in viewpoint.
<p align="center">
<br>
<img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Processed_images/SURF_D.png">
</p>
### Classification
The SURF descriptors extracted from each image are different in number with the same dimension (64). However, a multiclass SVM requires uniform dimensions of feature vector as its input. Bag of Features (BoF) is therefore implemented to represent the features in histogram of visual vocabulary rather than the features as proposed. The descriptors extracted are first quantized into 150 clusters using K-means clustering. Given a set of descriptors, where K-means clustering categorizes numbers of descriptors into K numbers of cluster center.
The clustered features then form the visual vocabulary where each feature corresponds to an individual sign language gesture. With the visual vocabulary, each image is represented by the frequency of occurrence of all clustered features. BoF represents each image as a histogram of features, in this case the histogram of 24 classes of sign languages gestures.
#### Bag of Features model
Following Steps are followed to achieve this:
* The descriptors extracted are first clustered into 150 clusters using K-Means clustering.
* K-means clustering technique categorizes m numbers of descriptors into x number of cluster centre.
* The clustered features form the basis for histogram i-e each image is represented by frequency of occurrence of all clustered features.
* BoF represents each image as a histogram of features, in our case the histogram of 24 classes of sign language is generated.
#### Classifiers
After obtaining the baf of features model, we are set to predict results for new raw images to test our model. Following classifiers are used :
+ Naive Bayes
+ Logistic Regression classifier
+ K-Nearest Neighbours
+ Support Vector Machines
+ Convolution Neaural Network
### Results
Results can be visualized by running file [visualize_submissions.py](https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Visualization/visualize_submissions.py).
#### Accuracy without SURF
<p align="center">
<br>
<img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Visualization/accuracy_without_surf.png">
<br>
</p>
#### Accuracy with SURF
<p align="center">
<br>
<img align="center" src="https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition/blob/master/Visualization/acuracy_with_surf.png">
</p>
没有合适的资源?快使用搜索试试~ 我知道了~
Indian-Sign-Language-Recognition-master.zip_Sign to voice_sign l
共29个文件
png:15个
py:9个
md:1个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 98 浏览量
2022-09-24
22:27:02
上传
评论
收藏 548KB ZIP 举报
温馨提示
Communication betwem normal and deaf people
资源推荐
资源详情
资源评论
收起资源包目录
Indian-Sign-Language-Recognition-master.zip (29个子文件)
Indian-Sign-Language-Recognition-master
Bag of Features
preprocessing_orb.py 7KB
preprocessing_surf.py 8KB
classification
cnn.py 5KB
saved_cnn.py 3KB
classification.py 2KB
Image Preprocessing
preprocessing.py 1KB
surf_image_processing.py 3KB
image_processing.py 2KB
Visualization
recognition_rate.png 23KB
flowchart.jpg 41KB
accuracy_without_surf.png 14KB
visualize_submissions.py 2KB
acuracy_with_surf.png 13KB
LICENSE 1KB
.gitignore 1KB
README.md 7KB
Alphabets.docx 332KB
Processed_images
canny edge detection_U.png 1KB
BGR2HSV.png 21KB
SURF.png 30KB
SURF_U.png 22KB
canny edge detection.png 2KB
SURF_D.png 23KB
canny edge detection_W.png 1KB
BGR2GRAY.png 22KB
canny edge detection_D.png 1KB
medianBlur.png 1013B
SURF_W.png 24KB
masked.png 11KB
共 29 条
- 1
资源评论
小波思基
- 粉丝: 72
- 资源: 1万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功