MMM-Gestures是MagicMirror的无头第3方模块，允许通过两种类型的红外传感器的手势控制MagicMir.zip

# MMM-Gestures MMM-Gestures is a head-less 3rd party module for [MagicMirror](https://github.com/MichMich/MagicMirror) that allows to control the Magic Mirror via gestures of two types of infrared sensors. Gestures can be up, down, left, right, far and close movements of the hand in front of a gesture sensor (APDS-9960) and present and away gestures in front of an IR distance sensor (GP2Y0A21YK). ## Functionality * Control magic mirror modules via gestures, e.g. * scroll through news via left / right gestures, show news details (description, full news article) via up gesture * show compliment when person stands in front of mirror via present gesture * reload mirror via far / close gesture * Energy saving through turning off the monitor 5 minutes after use, and turning on the monitor if a person stands in front of the mirror ## Hardware Setup In order to receive gesture events the following hardware is required: * Distance sensor: GP2Y0A21YK, incl. connector cable * Gesture sensor: APDS-9960 on breakout board with gesture sensor on one side, but no other electronic component on the same side for easier assembly (can be found on Aliexpress) * Micro-controller: Arduino (Uno), incl. USB cable * Jumper cable to connect APDS-9960 with Arduino * Pin headers to solder to connector cable of GP2Y0A21YK * Material to hold the sensors, e.g. laser cut sensor module from "Hardware Sensor Module\Mirror Gesture Module.ai" ### Arduino and Sensor Setup The Arduino collects gesture events from gesture sensor APDS-9960 and distance from distance sensor GP2Y0A21YK (10-80cm), which are then forwarded on the serial port as text. The circuit for an Arduino Uno: * Input 1: APDS-9960 on digital pin 2 (interrupt) + I2C (SDA on pin A4, SCL on pin A5) + GND & VCC (3.3V) * Input 2: GP2Y0A21YK on analog pin 0 (analog) + GND & VCC (5V) * Output: serial out on USB ### Laser Cut Sensor Module In order to place the sensors below the mirror you can use "Hardware Sensor Module\Mirror Gesture Module.ai" to cut out wholes of the right size for the APDS-9960 and GP2Y0A21YK sensor. The file contains three elements, in the order they need to be "printed": * Back side: The first "sheet" of 4mm plywood needs to be processed from both sides. The backside cuts out of the whole module (need to adjust outer rectangle width and height to your mirror) and centered engravings for letting the sensors sink into the wood. The front of the gesture sensor APDS-9960 needs to be leveled with the wooden front of the sensor module. On a Zing 40 with 4mm plywood I used as engraving setting speed: 18% and power: 100% to engrave deep enough into the wood. * Front side: The front side of the first "sheet" of 4mm plywood has cut-outs for the sensors' fronts and a decorative engraving. * Frame: In order to hold the glass, panel and backlight, two or three more 4mm plywood sheets should be lasered using the third element from the file. This sheet has generous cut-outs for the sensors and wiring. After wiring the sensors to the Arduino they can be glued to the first sheet from the back. Once dried, the other two to tree frames can be glued on the back of the first sheet. ### Arduino Software Setup In order to compile you'll need to copy the following libraries to Arduino's library folder. The libraries are also included in this git repository. * RunningMedian library, taken from https://github.com/RobTillaart/Arduino/tree/master/libraries/RunningMedian * DistanceSensor library for GP2Y0A21YK, taken from https://github.com/jeroendoggen/Arduino-distance-sensor-library/tree/master/DistanceSensor * SparkFun_APDS9960 library for APDS-9960 gesture sensor, taken from https://github.com/sparkfun/APDS-9960_RGB_and_Gesture_Sensor ### Arduino - Raspberry Pi Setup Once the Arduino is connected to both sensors and the software is setup it will send gesture events to the Raspberry Pi via USB. On the Raspberry Pi a Node.js programm will listen to these events and make them available via a WebSocket server. ## Installation 1. Navigate into your MagicMirror's `modules` folder and execute `git clone https://github.com/thobach/MMM-Gestures.git`. 2. Install dependencies via: `cd MMM-Gestures && npm install` 3. Follow below steps to install the Gestures Node.js App and add the module to your Magic Mirror configuration Note: If you use an older Raspberry Pi 2 or operating system (vs. 3), you may need to change the USB device name in https://github.com/thobach/MMM-Gestures/blob/master/node_helper.js#L104 from `ACM` to `USB`. ### Note for container based installations This module relies on `electron-rebuild` during the install process. If you run MagicMirror in a container (e.g. as described at )https://khassel.gitlab.io/magicmirror/), your host operating system may not recognize the electron version and give you the error `An unhandled error occurred inside electron-rebuild Unable to find electron's version number of MagicMirror, either install it or specify an explicit version`. To avoid this, you'll need to execute the install command from within the container: 1. Run `docker exec -it mm bash`, assuming your container is named `mm` 2. Perform above installation steps 1-3 ### Gestures Node.js App on Raspberry Pi Communication between Raspberry Pi and Arduino happens via the serial port (USB). The `node-helper.js` app from this project creates the connection between the two systems (Raspberry Pi and Arduino), forwards gesture and distance events to the web user interface via the MagicMirror's built-in socket communication and also controls the HDMI display to save power if nobody has interacted with or stood in front of the mirror for 5 minutes. ### Embedding MMM-Gestures In order to load the MMM-Gestures module you will need to add the following configuration to your config/config.js file. ````javascript modules: [ { module: 'MMM-Gestures', }, ] ```` ## Reacting to Gestures The MMM-Gestures.js Magic Mirror module listens to socket events from the `node-helper.js` app and converts them to Magic Mirror-internal events that can be received via the built-in notificationReceived() function. The received event has the following format: * notification parameter: 'GESTURE' * payload parameter: { gesture: 'UP/DOWN/LEFT/RIGHT/FAR/CLOSE/AWAY/PRESENT' } Sample code for reacting to gesture events: ````javascript notificationReceived: function(notification, payload, sender) { Log.info(this.name + " - received event"); if(notification == 'GESTURE'){ Log.info(this.name + " - received gesture"); var gesture = payload.gesture; // actually RIGHT, because gesture sensor is built in upside down if(gesture.startsWith('LEFT')){ Log.info(this.name + " - received right"); // adjust some internal representation ... // update display this.updateDom(100); } // actually LEFT, because gesture sensor is built in upside down else if(gesture.startsWith('RIGHT')){ ... } // gesture event that was neither LEFT or RIGHT received else { Log.info(this.name + " - received other: " + gesture); } } }, ```` By default this module looks for the compliments module and only shows compliments when someone stands in front of the mirror. This is done by hiding the module by default, showing it when the "PRESENT" gesture is received and hiding the module again when the "AWAY" gesture is received. Further the user interface is reloaded when a FAR gesture is received, which can be useful user interface to testing purposes. Available gestures: * Distance sensor GP2Y0A21YK gestures: * AWAY (fired when person is more than ~50cm away from the sensor) * PRESENT (fired when person is less than ~50cm away from the sensor) * Gesture sensor APDS-9960 gestures (only activated when person is in front of the mirror, reacts to movements e.g. of a hand, 3-10cm in front of the sensor): * LEFT (left to right movement with an object) * RIGHT (right to left movement with an object) * UP (bottom to top movemen