RDIoT Demo

MQTT + Kafka + Amazon ElasticSearch Service [P015]

https://www.youtube.com/watch?v=-8B8Yp52XMY

* GitHub : https://github.com/rdiot/rdiot-p015

* Parts

- Arduino UNO

- Ethernet W5100 Shield (W5100) [B004]

- LCD1602 I2C (LCD1602) [D016]

- Photo Resistor Module (KY-018) [S002]

- Temperature and humidity DHT22 (DHT22) [S063]

- TPM-300 Air Quality Module (TPM-300) [S092]

- Samsung ARTIK 5 (ARTIK5-V0.5) [B023]

- Raspberry Pi 3 B Model (RASPBERRY-PI-3-B) [B088] x 3ea : for Kafka Cluster

- USB Power Supply

- Ethernet Hub

* Contents

1. Amazon ElasticSearch Service Setup

 - Define domain

- Configure cluster

- Set up access

- Review

- Amazon Elasticsearch Service dashboard

2. ES Information example 

- ElasticSearch Endpoint : https://search-rdiot-aws-es-2yolnmmxbjghreiywdbh4yrsay.ap-northeast-2.es.amazonaws.com

- Kibana URL : https://search-rdiot-aws-es-2yolnmmxbjghreiywdbh4yrsay.ap-northeast-2.es.amazonaws.com/_plugin/kibana/

3. MQTT - Kafka - Bridge (json)

- source : https://github.com/rdiot/rdiot-p015/blob/master/MqttToKafkaReConn.java

- result 

{"time": 1511884443315,"value":724}

{"time": 1511884444405,"value":724}

{"time": 1511884445488,"value":723}

{"time": 1511884446579,"value":724}

{"time": 1511884447657,"value":726}

4. install logstash : tested logstash-2.4.1 

$ cd /data1/logstash

$ wget https://download.elastic.co/logstash/logstash/logstash-2.4.1.tar.gz

$ tar zxvf logstash-2.4.1.tar.gz

$ ln -s logstash-2.4.1 logstash

$ sudo apt-get install ant texinfo openjdk-8-jdk build-essential

$ git clone https://github.com/jnr/jffi.git

$ export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-armhf"

$ cd jffi

$ ant jar

$ sudo cp build/jni/libjffi-1.2.so /data1/logstash/logstash/vendor/jruby/lib/jni/arm-Linux

5. logstash config : tested logstash-2.4.1 

- source : https://github.com/rdiot/rdiot-p015/blob/master/logstash-kafka-aws-es.conf

input {

   kafka {

   zk_connect => "kafka-pi-01:2181"

   group_id => "logstash"

   topic_id => "cds"

   consumer_threads => 2

   decorate_events => true

  }

}

filter {

  json {

    source => "message"

  }

  date {

    match => [ "time", "UNIX"]

    target => "time_new"

  }   

}

output {

  elasticsearch {

    index => "test-%{+YYYY.MM.dd}"

    hosts => ["https://search-rdiot-aws-es-2yolnmmxbjghreiywdbh4yrsay.ap-northeast-2.es.amazonaws.com"]

    codec => json

  }

  stdout {

    codec => "rubydebug"

  }

}

6. Amazon ElasticSearch Query Sample

7. Kibana : Search & Chart 

ARTIK MQTT + Raspberry Pi Apache Kafka Cluster Bridge [P014]

* Arduino MQTT Client to ARTIK MQTT Broker

* ARTIK MQTT Broker & Raspberry Pi Kafka Cluster

* Run MQTT-Kafka-Bridge

* Kafka Console Consumer

* KafkaMonitor

https://www.youtube.com/watch?v=JIEIv-pEFFs

*GitHub : https://github.com/rdiot/rdiot-p014.git

* Parts

- Arduino UNO

- Ethernet W5100 Shield (W5100) [B004]

- LCD1602 I2C (LCD1602) [D016]

- Photo Resistor Module (KY-018) [S002]

- Temperature and humidity DHT22 (DHT22) [S063]

- TPM-300 Air Quality Module (TPM-300) [S092]

- Samsung ARTIK 5 (ARTIK5-V0.5) [B023]

- Raspberry Pi 3 B Model (RASPBERRY-PI-3-B) [B088] x 3ea

- USB Power Supply

- Ethernet Hub

* Contents

1.  ARTIK MQTT Start 

- mqtt broker (1883)

$ mosquitto &

2. Add MQTT Topics in the node-red

- node-red (1880)

$ node-red &

3. Arduino MQTT Client 

- source

https://github.com/rdiot/rdiot-p014/blob/master/arduino_mqtt_client.ino

4. MQTT Kakfa Bridge
- source

https://github.com/rdiot/rdiot-p014/blob/master/MqttToKafkaReConn.java

- maven 

    <!--  mqtt-client -->

<dependency>

    <groupId>org.fusesource.mqtt-client</groupId>

    <artifactId>mqtt-client</artifactId>

    <version>1.14</version>

</dependency>

<!-- kafka -->

<dependency>

<groupId>org.apache.kafka</groupId>

<artifactId>kafka_2.9.1</artifactId>

<version>0.8.2.1</version>

</dependency>

- run script

https://github.com/rdiot/rdiot-p014/blob/master/mqtt-kafka-bridge.sh

$ java -jar mqtt-kafka-bridge-0.0.1-SNAPSHOT.jar 

$ nohup java -jar mqtt-kafka-bridge-0.0.1-SNAPSHOT.jar > output.txt &

5. Kafka Console Consumer 

- run script

https://github.com/rdiot/rdiot-p014/blob/master/consumer.sh

$ /data1/kafka/kafka/bin/kafka-console-consumer.sh --zookeeper kafka-pi-01:2181,kafka-pi-02:2181,kafka-pi-03:2181 --topic $1

- run command

$ ./consumer.sh temperature,humidity,cds,airGrade

$ ./consumer.sh temperature

Crash Collision Sensor Detection Module [S230]

https://www.youtube.com/watch?v=1yeZcZQk-Y8

*GitHub : https://github.com/rdiot/rdiot-s230.git

* Specs 

Wrobot Crash Sensor is arduino compatible and can be used for for robot collision detection, touch collision detection. 

It is based on the high sensitivity crash sensor. 

With the Arduino sensor expansion board, in combination, it can be very easy to achieve robot collision detection, touch collision detection. 

When touch, output low level ; when release, maintain a high level

Wrobot Crash Sensor pin definitions : (1) Output (2) Vcc (3) GND

Features :

1.Based on the high sensitivity crash sensor

2.Can be used for robot collision detection, touch collision detection

3.Type : Digital

4.Sensitivity : High

5.Power Supply : 5V

6.Pin Definitions : (1) Output (2) Vcc (3) GND

7.When touch, output low level ; when release, maintain a high level

8.Apply to a variety of platforms including Arduino/51/AVR/ARM

* Contents

- connect

OUT - D3

VCC - 5V

GND - GND

- Key Code