RDIoT Demo

Raspberry Pi Apache Kafka Cluster [P013]

https://www.youtube.com/watch?v=dAVwsc1Zhfo

* Parts

- Raspberry Pi 3 B Model (RASPBERRY-PI-3-B) [B088] x 3ea

- USB Power Supply

- Ethernet Hub

* Contents

- Installation & Setup

1. JDK

$ sudo apt-get install openjdk-8-jdk 

2. Kafka

$ cd 

$ mkdir kafka

$ wget http://apache.tt.co.kr/kafka/0.10.1.0/kafka_2.11-0.10.1.0.tgz

$ tar zxvf kafka_2.11-0.10.1.0.tgz

3. hosts

$ vi /etc/hosts

192.168.0.11 kafka-pi-01 

192.168.0.12 kafka-pi-02 

192.168.0.13 kafka-pi-03 

4. Config Zookeeper 

$ vi config/zookeeper.properties (added)

initLimit=5 

syncLimit=2 

server.1=kafka-pi-01:2888:3888 

server.2=kafka-pi-02:2888:3888 

server.3=kafka-pi-03:2888:3888

$ mkdir /tmp/zookeeper (each server)

$ echo 1 > /tmp/zookeeper/myid 

$ echo 2 > /tmp/zookeeper/myid 

$ echo 3 > /tmp/zookeeper/myid 

5. Config Kafka

$ vi config/server.properties (each server 1,2,3)

broker.id=1 

zookeeper.connect=kafka-pi-01:2181,kafka-pi-02:2181,kafka-pi-03:2181

* Demo Test Script

1. Zookeeper Cluster Start

/home/dragon/kafka/kafka/bin/zookeeper-server-start.sh -daemon /home/dragon/kafka/kafka/config/zookeeper.properties

2. Kafka Cluster Start

/home/dragon/kafka/kafka/bin/kafka-server-start.sh -daemon /home/dragon/kafka/kafka/config/server.properties

3. Create topics

/home/dragon/kafka/kafka/bin/kafka-topics.sh --create --zookeeper kafka-pi-01:2181 --replication-factor 3 --partitions 5 -topic rdtest

4. Producer 

/home/dragon/kafka/kafka/bin/kafka-console-producer.sh --broker-list kafka-pi-01:9092 --topic rdtest

5. Consumer

/home/dragon/kafka/kafka/bin/kafka-console-consumer.sh --zookeeper kafka-pi-01:2181 --topic rdtest --from-beginning

6. Start KafkaOffsetMonitor

java -cp KafkaOffsetMonitor-assembly-0.2.1.jar \

com.quantifind.kafka.offsetapp.OffsetGetterWeb \

--zk kafka-pi-01:2181,kafka-pi-02:2181,kafka-pi-03:2181 \

--port 8080 \

--refresh 10.seconds \

--retain 2.days &

7. Monitor Kafka 

http://192.168.0.11:8080

8. How to solve the "map failed" problem in the 32bit raspberry pi.

- edit kafka-run-class.sh 

- replace the KAFKA_JVM_PERFORMANCE_OPTS as follows

# JVM performance options

if [ -z "$KAFKA_JVM_PERFORMANCE_OPTS" ]; then

  #KAFKA_JVM_PERFORMANCE_OPTS="-server -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSScavengeBeforeRemark -XX:+DisableExplicitGC -Djava.awt.headless=true"

  KAFKA_JVM_PERFORMANCE_OPTS="-server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true"

fi

- add KAFKA_HEAP_OPTS under 1G

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

Call server API for multiple sensor information [P002]

https://www.youtube.com/watch?v=6oUOoBihOfo

*GitHub : https://github.com/rdiot/rdiot-p002.git

* Parts

- Arduino Uno R3

- Ethernet W5100 Shield (W5100) [B004]

- LCD1602 (HD44780) [D002]

- Temperature and humidity DHT11 sensor module (KY-015) [S008]

- Button

* Contents

- Collect multiple (two temperature / humidity) sensor values on one sensor and load them into the server

Air Quality Sensor Measurement RGB Matrix Output [P004]

https://www.youtube.com/watch?v=BxdZ1Tk4Mmw

https://www.youtube.com/watch?v=yhVOXsE28B4

 https://www.youtube.com/watch?v=vl9jQZMoH-o

*GitHub : https://github.com/rdiot/rdiot-p004.git

* Parts

- Arduino Mega

- Air Quality Sensor (MQ135) [S037]

- 32x64 RGB LED Matrix P4 (LM-P4-SMD-RGB-LED) [D025]

- SI7021 Humidity Sensor (GY-21) [S068]

- Photo Resistor Module (KY-018) [S002]

* Contents

- Key Code

#include <Adafruit_GFX.h>   // Core graphics library

#include <RGBmatrixPanel.h> // Hardware-specific library

#include <MQ135.h>

#include <Wire.h>

int pin = A5; // Air Quality Sensor

// LED Matrix 

#define OE   9

#define LAT 10

#define CLK 11

#define A   A0

#define B   A1

#define C   A2

#define D   A3

RGBmatrixPanel matrix(A, B, C, D, CLK, LAT, OE, false, 64);

MQ135 gasSensor = MQ135(pin);

#define RZERO 76.63

double rzero;

double result;

int i=0;

double rzero;

double result;

void setup() {
  matrix.begin(); // LED Matrix init
  matrix.drawPixel(0, 0, matrix.Color333(7, 7, 7));
  delay(500);

  matrix.fillRect(0, 0, 64, 32, matrix.Color333(0, 7, 0));
  delay(3000);
}

void loop() {

  matrix.fillScreen(matrix.Color333(0, 0, 0));
  matrix.setCursor(1, 1);    // start at top left, with 8 pixel of spacin

  if (i==0) {
   rzero = gasSensor.getRZero(); // float
  }

  if (i>0) { 
   result = gasSensor.getRZero();
   rzero = (rzero + result)/2;
  }

  float co2_ppm = gasSensor.getPPM();

  matrix.println("rz:"+(String)rzero+" ");
  matrix.setCursor(1, 9);    // start at top left, with 8 pixel of spacing
  matrix.println("rs:"+(String)result+" ");
  matrix.setCursor(1, 16);    // start at top left, with 8 pixel of spacing
  matrix.println("D:"+(String)analogRead(pin)+" ");
  matrix.setCursor(1, 23);    // start at top left, with 8 pixel of spacing
  matrix.println("A:"+(String)co2_ppm+"ppm");

  i++;

  if(co2_ppm > 10)

  {

    matrix.drawRect(0, 0, 64, 32, matrix.Color333(0, 4, 7));

  }

  else

  {

  // matrix.drawRect(0, 0, 64, 32, matrix.Color333(0, 4, 7));

  }

  //cool down
  delay(1000);

}