Articles Embedded Projects Setting up InfluxDB and Grafana for storing and viewing Arduino measured sensor values

Setting up InfluxDB and Grafana for storing and viewing Arduino measured sensor values

Saeid Yazdani 01-01-2017 29-01-2017Embedded Projects

Sending data to InfluxDB using Arduino over HTTP

We are going to use an Arduino (any model) with an Ethernet shield. You can also perform the same task with WiFi shield with some minor changes in the code.

Arduino Mega with Ethernet shield

The code

Since I only have 1 Arduino at hand and no actual sensors, I am going to emulate the test scenario in the code. But You can of course change to code according to your needs. The important part of the code is how to write into the buffer and how to calculate exact length of the content so we can make the HTTP parser of InfluxDB happy! This can be done through the magic of sprintf function which can write into a buffer char array and return number of written bytes.

Here is the code, I have comment it throughly so it should be obvious what it does. If you have any questions, feel free to ask in the comment section or in the embedonix’s forum.


#include <Ethernet.h>

/** Ethernet client instance */
EthernetClient client;

/** Default local IP address, in case of DHCP IP assignement fails */
const IPAddress eth_default_ip( 192,  168,  0,  99);

/** MAC Address of the ethernet shield */ 
byte eth_mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0xCD, 0xB8 };

/** Server address wich runs InfluxDB */
const byte eth_server[] = {192, 168, 0, 1};

/** InfluxDB HTTP port */
const int eth_port = 8086;


/** Size of the buffer for HTTP content */
const int bufferSize = 2048;

/** An character array filled with null terminate chars */
char buf[bufferSize] = {'\0'};

//some variables for generating places
const char kitchen[] = "kitchen";
const char garage[] = "garage";
const char living_room[] = "living_room";
const char basement[] = "basement";
int lastGivenPlace = 0;

/**
 * @brief: Returns a place out of 4 places we have!
 */
const char* get_place() {
  switch(lastGivenPlace) {
    case 0:
    lastGivenPlace = 1;
    return &kitchen[0];

    case 1:
    lastGivenPlace = 2;
    return &garage[0];

    case 2:
    lastGivenPlace = 3;
    return &living_room[0];

    case 3:
    lastGivenPlace = 0;
    return &basement[0];    
  }

  return "unknown_place";
}

/**
 * @brief Starts the ethernet shield as client
 */
bool eth_start(){
   Ethernet.begin(eth_mac, eth_default_ip);
   delay(2000); //delay to allow connection to be done

   //do a fast test if we can connect to server
   int conState = client.connect(eth_server, eth_port);

   if(conState > 0) {
    Serial.println("Connected to InfluxDB server");
    client.stop();
    return true;
   }

  //print the error number and return false
  Serial.print("Could not connect to InfluxDB Server, Error #");
  Serial.println(conState);
  return false;
}

/**
 * @brief Send  HTTP data to InfluxDB
 * @param data  Pointer to the beginning of the buffer
 * @param dataSize  Number of valid characters to send
 */
void eth_send_data(char* data, int dataSize) {
  //first we need to connect to InfluxDB server   
  int conState = client.connect(eth_server, eth_port);

  if(conState <= 0) { //check if connection to server is stablished
    Serial.print("Could not connect to InfluxDB Server, Error #");
    Serial.println(conState);
    return;
  }
  
  //Send HTTP header and buffer
  client.println("POST /write?db=embedonix HTTP/1.1");
  client.println("Host: www.embedonix.com");
  client.println("User-Agent: Arduino/1.0");
  client.println("Connection: close");
  client.println("Content-Type: application/x-www-form-urlencoded");
  client.print("Content-Length: ");
  client.println(dataSize);
  client.println();
  client.println(data);

  delay(50); //wait for server to process data

  //Now we read what server has replied and then we close the connection
  Serial.println("Reply from InfluxDB");
  while(client.available()) { //receive char
    Serial.print((char)client.read());
  }  
  Serial.println(); //empty line

  client.stop();  
}

/**
 * @brief Setups the peripherals
 */
void setup() {
  //Serial interface for debugging purposes
  Serial.begin(115200);
  delay(1000);
  eth_start(); 
}



/**
 * @brief Main loop
 */
void loop() {

  //A loop to send some data
  for(int i = 30; i < 50  ; i++) {
    
    //this will be number of chars written to buffer, return value of sprintf
    int numChars = 0;
    
    //First of all we need to add the name of measurement to beginning of the buffer
    numChars = sprintf(buf, "my_house_data,");
    
    //tag should have an space at the end 
    numChars += sprintf(&buf[numChars], "SOURCE=arduino_1,PLACE=%s ", get_place());
    
    //after tags, comes the values!
    numChars += sprintf(&buf[numChars], "TEMP=%d,", i);
    numChars += sprintf(&buf[numChars], "HUMIDITY=%.2f,", i * 1.03);
    numChars += sprintf(&buf[numChars], "LIGHT=%.2f", i * 10.5);

    //Print the buffer on the serial line to see how it looks
    Serial.print("Sending following dataset to InfluxDB: ");
    Serial.println(buf);

    //send to InfluxDB
    eth_send_data(buf, numChars);

    //we have to reset the buffer at the end of loop
    memset(buf, '\0', bufferSize);
    delay(1000); //some small delay!
  }
  
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

#include <Ethernet.h>

/** Ethernet client instance */

EthernetClient client;

/** Default local IP address, in case of DHCP IP assignement fails */

const IPAddress eth_default_ip( 192, 168, 0, 99);

/** MAC Address of the ethernet shield */

byte eth_mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0xCD, 0xB8 };

/** Server address wich runs InfluxDB */

const byte eth_server[] = {192, 168, 0, 1};

/** InfluxDB HTTP port */

const int eth_port = 8086;

/** Size of the buffer for HTTP content */

const int bufferSize = 2048;

/** An character array filled with null terminate chars */

char buf[bufferSize] = {'\0'};

//some variables for generating places

const char kitchen[] = "kitchen";

const char garage[] = "garage";

const char living_room[] = "living_room";

const char basement[] = "basement";

int lastGivenPlace = 0;

/**

* @brief: Returns a place out of 4 places we have!

const char* get_place() {

switch(lastGivenPlace) {

case 0:

lastGivenPlace = 1;

return &kitchen[0];

case 1:

lastGivenPlace = 2;

return &garage[0];

case 2:

lastGivenPlace = 3;

return &living_room[0];

case 3:

lastGivenPlace = 0;

return &basement[0];

}

return "unknown_place";

}

/**

* @brief Starts the ethernet shield as client

bool eth_start(){

Ethernet.begin(eth_mac, eth_default_ip);

delay(2000); //delay to allow connection to be done

//do a fast test if we can connect to server

int conState = client.connect(eth_server, eth_port);

if(conState > 0) {

Serial.println("Connected to InfluxDB server");

client.stop();

return true;

}

//print the error number and return false

Serial.print("Could not connect to InfluxDB Server, Error #");

Serial.println(conState);

return false;

}

/**

* @brief Send HTTP data to InfluxDB

* @param data Pointer to the beginning of the buffer

* @param dataSize Number of valid characters to send

void eth_send_data(char* data, int dataSize) {

//first we need to connect to InfluxDB server

int conState = client.connect(eth_server, eth_port);

if(conState <= 0) { //check if connection to server is stablished

Serial.print("Could not connect to InfluxDB Server, Error #");

Serial.println(conState);

return;

}

//Send HTTP header and buffer

client.println("POST /write?db=embedonix HTTP/1.1");

client.println("Host: www.embedonix.com");

client.println("User-Agent: Arduino/1.0");

client.println("Connection: close");

client.println("Content-Type: application/x-www-form-urlencoded");

client.print("Content-Length: ");

client.println(dataSize);

client.println();

client.println(data);

delay(50); //wait for server to process data

//Now we read what server has replied and then we close the connection

Serial.println("Reply from InfluxDB");

while(client.available()) { //receive char

Serial.print((char)client.read());

}

Serial.println(); //empty line

client.stop();

}

/**

* @brief Setups the peripherals

void setup() {

//Serial interface for debugging purposes

Serial.begin(115200);

delay(1000);

eth_start();

}

/**

* @brief Main loop

void loop() {

//A loop to send some data

for(int i = 30; i < 50 ; i++) {

//this will be number of chars written to buffer, return value of sprintf

int numChars = 0;

//First of all we need to add the name of measurement to beginning of the buffer

numChars = sprintf(buf, "my_house_data,");

//tag should have an space at the end

numChars += sprintf(&buf[numChars], "SOURCE=arduino_1,PLACE=%s ", get_place());

//after tags, comes the values!

numChars += sprintf(&buf[numChars], "TEMP=%d,", i);

numChars += sprintf(&buf[numChars], "HUMIDITY=%.2f,", i * 1.03);

numChars += sprintf(&buf[numChars], "LIGHT=%.2f", i * 10.5);

//Print the buffer on the serial line to see how it looks

Serial.print("Sending following dataset to InfluxDB: ");

Serial.println(buf);

//send to InfluxDB

eth_send_data(buf, numChars);

//we have to reset the buffer at the end of loop

memset(buf, '\0', bufferSize);

delay(1000); //some small delay!

}

Notes regarding the code

Although I have commented the code thoroughly, here is a brief description of the code:

Lines 4 to 16 are variables required for setting up and connecting to the server using the Ethernet shield. In my case, the Arduino was connected to my own laptop in a loop-back configuration, so the server address for InfluxDB would be obviously 192.168.0.1. If you are planning to connect your Arduino to a router or you are using WiFi, you must make sure to put the correct address in the eth_server[] variable!
In line 20 we have defined an integer equal to 2048, which is the size of our buffer to write the insertion query in it.
Line 23 is the definition and initialization of the buffer.
the get_place function is just used to get an imaginary place out of 4 places
the eth_send_data function is used to generate and send the correct HTTP POST request to the InfluxDB server
the loop function is used to continuously send some values in the range of 30 to 50 as temperature and a multiple of it as humidity and light values

So all and all, what you need to pay attention is the correct usage of writing a message into a character array buffer using sprintf function and using its return values to know where to continue inserting into the buffer.

In the next step, we will see how does the data is being transmitted and how to see the raw data in the InfluxDB’s administration panel.

Saeid Yazdani

An Electronics Engineer with passion in Embedded Systems, Wireless Sensor Networks, Assembly, Desktop and Mobile Development, Web Development and generally anything that has to do with hardware and software of any kind!

View all posts by Saeid Yazdani →

27 Comments:

Jean-Francois Payeur
26-01-2017 at 00:12

Thanks for sharing !!!

Code “as-is” is giving me very weird errors en compiling for arduino 2560
Still investigating on this

C:\Program Files (x86)\Arduino\hardware\arduino\avr\cores\arduino\main.cpp: In function 'main': C:\Program Files (x86)\Arduino\hardware\arduino\avr\cores\arduino\main.cpp:51:1: error: unable to find a register to spill in class 'NO_REGS' } ^ C:\Program Files (x86)\Arduino\hardware\arduino\avr\cores\arduino\main.cpp:51:1: error: this is the insn: (insn 294 291 297 19 (set (mem:QI (post_dec:HI (reg/f:HI 32 __SP_L__)) [0 S1 A8]) (subreg:QI (reg/f:HI 271) 1)) \\WIN7PRO\__Projets\Grid Tie solar et vent\arduino\to influxdb\v033 test influx\v033\v033.ino:144 1 {pushqi1} (expr_list:REG_ARGS_SIZE (const_int 5 [0x5]) (nil))) C:\Program Files (x86)\Arduino\hardware\arduino\avr\cores\arduino\main.cpp:51: confused by earlier errors, bailing out lto-wrapper: C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avr-gcc returned 1 exit status c:/program files (x86)/arduino/hardware/tools/avr/bin/../lib/gcc/avr/4.9.2/../../../../avr/bin/ld.exe: lto-wrapper failed
collect2.exe: error: ld returned 1 exit status
exit status 1 Error compiling for board Arduino/Genuino Mega or Mega 2560.

but werirdly compile fine for arduino due !
( from all I know, back end compiler is not the same at all )

Using arduino 1.8.1 on windows 10

Reply
- Saeid Yazdani
  26-01-2017 at 00:23
  
  I have only tested this with Due. I’m not sure what the problem could be as I don’t have any other arduinos to test again. I just guess it has something tido with your Ethernet library.
  
  Reply
Jean-Francois Payeur
26-01-2017 at 02:13

Thank, I tought you were using a 2560 .
BTW You dont need the board to test compilation

I got the stock etherne lib
Playing around commenting uncomenting lines, still got weird compil errors

I’ll use a Due if its the only issue , just a bit more expensive

Reply
Jean-Francois Payeur
26-01-2017 at 02:29

Found a work around

Seems a documented bug in the latest GCC Compiler
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60040

Aplied a similar work around I got from here
https://github.com/arduino/Arduino/issues/3972

I’ve Added
void eth_send_data(char* data, int dataSize) __attribute__((__optimize__(“O2”)));

On line 2 of your code
Now compile Fine for 2560 , dont ask me more details !!!!!

Thanks !

Reply
- Saeid Yazdani
  26-01-2017 at 08:04
  
  Thanks for sharing!
  
  Reply
Jean-Francois Payeur
28-01-2017 at 04:14

Eratum
You need 2 “spaces” before “__attribute__”
Dont ask me why , but it is not the same

void eth_send_data(char* data, int dataSize) __attribute__((__optimize__(“O2”)));

Reply
Jean-Francois Payeur
28-01-2017 at 04:17

Sorry … itsjust because I ve taken back My own comment and it wasent compiling again

The issue is not the double space at all

it is because this web page convert the ” character to another type of ”
Just have to type it back to the “real” ” one …. hard to explain

Reply
Jean-Francois Payeur
29-01-2017 at 01:38

Just gave up, moved it all to arduino due, still having compilation error for 2560, but all working fine on due. Rest of instructions are all good.

Except minor error, on page 5:
$sudo dpkg -i https://grafanarel.s3.amazonaws.com/builds/grafana_4.0.2-1481203731_amd64.deb
that should read
$sudo dpkg -i grafana_4.0.2-1481203731_amd64.deb

Reply
- Saeid Yazdani
  29-01-2017 at 10:46
  
  Thanks, I edited the problem on the last page. I do not know why the code is not woking on Mega since the code is pretty much standard C.
  
  Reply
Orange Juice
12-02-2017 at 12:59

Hi. Code is working fine, I also needed to Optimize due to using a Mega. I am actually using a ENC28J60 with the UIPEthernet which is drop in for Ethernet.h however I dont get any respone from the server shown in Serial Monitor just the “Reply from InfluxDB”, The information is reaching influx it just doesnt display the 204 error it should in serial monitor.

Dont suppose you have any pointers, not that its essential as the code is working.

Reply
- Saeid Yazdani
  12-02-2017 at 13:12
  
  I think you should play around with the delay value between sending and waiting for the response. Give it a try.
  
  Reply
- Saeid Yazdani
  12-02-2017 at 13:15
  
  And perhaps you can show the changes you made to make it work on Mega? maybe that’s the problem. Jean-Francois Payeur comments show that it doesn’t work on Mega, although I have never tried it with Mega so can’t say for sure.
  
  Reply
Orange Juice
12-02-2017 at 15:26

Thanks for responding, great tutorial by the way.

I followed Jean’s instructions as I got the same compile error an it was the GCC .

Only difference in code compared with yours is i added this at the top of the skect.
#include

void eth_send_data(char* data, int dataSize) __attribute__((__optimize__(“O2”)));

I will check delay and have a look into if it is a issue with UIPEthernet and syntax

Reply
Orange Juice
12-02-2017 at 16:28

I managed to get it working by replacing availible with connected.

Serial.println(“Reply from InfluxDB”);
while(client.connected()) { //receive char
Serial.print((char)client.read());
}

Thanks for a great tutorial.

Reply
- Saeid Yazdani
  12-02-2017 at 20:43
  
  Thanks for sharing your discoveries 😉 I think I have to reflect that in the article.
  
  Reply
Pingback: InfluxDB Week in Review – Feb 13, 2017 | InfluxData
Orange Juice
15-02-2017 at 21:34

It could be because I’m using the UIPEthernet library.

I regret to inform the test I had running which was taking humidity readings from 3x DHT11 stopped sending information to influx after 60hours.
Unsure on why it failed as it wasnt plugged into the computer so didnt have Serial Monitor open 🙁 I have reset the Arduino and hope for a better result, maybe it was my cat.

Reply
Pingback: Comunicação Arduino+InfluxDB+Grafana Parte 1 | Daniel Ortega
AM
17-06-2017 at 05:45

Excellent tutorial! But im having a problem that a copple minutes after starts sending data, arduino cant send any more messages, it just cant connect. If i restart arduino, it works again. Any ideas?

Reply
- Saeid Yazdani
  17-06-2017 at 10:30
  
  This might be due to bugs in original ethernet library of Arduino. Try to use a third party library and see if it helps.
  
  Reply
ain
31-05-2018 at 10:09

how can i post each field with multiple field value.
batch data sent to influxdb.

Reply
- Admin
  04-06-2018 at 13:56
  
  It is possible according to influxdb documentation, I do not know myself but I have seen it in their documents. Search for something like “sending multiple data in single query” or something similar on their documentation page.
  
  Reply
Santiago
04-06-2018 at 11:48

Hi!

I found your code very useful, but the thing I do not see is how the data is written into your created database. I mean how do you tell the Arduino to send the data to the database named “embedonix”? In my case the data is being sent (at least the serial monitor says so) but if i check for the data, the databse appears blank.

Thank you very much

Reply
- Admin
  04-06-2018 at 13:55
  
  Please check to log file of influxdb (or run influxd in standalone mode to see std output) and see if you receive any http request or not.
  
  Reply
Rahulan
13-08-2018 at 14:23

I planned to do this for my house but instead of Ethernet shield i’m gonna use esp8266 wifi module.So can i do some changes to your code and use it for my project?

Reply
- Admin
  17-08-2018 at 11:45
  
  Yes the communication method does not matter as long as it works 🙂
  
  Reply
Mayank
12-06-2019 at 08:52

I cant seem to establish a connection with my InfluxDB server. Influx is running on a virtual machine (Ubuntu) and I am trying to send data from Arduino installed on Windows using a NodeMCU

Reply

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Setting up InfluxDB and Grafana for storing and viewing Arduino measured sensor values

Sending data to InfluxDB using Arduino over HTTP

Contents

The code

Notes regarding the code

Saeid Yazdani

27 Comments:

Leave a Reply Cancel reply

Latest News

Categories

Donate!

EMBEDONIX

Meta

TOP Meme