MQTT pubish is slow to send with ESP8266 & NodeMCU

0 votes

So, I tested the master branch and dev branch in the firmware produced with NodeMCU custom builds (http://nodemcu-build.com/) upon the ESP8266 MCU.
And, I made a few tweaks in the code for connecting to the MQTT broker which I found here: https://github.com/nodemcu/nodemcu-firmware
These are the changes I made:
- removing the subscriber
- adding a loop of 100 messages around --> m:publish("/topic","hello",0,0, function(conn) print("sent") end)

And, this is how it looks like now:
-- init mqtt client with keepalive timer 120sec
m = mqtt.Client("clientid", 120, "user", "password")

m:on("connect", function(con) print ("connected") end)
m:on("offline", function(con) print ("offline") end)

-- m:connect( host, port, secure, auto_reconnect, function(client) )
-- for secure: m:connect("192.168.11.118", 1880, 1, 0)
-- for auto-reconnect: m:connect("192.168.11.118", 1880, 0, 1)
m:connect("192.168.11.118", 1880, 0, 0, function(conn) print("connected") end)

-- publish a message with data = hello, QoS = 0, retain = 0
local i = 1
while i < 10 do
  m:publish("/topic","hello",0,0, function(conn) print("sent") end)
  i = i + 1
end

m:close();  

My message broker is the Eclipse Mosquitto and I have made sure to launch a subscriber on all topic #.
Although the messages are correct, but their arrival is very slow on the subscriber, like a delay of 1 second for each message. 

Now, I've been trying to figure out why, but have come up with no luck yet.

So, I did a few experiments and came to a conclusion that the messages get stored in the form of a queue in the memory, which is then read by a timer that sends it through MQTT. Now, sending too many messages just increases this queue but fails in delivering the messages together. Moreover, just after a few messages get queued, the device reboots as if it crashed due to lack of memory. 
 

Does anybody have a clue so as to what exactly is going on? My theory is just based on assumptions as of now.   

Aug 1, 2018 in IoT (Internet of Things) by Bharani
• 4,550 points
636 views

1 answer to this question.

0 votes

I think you're almost correct and on the right track. The MQTT broker for NodeMCU does engage an internal queue for delivering messages. That is what its architecture was changed into to support the asynchronous API that NodeMCU uses. So, due to this asynchronous nature, cases with successive calls to m.publish results in a time lag for the delivery of the 1st message before the next message could get triggered. And if these successive calls were published in a loop, the firmware would just have crashed even before queuing could start.
Now, I'm not saying such an issue is easy to solve but it's explanation isn't too tough to understand. I've gone ahead and done some minor changes to your code, have a look:
Code:
m = mqtt.Client("clientid", 120, "user", "password")

m:connect("m20.cloudmqtt.com", port, 0, function(conn) 
    print("MQTT connected")
    for i=1,10 do
      print("MQTT publishing...")
      m:publish("/topic", "hello", 0, 0, function(conn) 
        print("MQTT message sent")
        print("  heap is " .. node.heap() .. " bytes")
      end)
      print("  heap is " .. node.heap() .. " bytes in loop " .. i)
    end
end)

And, because we know that calls to m.publish in case of NodeMCU are asynchronous in nature, you can check the outputs to understand that the heap space available when publishing messages is very less in contrast to the high availability of heap space when the queue becomes empty.
Output:
MQTT connected
MQTT publishing...
  heap is 37784 bytes in loop 1
MQTT publishing...
  heap is 37640 bytes in loop 2
MQTT publishing...
  heap is 37520 bytes in loop 3
MQTT publishing...
  heap is 37448 bytes in loop 4
MQTT publishing...
  heap is 37344 bytes in loop 5
MQTT publishing...
  heap is 37264 bytes in loop 6
MQTT publishing...
  heap is 37192 bytes in loop 7
MQTT publishing...
  heap is 37120 bytes in loop 8
MQTT publishing...
  heap is 37048 bytes in loop 9
MQTT publishing...
  heap is 36976 bytes in loop 10
sent
  heap is 38704 bytes
sent
  heap is 38792 bytes
sent
  heap is 38856 bytes
sent
  heap is 38928 bytes
sent
  heap is 39032 bytes
sent
  heap is 39112 bytes
sent
  heap is 39184 bytes
sent
  heap is 39256 bytes
sent
  heap is 39328 bytes
sent
  heap is 39400 bytes

answered Aug 1, 2018 by nirvana
• 3,060 points

Related Questions In IoT (Internet of Things)

0 votes
1 answer

What is the best way to retrofit existing alarm PIR with esp8266/etc?

Here is an explanation that best fits ...READ MORE

answered Sep 4, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
52 views
0 votes
1 answer

Send LoRa measures to Fiware IOT agent via MQTT

down vote I dont know if I understood ...READ MORE

answered Jul 13, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
272 views
0 votes
1 answer

What is the Best practice to create automatic reports with Cumulocity?

There is a new feature regarding reporting ...READ MORE

answered Nov 23, 2018 in IoT (Internet of Things) by Shubham
• 13,290 points
52 views
0 votes
1 answer
0 votes
1 answer

MQTT on ESP8266 with NodeMCU - problems with publishing

Perhaps this was solved by more recent ...READ MORE

answered Aug 13, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
236 views
0 votes
1 answer

How to send a data from arduino uno to a webpage through esp8266 wifi module?

You are missing a few \r\n and the length ...READ MORE

answered Aug 9, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
1,252 views
0 votes
1 answer
0 votes
1 answer

Implementing MQTT for a building

I'm not really sure if its possible ...READ MORE

answered Aug 29, 2018 in IoT (Internet of Things) by anonymous
24 views
0 votes
1 answer

Is it possible to use .Net for the ESP8266?

You could write a C# marshaller, but ...READ MORE

answered Nov 19, 2018 in IoT (Internet of Things) by nirvana
• 3,060 points
139 views
0 votes
1 answer