Api reference¶
MQTT/UDP is implemented in five languages, but implementations differ. Most
complete and fast developing are Java and Python versions. Others
follow a bit later. Please see map of languages and features on a project Wiki.
C Language API Reference¶
There is a native MQTT/UDP implementation in C. You can browse sources at https://github.com/dzavalishin/mqtt_udp/tree/master/lang/c repository.
Lets begin with a simplest examples.
Send data:
Listen for data:
Now lets look at the packet structure definition:
- from_ip
- Ip address of message sender. Usually ignored.
- ptype
- Packet type. You will be interested in
`PTYPE_PUBLISH`
most of time.
See`mqtt_udp_defs.h`
for more. - pflags
- Flags specific for each type. Ignore. Current version of MQTT/UDP
does not use them at all, and in any case everything critical will be
processed by library. - total
- This field is internal for library.
- pkt_id
- Packet id. Leave zero for outgoing packets, and sequential number
will be provided. In incoming packets it will be filled if sender
supports TTRs (extended packet structure). - topic and topic_len
- Message topic, NULL terminated. Length of topic in bytes.
- value and value_len
- Message value, also NULL terminated. Length of value in bytes.
- is_signed
- This packet has correct digital signature.
Functions¶
Send PUBLISH packet:
Send SUBSCRIBE packet:
Send PINGREQ packet, ask others to respond:
Send PINGRESP packet, tell that you’re alive:
Start loop for packet reception, providing callback to be called
when packet arrives:
Dump packet structure. Handy to debug things:
Set minimal time between outgoing packets (msec), control maximum send speed:
Set callback to handle internal errors such as net IO error:
User error handler can:
- Return zero: caller must attempt to ignore error, if possible.
- Return err_no: caller must return with error in turn, if possible.
- Exit (or restart application completely) if error is supposed to be fatal.
Handler can also be used for logging.
Remote configuration¶
This part of API lets user to configure program/device by network. There is a detailed
description in Passive remote configuration and in the Python part of this book,
see Module mqttudp.rconfig. Here is description of C implementation.
Set up remote config subsystem:
#include "runtime_cfg.h"mqtt_udp_rconfig_item_trconfig_list[]={{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_TOPIC,"Switch 1 topic","topic/sw1",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_TOPIC,"Switch 2 topic","topic/sw2",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_TOPIC,"Di 0 topic","topic/di0",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_TOPIC,"Di 1 topic","topic/di1",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_OTHER,"MAC address","net/mac",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_INFO,"Switch 4 topic","info/soft",{.s=DEVICE_NAME}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_INFO,"Switch 4 topic","info/ver",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_INFO,"Switch 4 topic","info/uptime",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_OTHER,"Name","node/name",{.s=0}},{MQ_CFG_TYPE_STRING,MQ_CFG_KIND_OTHER,"Location","node/location",{.s=0}},};intrconfig_list_size=sizeof(rconfig_list)/sizeof(mqtt_udp_rconfig_item_t);intrc=mqtt_udp_rconfig_client_init(mac_string,rconfig_rw_callback,rconfig_list,rconfig_list_size);if(rc)printf("rconfig init failed, %dn",rc);
Each array item is one parameter to be set up remotely. The only type supported now is MQ_CFG_TYPE_STRING
. Kinds:
- MQ_CFG_KIND_INFO
- Read-only information about this instance (program or device)
- MQ_CFG_KIND_TOPIC
- Is a configurable topic name, used to publish or receive information.
- MQ_CFG_KIND_OTHER
- Any other parameter type. (R/W and not topic)
Third item field is human-readable item description, currently it is not used, but will be translated to
configuration tool. Fourth item is identification of configurable item, both for local and remote side.
For remote side it is sent as part of configuration message topic and is shown to user as configuration
iem description. Last field is current parameter value. For read-only parameters you can just put any string
pointer here. For R/W string must be malloc’ed (or set with mqtt_udp_rconfig_set_string()
).
To be precise:
- type
- Data type for
.value
, must be MQ_CFG_TYPE_STRING as for now. - kind
- Kind of item, see above. If kind is MQ_CFG_KIND_TOPIC,
.topic
field must begin with “topic/”. - name
- Human-readable description, unused now.
- value
- Current value. You will be using
.value.s
union field. - opaque
- Not used or interpreted, use as pointer to external storage for this item,
internal item index or function pointer to read/set item as you wish.
Now lets look at available functions.
Init subsystem:
- mac_string
- Id string (12 bytes) used as unique id of this configurable instance. MAC address of device is
a good candidate. - rconfig_rw_callback
- Callback called by subsystem to ask you provide current value for item or get new setting
after instance item was remotely set up. Prototype isintrconfig_rw_callback(intpos,intwrite)
,
wherepos
is item position (index) in array andwrite
is nonzero if callback shall
get new setting from instance array and save it somewhare for next run. If zero, callback
must read saved instance value and callmqtt_udp_rconfig_set_string()
for it.
Set item value:
- pos
- Item position (index) in array
- string
- New value
Get item value checking kind:
- pos
- Item position (index) in array
- kind
- Expected kind for item. If not, global error callback is called and
NULL
is returned.
This function is supposed to be used to get configurable topic for outgoing message
so usually this parameter isMQ_CFG_KIND_TOPIC
.
Find item by .value
string:
- search
- String value to be found.
- kind
- Only lines of this kind will match. This function is supposed to look up
incoming items topics to find if some of configurable topics match. So
this parameter usually isMQ_CFG_KIND_TOPIC
.
Please study this API use example in sample remote config C application.
UDP IO interface¶
Default implementation uses POSIX API to communicate with network, but for
embedded use you can redefine corresponding functions. Here are things to
reimplement.
Receive UDP packet. Returning value is number of bytes in packet received or
negative error code. Must return sender’s address in src_ip_addr
:
Broadcast UDP packet:
Send UDP packet (actually not used now, but can be later):
Create UDP socket which can be used to send or broadcast:
Prepare socket for reception on MQTT_PORT:
Close UDP socket:
Java Language API Reference¶
There is a native MQTT/UDP implementation in Java. You can browse sources at https://github.com/dzavalishin/mqtt_udp/tree/master/lang/java repository.
Again, here are simplest examples.
Send data:
Listen for data:
Listen for packets¶
See Example Java code.
Here it is:
Now what we are doung here. Our class Sub
is based on SubServer
, which is doing all the reception job, and calls processPacket
when it got some data for you. There are many possible types of packets, but for now we need just one, which is
PublishPacket
. Hence we check for type, and convert:
Now we can do what we wish with data we got using pp.getTopic()
and pp.getValueString()
.
Listen code we’ve seen in a first example is slightly different:
Used here PacketSourceServer
, first of all, starts automatically, and uses Sink
you pass to setSink
to pass packets received to you. The rest of the story is the same.
There is another, more complex listen server class, `PacketSourceMultiServer`
.
Instance of it can provide incoming packets to more than one listener:
Python Language API Reference¶
As you already guessed, python implementation is native too. You can browse
sources at https://github.com/dzavalishin/mqtt_udp/tree/master/lang/python3
repository.
There is also lang/python directory, which is for older 2.x python environment,
but it is outdated. Sorry, can’t afford to support it. If you need python 2.x,
you can backport some python3 code, it should be quite easy.
Let’s begin with examples, as usual.
Send data:
Listen for data:
Module mqttudp.engine¶
Main package, implements MQTT/UDP protocol.
Paccket class:
Functions:
Match topic name against a pattern, processing and # wildcards, returns True on match:
Turn of automatic protocol replies:
Set minimum time between packets sent, msec:
Set network address to listen at (choose incoming packets network interface). Address must
be equal to address of some network interface:
Set network address to broadcast to (choose outgoing packets network interface). Address must
be broadcast address for some of existing network interfaces. Ask local network administrator
if unsure:
Module mqttudp.rconfig¶
Additional module implemening passive remote configuration client
(party that is being configured) implementation.
There is a complete demonstration example exist.
To see example working please run mqtt_udp_rconfig.py
first
and mqtt_udp_view after it. In viewer please press middle toolbar button
to open remote configuration window. This window will show all
running MQTT/UDP instances that can be configured. There must be
Pythontestnode
among them. Select its tab. You will see all
the configurable items (from init_items
dictionary) as text
fields. Meanwhile mqtt_udp_rconfig.py
will be sending a random
number with “test” topic. Enter new topic name in a field near
“topic: test” description and press nearest button to send new
setting to program. Notice that now it sends random data with a topic
you just set up.
Now lets look at example code (see examples/mqtt_udp_rconfig.py):
importmqttudp.rconfigasrcfginit_items={## read only"info/soft":"Pyton example","info/ver":"0.0","info/uptime":"?",## common instance info"node/name":"Unnamed","node/location":"Nowhere",# items we want to send out"topic/test":"test","topic/ai0":"unnamed_ai0","topic/di0":"unnamed_di0","topic/pwm0":"unnamed_pwm0",}defsend_thread():whileTrue:n=str(random.randint(0,9))print("Send " n)rcfg.publish_for("test",n)time.sleep(2)if__name__=="__main__":print("Will demonstrate remote config")rcfg.init(init_items)st=threading.Thread(target=send_thread,args=())st.start()mq.listen(rcfg.recv_packet)
This example shows how to use remote configuration subsystem. Dictionary
init_items
contains list of items which can possiblly be configured
remotely. Different elements are used in a different ways.
In general each item in a list is a configurable thing. For example,
"node/location":"Nowhere"
is item which name is node/location
and initial value is Nowhere
. (It is supposed as a memo for user
to know where an appliance or computer running this code is installed.)
Another example is "topic/ai0":"unnamed_ai0"
– it is supposed
to be a configurable topic name that device uses to send data from
some analogue input. User must configure topic name and it will be
used by node to send data.
Generally item keys consist of two parts
separated with slash: "topic/pwm0"
, "info/uptime"
or
"node/name"
. Left part is named kind and defines the way item
is processed. Here is a list of known kinds.
info
- Read only description of node/program.
node
- General node information or settings.
topic
- This kind is a configurable value that is a topic name.
net
- Reserved for network settings.
Please see more on kinds in Passive remote configuration.
Lets go on with code. Line rcfg.init(init_items)
sets subsystem
up. Remote config subsystem first loads previous settings from .ini file
(you can set file name with set_ini_file_name(fn)
function) and
fills all items from given init_items
dictionary that was not read
from .ini file. Items with info
kind are taken from init_items
in any way.
After init your program continues working, but must call recv_packet
function of mqttudp.rconfig
for each incoming MQTT/UDP packet for
remote configuration to work.
There are three ways to use configured parameters.
- Just read parameter
- You can just call
get_setting(name)
for needed item to get current
configured value. For example,get_setting("net/mac")
or, say,
lcd_print(get_setting("node/location"))
. If your node will be
reconfigured in run time, on next call there will be new value. - Send data for configurable topic
- By calling
publish_for(item_of_topic,data)
you will send data
to a topic which is configured by item with, guess what,topic
kind.
See abovercfg.publish_for("test",n)
– this line looks up config
item namedtopic/test
, and uses its value as a topic to publisn
value of variblen
to. - Check incoming packet topic
- On receiving incoming PUBLISH packet, you can use
is_for(topic_item,topic)
function, which checkstopic
parameter to be equal to value of config
item named"topic/" topic_item
, such asis_for("pwm0",topic)
will
returnTrue
iftopic
variable contains string equal to value of config
item"topic/pwm0"
.
Only thing left to mention is that you can set callback with call to set_on_config(callback)
and it will be called if remote configuration happens. Config item name and
new value will be passed as parameters.
Lua Language API Reference¶
You can browse sources at https://github.com/dzavalishin/mqtt_udp/tree/master/lang/lua repository.
Basic examples in Lua.
Send data:
Listen for data:
CodeSys ST Language API Reference¶
Sorry, due to PLC limitations, there is no clear API in this code example, just integrated protocol
and client code example.
PLC is specific: it runs all its programs in loop and it is assumed that each program is running
without blocking and does not spend too much time each loop cycle. There’s usually a watch dog
that checks for it. Hence, ST implementation is cycling, sending just one topic per loop cycle.
Actual API is simple:
Here is how it is used in main program:
PROGRAM MQTT_PRG
VAR
STEP : INT := 0;
socket : DINT := SOCKET_INVALID;
wOutPort : INT := 1883;
addr : SOCKADDRESS;
END_VAR
CASE STEP OF
0:
socket := SysSockCreate( SOCKET_AF_INET, SOCKET_DGRAM, SOCKET_IPPROTO_UDP );
addr.sin_family := SOCKET_AF_INET;
addr.sin_port := SysSockHtons( wOutPort );
addr.sin_addr := 16#FFFFFFFF; (* broadcast *)
1: MQ_SEND_REAL( socket, addr, 'PLK0_WarmWater', WarmWaterConsumption );
2: MQ_SEND_REAL( socket, addr, 'PLK0_ColdWater', ColdWaterConsumption );
3: MQ_SEND_REAL( socket, addr, 'PLK0_activePa', activePa_avg );
4: MQ_SEND_REAL( socket, addr, 'PLK0_Va', Va );
ELSE
IF socket <> SOCKET_INVALID THEN
SysSockClose( socket );
END_IF
socket := SOCKET_INVALID;
END_CASE
STEP := STEP 1;
IF socket = SOCKET_INVALID THEN
STEP := 0;
END_IF
END_PROGRAM
Codesys st language api reference¶
Sorry, due to PLC limitations, there is no clear API in this code example, just integrated protocol
and client code example.
Differences from mqtt¶
MQTT/UDP is based on classic MQTT, but differs a bit. First of all, just a subset of
packet types used and, of course, as there is no broker there is no need for CONNECT,
CONNACK or DISCONNECT.
Additionally, MQTT/UDP does not send or expect variable header (packet ID field)
present in some MQTT packets.
Current implementation also ignores packet flags completely, but it will change later.
Most implementations support Tagged Tail Records addition to the protocol, which
extends and replaces variable header in an extensible and flexible way.
Tagged tail records can be used to add any kinds of additional information to
the classic MQTT packets, but the most noticeable use of TTRs in MQTT/UDP is
digital signature.
Please read detailed description at project Wiki.
Functions¶
Send PUBLISH packet:
Send SUBSCRIBE packet:
Send PINGREQ packet, ask others to respond:
Send PINGRESP packet, tell that you’re alive:
Start loop for packet reception, providing callback to be called
when packet arrives:
Dump packet structure. Handy to debug things:
Set minimal time between outgoing packets (msec), control maximum send speed:
Set callback to handle internal errors such as net IO error:
How do i start tcpview?
Starting the TCPView program on your computer system will automatically enumerate all active TCP and UDP endpoints. This immediately resolves all IP addresses to their domain name versions. Hover onto the menu tab or click the toolbar button to toggle the display of these fixed names.
Highlighted in yellow are the endpoints that have changed from one upgrade to the next. Shown in red are the deleted ones while flashed in green are considered to be the new endpoints.
Selecting the File|Close Connections or pressing the right-click button will lead you to close the established TCP/IP connections. The TCPView’s output window can be stored in a certain folder or file location through the ‘Save’ menu option. Using its Tcpvcon feature is just similar to operating the built-in Windows Netstat utility.
Listen for packets¶
See Example Java code.
Here it is:
Now what we are doung here. Our class Sub is based on SubServer, which is doing all the reception job, and calls processPacket
when it got some data for you.
Now we can do what we wish with data we got using pp.getTopic() and pp.getValueString().
Listen code we’ve seen in a first example is slightly different:
Used here PacketSourceServer, first of all, starts automatically, and uses Sink you pass to setSink
to pass packets received to you. The rest of the story is the same.
There is another, more complex listen server class, `PacketSourceMultiServer`.
Instance of it can provide incoming packets to more than one listener:
Module mqttudp.engine¶
Main package, implements MQTT/UDP protocol.
Paccket class:
Functions:
Match topic name against a pattern, processing and # wildcards, returns True on match:
Turn of automatic protocol replies:
Set minimum time between packets sent, msec:
Set network address to listen at (choose incoming packets network interface). Address must
be equal to address of some network interface:
Module mqttudp.rconfig¶
Additional module implemening passive remote configuration client
(party that is being configured) implementation.
There is a complete demonstration example exist.
To see example working please run mqtt_udp_rconfig.py first
and mqtt_udp_view after it. In viewer please press middle toolbar button
to open remote configuration window. This window will show all
running MQTT/UDP instances that can be configured.
There must be
Pythontestnode among them. Select its tab. You will see all
the configurable items (from init_items dictionary) as text
fields.
Meanwhile mqtt_udp_rconfig.py will be sending a random
number with “test” topic. Enter new topic name in a field near
“topic: test” description and press nearest button to send new
setting to program. Notice that now it sends random data with a topic
you just set up.
Now lets look at example code (see examples/mqtt_udp_rconfig.py):
importmqttudp.rconfigasrcfginit_items={## read only"info/soft":"Pyton example","info/ver":"0.0","info/uptime":"?",## common instance info"node/name":"Unnamed","node/location":"Nowhere",# items we want to send out"topic/test":"test","topic/ai0":"unnamed_ai0","topic/di0":"unnamed_di0","topic/pwm0":"unnamed_pwm0",}defsend_thread():whileTrue:n=str(random.randint(0,9))print("Send " n)rcfg.publish_for("test",n)time.sleep(2)if__name__=="__main__":print("Will demonstrate remote config")rcfg.init(init_items)st=threading.Thread(target=send_thread,args=())st.start()mq.listen(rcfg.recv_packet)
This example shows how to use remote configuration subsystem. Dictionary
init_items contains list of items which can possiblly be configured
remotely. Different elements are used in a different ways.
Multiple smart switches¶
Some wall switches are controlling the same device. All of them send
and read one topic which translates on/off state for the device.
Of course, if one switch changes the state, all others read the state broadcast
and note it, so that next time each switch knows, which state it should
switch to.
It is possible, of course, that UDP packet from some switch will be lost.
So when you switch it, nothing happens. What do you do in such a situation?
Turn switch again, of course, until it works!
In this example I wanted to illustrate that even in this situation UDP
transport is not really that bad.
Organize ucp and tcp lists
TCPView is a handy and clean networking suite to view and manage all your UCP and TCP endpoints. It will present the application’s name along with its opened connections. There is a ‘resolve addresses’ selection that empowers you to learn a lot more about the particular internet server beyond its IP address. This program should easily replace the Netstat program that comes with Windows.
Packets and general logic¶
Differences from MQTT¶
MQTT/UDP is based on classic MQTT, but differs a bit. First of all, just a subset of
packet types used and, of course, as there is no broker there is no need for CONNECT,
CONNACK or DISCONNECT.
Additionally, MQTT/UDP does not send or expect variable header (packet ID field)
present in some MQTT packets.
Current implementation also ignores packet flags completely, but it will change later.
Most implementations support Tagged Tail Records addition to the protocol, which
extends and replaces variable header in an extensible and flexible way.
Tagged tail records can be used to add any kinds of additional information to
the classic MQTT packets, but the most noticeable use of TTRs in MQTT/UDP is
digital signature.
Please read detailed description at project Wiki.
Possible topologies¶
Here is a list of more or less obvious scenarios for MQTT/UDP
Multiple smart switches¶
Some wall switches are controlling the same device. All of them send
and read one topic which translates on/off state for the device.
Of course, if one switch changes the state, all others read the state broadcast
and note it, so that next time each switch knows, which state it should
switch to.
It is possible, of course, that UDP packet from some switch will be lost.
So when you switch it, nothing happens. What do you do in such a situation?
Turn switch again, of course, until it works!
In this example I wanted to illustrate that even in this situation UDP
transport is not really that bad.
Predefined items¶
Protocol implementations expect some item kind/name pairs to mean special thins.
Here is a list of know ones.
info/soft
- Readonly, name of software running in this node.
info/ver
- Readonly, version of software.
info/uptime
- Text string, uptime: “255d 02:12:34” or “5y 2m 13d 23:55:01”
node/name
- Name of this MQTT/UDP node. Human readable, not interpreted. “Weather sensors”
or “A/C control” node/location
- Where you can find it: “Kitchen”, “building 12, 2nd floor, room 212”, whatever.
net/mac
- Network MAC address, if not hardwired.
net/ip
- Network IP address, if static.
net/mask
- Netmask.
net/router
- Network default route.
Other network settings can be put to net kind.
Qos server¶
Nowadays UDP is quite reliable (typical loss rate is < 0.5% even in a busy network),
but not 100% reliable. Things are easy in point to point communications. Send a message,
wait for acknowledgement, resend if none received in a reasonable time.
It is makes sense that we can build a map of nodes that listen to us by collecting
their responces. But we want to keep MQTT/UDP implementation simple and this is not
that simple. And not any node needs such high a reliability.
The idea is to add separate server on a network that will build lists of listeners
for each topic, collect low-QoS ack packets and sent one high-QoS ack packet
to topic publisher(s).
Note that such server is not a single point of failure. First of all, there can be more
than one instance of QoS server. Second, even if QoS server fails, nodes continue to
send data. Though, each packet will be resent few times, but it is not a communications
failure.
Reliability¶
As MQTT/UDP is based on UDP protocol, which does not guarantee packet delivery, one can suppose that MQTT/UDP is not reliable. Is it?
Not at all.
If we use it for repeated updates, such as sensor data transfer, UDP is actually more reliable, than TCP! Really. If our network drops each
second packet, TCP connection will be effectively dead, attempting to resend again and again outdated packets which are not needed anymore.
Actualy, simple test was made [1] to ckeck UDP reliability. One host in my house’s local net was generating MQTT/UDP traffic as fast as
possible and other checked packets to be sequent, counting speed and error rate. Two IPTV units was started to show HD content and one
of the computers was copying some few GBytes to file server.
Anyway, I’m going to add completely reliable mode to MQTT/UDP in near future.
Footnotes
Udp io interface¶
Default implementation uses POSIX API to communicate with network, but for
embedded use you can redefine corresponding functions. Here are things to
reimplement.
Receive UDP packet. Returning value is number of bytes in packet received or
negative error code. Must return sender’s address in src_ip_addr:
Broadcast UDP packet:
Send UDP packet (actually not used now, but can be later):
Create UDP socket which can be used to send or broadcast:
Prepare socket for reception on MQTT_PORT:
Close UDP socket:
Welcome to mqtt/udp¶
Package version 0.5
You can get this document in PDF format.
Introduction¶
MQTT/UDP is a simplest possible protocol for IoT, smart home applications and
robotics. As you can guess from its name, it is based on MQTT (which is quite
simple too), but based on UDP and needs no broker.
Fast track for impatient readers: MQTT/UDP native implementations exist in Java,
Python, C, Lua and PLC specific ST language. See corresponding references:
If you want to test MQTT/UDP on a real hardware, take a look at Sketches
part. Ready made software is described in Integration and tools part.
Now some words on MQTT/UDP idea. It is quite simple. Broker is a single point
of failure and can be avoided. Actual
traffic of smart home installation is not too big and comes over a separated (by firewall)
network. There are many listeners that need same data, such as:
- main UI subsystem (such as OpenHAB installation)
- special function controllers (light, climate units)
- per-room or per-function controllers (kitchen ventilation, bath room sensors, room CO2 sensors, etc)
- in-room displays (room and outdoor temperature)
All these points generate some information (local sensors, state) and need
some other information.
So, MQTT/UDP is sending data with UDP broadcast. It means that every message
is simuloneusly sent to all possible recipients with just one network packet.
Every listener selects packets it wants to listen to and processes them as it wishes.
As a result, minmal MQTT/UDP implementation is extremely simple. Though, there are more
options exist which are described later.
Main use cases for MQTT/UDP are covered below.
Possible topologies¶
Here is a list of more or less obvious scenarios for MQTT/UDP
Multiple smart switches¶
Some wall switches are controlling the same device. All of them send
and read one topic which translates on/off state for the device.
Of course, if one switch changes the state, all others read the state broadcast
and note it, so that next time each switch knows, which state it should
switch to.
It is possible, of course, that UDP packet from some switch will be lost.
So when you switch it, nothing happens. What do you do in such a situation?
Turn switch again, of course, until it works!
In this example I wanted to illustrate that even in this situation UDP
transport is not really that bad.
Reliability¶
As MQTT/UDP is based on UDP protocol, which does not guarantee packet delivery, one can suppose that MQTT/UDP is not reliable. Is it?
Not at all.
If we use it for repeated updates, such as sensor data transfer, UDP is actually more reliable, than TCP! Really. If our network drops each
second packet, TCP connection will be effectively dead, attempting to resend again and again outdated packets which are not needed anymore.
And MQTT/UDP will just loose half of readings, which is not really a problem for 99% of installations. So, TCP helps just if packet loss rate
is quite low.
Actualy, simple test was made [1] to ckeck UDP reliability. One host in my house’s local net was generating MQTT/UDP traffic as fast as
possible and other checked packets to be sequent, counting speed and error rate. Two IPTV units was started to show HD content and one
of the computers was copying some few GBytes to file server. Result was quite surprising: MQTT/UDP error rate grew to… 0.4% with about 50K
packets/second, but TV sets stopped showing, being, obviusly, starved.
Anyway, I’m going to add completely reliable mode to MQTT/UDP in near future.
Footnotes
Remote configuration¶
MQTT/UDP can be used as a remote configuration protocol. There are two basic
modes of remote configuration: acive and passive. Active remote configuration mode
is needed when node to be configured has no local storage at all and has to
request all settings at each restart. Passive mode is good for nodes that keep
settings from start to start (in local FS or NVRAM) and configuration process is
executed for them once or rarely.
Passive remote configuration¶
Passive remote configuration is divided in three steps. First, when user
starts configurator GUI application, application sends (and repeats from time
to time) a SUBSCRIBE request. Second, any MQTT/UDP nodes that support remote
configuration see that request and answer with PUBLISH packet for all items
that can be set up. Last, configurator application sends back message with a new
item value.
All the configurable items must have special topic name: $SYS/conf/{node-id}/{kind}/{name}
.
For example: $SYS/conf/024F55A20001/info/uptime
.
Predefined items¶
Protocol implementations expect some item kind/name pairs to mean special thins.
Here is a list of know ones.
info/soft
- Readonly, name of software running in this node.
info/ver
- Readonly, version of software.
info/uptime
- Text string, uptime: “255d 02:12:34” or “5y 2m 13d 23:55:01”
node/name
- Name of this MQTT/UDP node. Human readable, not interpreted. “Weather sensors”
or “A/C control” node/location
- Where you can find it: “Kitchen”, “building 12, 2nd floor, room 212”, whatever.
net/mac
- Network MAC address, if not hardwired.
net/ip
- Network IP address, if static.
net/mask
- Netmask.
net/router
- Network default route.
Other network settings can be put to net
kind.
What is tcpview used for?
TCPView is a program ideal for people who are interested in knowing the backend of things on their computer. For instance, it informs you of what actually happens when an address is typed into a web browser, or when online games and antivirus applications are at play.
A lot of internet points such as web servers and DNS, routers, gateways, switches, etc. get to work whenever you initiate an action online. The speed with which the output arrives depends on the nodes the signal is made to travel through.
The lesser the nodes involved, the speedier the entire process. This free-of-charge networking information tool shows you the complete information about your PC’s TCP/UDP endpoints. When you launch the program, a classic window opens up, comprising the main area where the processes get listed.
Work in progress¶
There are parts of protocol or additional components design that
not finished completely, and are subject of discussion.
QoS Server¶
Nowadays UDP is quite reliable (typical loss rate is < 0.5% even in a busy network),
but not 100% reliable. Things are easy in point to point communications. Send a message,
wait for acknowledgement, resend if none received in a reasonable time. But
MQTT/UDP is broadcast protocol. Do we have to wait for ack from all nodes in a network?
Some nodes? Which ones?
It is makes sense that we can build a map of nodes that listen to us by collecting
their responces. But we want to keep MQTT/UDP implementation simple and this is not
that simple. And not any node needs such high a reliability.
The idea is to add separate server on a network that will build lists of listeners
for each topic, collect low-QoS ack packets and sent one high-QoS ack packet
to topic publisher(s).
Note that such server is not a single point of failure. First of all, there can be more
than one instance of QoS server. Second, even if QoS server fails, nodes continue to
send data. Though, each packet will be resent few times, but it is not a communications
failure. Last, but not least, sending node can stop resending after few acks with lower
QoS. For example, sending node can take for acknowledge one QoS 3 ack message and 2 or 3
QoS 2 ones.
Аргументы командной строки
Использование tcpvcon аналогично использованию встроенной утилиты Windows netstat.
Использование. Введите
команду tcpvcon [-a] [-c] [-n] [название процесса]
- -a Отображает
существующие конечные точки (по умолчанию отображаются установленные TCP-соединения). - -c Распечатать вывод в формате CSV.
- -n Запрещать
адреса.
Изменения в последней версии
- Это обновление исправляет ошибку при сортировке по столбцу состояния.
Инструкция по использованию
Когда вы запускаете TCPView, он перечисляет активные конечные точки TCP и UDP, преобразовывая все IP-адреса
в их версии доменного имени. Кнопка панели инструментов или пункт меню может
быть использована для переключения отображения разрешенных имен. В системах Windows XP TCPView показывает имя процесса, которому принадлежит
каждая конечная точка.
По умолчанию утилита обновляется
каждую секунду. Однако, можно воспользоваться пунктом Меню — Настройки —
Частота обновления. Там можно изменить значение на большее или меньшее.
Конечные точки, которые меняют состояние от одного обновления к другому,
выделены желтым. Те, которые были удалены, показаны красным, а новые конечные
точки показаны зеленым.
Вы можете закрыть
установленные TCP / IP-соединения, выбрав Файл — Закрыть
соединение, или щелкнув на соединение правой кнопкой мыши и выбрав «Закрыть
соединение» из контекстного меню.
Можно сохранить окно
вывода TCPView в файл, используя пункт меню «Сохранить».
Недостатки tcpview
- Отсутствие поддержки русского и украинского языков интерфейса.
Преимущества tcpview
- Бесплатное распространение продукта.
- Простой графический интерфейс пользователя.
- Возможность сохранения содержимого таблицы в файл.
- Возможность завершать процессы по необходимости.
- Наличие цветовой подсветки новых, удаленных и измененных процессов.
- Наличие консольного варианта утилиты, распространяемого совместно с основной программой.
- Отсутствие необходимости установки продукта.
- Малый размер программы, который составляет менее 1-го МБайта.
Заключение
Продукт рекомендуется системным администраторам как средство мониторинга и диагностики возможных проблем, связанных, например, с обновлением ПО или отсутствием дозвона с IP-АТС.
Addendums¶
Work in progress¶
There are parts of protocol or additional components design that
not finished completely, and are subject of discussion.
QoS Server¶
Nowadays UDP is quite reliable (typical loss rate is < 0.5% even in a busy network),
but not 100% reliable. Things are easy in point to point communications. Send a message,
wait for acknowledgement, resend if none received in a reasonable time. But
MQTT/UDP is broadcast protocol. Do we have to wait for ack from all nodes in a network?
Some nodes? Which ones?
It is makes sense that we can build a map of nodes that listen to us by collecting
their responces. But we want to keep MQTT/UDP implementation simple and this is not
that simple. And not any node needs such high a reliability.
The idea is to add separate server on a network that will build lists of listeners
for each topic, collect low-QoS ack packets and sent one high-QoS ack packet
to topic publisher(s).
Note that such server is not a single point of failure. First of all, there can be more
than one instance of QoS server. Second, even if QoS server fails, nodes continue to
send data. Though, each packet will be resent few times, but it is not a communications
failure. Last, but not least, sending node can stop resending after few acks with lower
QoS. For example, sending node can take for acknowledge one QoS 3 ack message and 2 or 3
QoS 2 ones.