代写COMP3310、代写C++/Java/Python编程

日期：2023-05-13 04:23

Page 1 of 5

COMP3310/ENGN3539/etc Assignment 3 – Testing MQTT

Introduction:

? This assignment is worth 15% of the final mark

? It is due by Tuesday 23 May 23.55 AEST (Canberra time)

? Late submissions will not be accepted, except in special circumstances.

? Any extensions should be requested well before the due date, with appropriate evidence. Please use

the extension-request link on wattle rather than direct emails.

Assignment 3

MQTT is the most common open IoT protocol being deployed today. It provides a publisher/subscriber

model, using a broker or server. It allows for an almost-unbounded number of sources to publish

information, each at their own rate, and subscribers to receive information (and act on it). As such, it is

designed to provide high-performance communication mechanisms, with minimal delays and excellent

scaling performance. We’ll use it to monitor the performance of some imaginary system: say counting the

total kilograms of minerals rushing by on a conveyor belt, that you can control. This assignment will look at

the functionality and performance of the publishers, brokers, the network and the subscribers.

This is a coding, analysis and writing assignment. You may code in C/Java/Python or any programming

language that a tutor can assess (hope that’s enough for everyone), and yes, you may use MQTT and other

helper libraries. The assessment will not rely solely on running on your code, but more on the data

gathering and your analysis. However, we will review the code and may briefly test it against our own

broker running in the usual lab-type environments or similar. You will need to identify in your

README/code any libraries you are using.

Please note, you will be working in pairs or small groups for a key part of this assignment, to test each

other’s systems across the Internet. If you don’t have somebody to work with, ask your tutors NOW to help

identify somebody in your tutorial group, or post to the Wattle Forum under the pinned Find-a-Partner

topic. You do not need to be in the same location at any time.

Submitting

You will be submitting two things: your code and your analysis report. Note that there will be two

submission links on the Wattle course-site:

1. Your code must be submitted to wattle as a zip file, with README instructions on how to

compile/run the components as appropriate.

2. Your analysis report (pdf) must be submitted via TurnItIn on the wattle site, so ensure you quote

and cite sources properly.

Each question has an indication of how much (rough maximum) you should write.

Outcomes

We’re assessing your understanding of MQTT, your code’s functionality in subscribing/publishing to a

broker, dealing with high message rates from a number of sources, measuring message performance, and

your insight to interpret what you’re seeing. You will be exploring MQTT functionality, the quality-of-service

(QoS) levels, describing how they work, why you would use different levels, and how they perform in realworld deployments given various publishers.

Page 2 of 5

Resources

You will need to set up your own MQTT server/broker for you to connect to as per the specifications below.

You and your partner can share the same broker, if that suits. The Mosquitto broker is perhaps the best

choice to run on your own computer, it has binaries and/or packages for most operating systems, but if you

find a better one you can use that. It must be at least MQTT v3.1.1 compliant (MQTT v5 is not necessary).

A key consideration is that you will need to allow connection to it over the Internet for part of the

assignment. If running it at home where you have a home modem/router that does NAT you will need to

work out how to configure it to allow remote connections (think about the port(s) you need to forward,

and how to manage any changing IP addresses). If you have a firewall running on your homenetwork/computer, you may need to unblock the appropriate ports there too. There are a few other ways

to solve this, e.g. a VPN service on your modem or a local computer, or tools like Cloudflare-tunnels or

Twingate-proxy. If it proves too difficult you might consider setting up a cloud server with a public IP

address on e.g. Amazon, Azure, Alibaba, etc. and installing a broker there. Yet another option is to set up a

(free) fully-hosted broker through a provider such as EMQX.io. Public brokers may have performance vary

non-deterministically due to other people hammering at random times.

Your broker should be configured for some basic authentication and access control, so try to use:

? username student and password 33102023

? require the client-id to start with 3310-

(Public brokers may limit your ability to configure this, so note your approach in your code/README.)

You can test the broker works by subscribing to the $SYS topics, which describe the server, and help get you

familiar with the information presented there - you will be using them for your analysis later.

Assignment programming:

You need to write three small programs1

. The diagram at the end might help to clarify the following.

? A Publisher:

o You will have 1, 2 or 3 instances of the Publisher running2

. This will help stress the broker

and the network. Each publisher needs its own unique client-id.

o Each Publisher will send a simple message to the broker, namely an incrementing counter

(0, 1, 2, 3, …). It will send those messages at a specific MQTT QoS level (0, 1 or 2), and with

a specific delay between messages (0ms, 1ms, 2ms, 4ms, 8ms, 16ms, 32ms).

o Each will publish to the topic counter/<instance>/<qos>/<delay>, so e.g. counter/1/1/16

implies Publisher-instance-1 sending at qos=1 and delay=16. It must only publish the one

stream at any given time, based on input from the Controller. At 0ms delay and qos=0 it

should be able to publish very quickly, potentially several thousand messages per second.

? A Controller: This will listen to the broker on three topics:

o request/qos

o request/delay

o request/instancecount

1 The Controller and Publisher could be in the same program, but then it needs to be efficiently listening and sending

potentially at high data rates at the same time. 2 It could also be a single multi-threaded program, though that may have some additional complexities and

performance limitations.

Page 3 of 5

Whenever the Controller gets a new message on any of these topics it needs to modify the

Publisher’s behaviour accordingly. All 1-3 publisher instances will run the same qos/delay

parameters. One way is to stop any current publisher processes (e.g. send a KILL signal, or at

worst put a 1.5-minute time limit on it running, see below) and start a new one with the new

parameters, say via command line arguments. Another approach might be through eventhandling and multi-threading within a combined publisher/controller, but it is up to you. Please

explain in your README file or code comments what approach you are taking.

? An Analyser: This will listen to the counter(s) on the broker and take measurements (below) to

report statistics on the performance of the publisher/broker/network combination. Those

measurements should be taken across the range of qos, delay and instance-count values as above,

so that you can compare them. Your Analyser will publish to the request/qos, request/delay and

request/instancecount topics, asking the Controller to get some number of Publishers to deliver

what your Analyser needs at that time. For the QoS, subscribe at the same QoS as requested from

the Publisher (i.e. if Publisher->broker qos=2 then Analyser->broker should have qos=2 as well). The

Analyser does not need to take all (7 delay * 3 qos * 3 instances) measurements sequentially in one

run, but the closer in time the measurements occur the better, as network (and broker) conditions

do change all the time.

You can test all of these at home on a local broker. It is only in the final measurements that your broker will

allow another student’s Analyser to access your accessible broker/Controller, and vice-versa.

Analysis and Reporting

In your report, please address the following:

1. Subscribe to some broker, you can use any mqtt client you like for this, to retrieve some $SYS/#

value. Wireshark the handshake for one example of each of the differing QoS-levels (0,1,2), include

screenshots in your report that show the wireshark capture (filter for mqtt), and briefly explain how

each QoS transfer works, and what it implies for message duplication and message order. Discuss

briefly in your report in which circumstances would you choose each QoS level, and offer an

example of a sensor that would use each of them. [around 1 page]

2. Run your Analyser against your partner’s broker, have it tell the broker (and so their Controller)

what you want their Publisher to send, and record information for 1 minute for each set of the

qos/delay/instance-count parameters.

(Tips: (i) only print to screen for debugging, otherwise it will slow your code down a lot. (ii) Check

with the broker owner what delay there might be between your request and their publisher

responding. (iii) Use the counter values to tell you what messages are arriving, or are not arriving as

expected, and what values they start/stop at when you subscribe to calculate the rates below. (iv)

try publishing at a very low rate (i.e. high-delay) to start with, to check your code)

a. Collect statistics, for each instance-count, delay and QoS level, and measure over the 1-

minute period:

i. The overall average rate of messages you actually receive from all publishers across

the period [messages/second].

ii. The rate of message loss you see [percentage].

(how many messages did you see, versus how many should you have seen)

iii. The rate of any out-of-order messages you see [percentage]

(i.e. how often do you get a smaller number after a larger number)

Page 4 of 5

iv. The mean and median inter-message-gap you see, compared to the requested

delay [milliseconds].

Only measure for actually consecutive counter-value messages, ignore the

gap if you miss any messages in between.

b. While measuring the above also

i. Subscribe to and record the $SYS topics, and identify what, if anything, on the

broker do any loss/misordered-rates correlate with. (Look at measurements under

e.g. ‘load’, as well as the ‘heap’ and ‘active clients’ and ‘messages’, anything that

seems relevant. See https://mosquitto.org/man/mosquitto-8.html though it may

not quite match your broker)

ii. Record a traceroute between the Analyser and the broker, and between the

Publisher and the broker (you’ll need the broker owner to provide the second one)

3. In your report: [around 2-3 pages of text, plus any charts]

a. Summarise your measurements. Explain what you expected to see, especially in relation to

the different QoS levels, and whether your expectations were matched.

b. Describe what correlations of measured rates with $SYS topics you expect to see and why,

and whether you do, or do not.

c. Report the traceroutes captured above.

4. Consider the broader end-to-end (internet-wide maybe) network environment, in a situation with

millions of sensors publishing regularly/irregularly to thousands of subscribers. Explain in your

report [around 1-2 pages]

a. What (cpu/memory/network) performance challenges there might be when using MQTT in

general, from the source publishing its messages, all the way through the network and

broker to your subscribing client application. If you lose messages, where might they be

lost?

b. How the different QoS levels may help, or not, in dealing with the challenges.

Amendments and Assessment

Any amendments to the assignment specification will be published in the wattle forum.

Your assignment will be assessed on

? Your code clarity, and documentation/comments (15%)

? your code subscribing and properly publishing/listening to the broker (15%), and

? your analysis report addressing the questions above (70%)

Page 5 of 5

Publisher

3

Publisher

2

Broker

Publisher

1 Controller

Analyser

MQTT:

QoS, Delay, Count

MQTT:

QoS, Delay, Count

MQTT:

1-3 streams of numbers

Step 1

Step 2

Step 3

Step 4

Step 5