COMP20008代做、代写Data Processing、Python编程语言调试、Python代做

日期：2020-09-18 11:14

COMP20008 Elements of Data Processing

Project 1

August 27, 2020

Due date

The assignment is worth 25 marks, (25% of subject grade) and is due 8:00am Monday

21st September 2020 Australia/Melbourne time.

Background

A web server has been setup at http: // comp20008-jh. eng. unimelb. edu. au: 9889/ main/

containing a number of media reports on Rugby games. As data scientists, we would like to

extract information from those reports and use that information to improve our understanding

of team performance.

Rugby scores

Understanding the rugby scoring system is important in order to be able to extract scores

from match reports. A rubgy score is listed as x-y where x and y are the number of points

obtained by each team. For example, the following are all valid scores:

10-8

16-0

4-12

Learning outcomes

The learning objectives of this assignment are:

To gain practical experience in written communication skills for data science projects.

To practice a selection of processing and exploratory analysis techniques through visualisation

discussed in lectures and workshops.

To practice crawling and scraping data from the Internet.

To practice using widely used Python library for data processing and gain experience

using library functions which may be unfamiliar and which require consultation of additional

documentation from resources on the Web.

1

COMP20008 2020 SM2

Your tasks

You are to perform a small data science project including some data processing and analysis

using Python. Your responses to Tasks 1-5 must be contained in a single .py file. Specifically,

you have the following tasks:

Task 1 (2 marks)

Crawl the http: // comp20008-jh. eng. unimelb. edu. au: 9889/ main/ website to find a complete

list of articles available.

Produce a csv file containing the URL and headline of each the articles your crawler has found.

The CSV file should have two column headings url and headline and be called task1.csv.

Note: You might want to start with a smaller website to test your crawling implementation

with this site (http: // comp20008-jh. eng. unimelb. edu. au: 9889/ sample/ ).

Task 2 (4 marks)

For each article found in Task 1,

a) extract the name of the first team mentioned in the article. You can find a list of team

names as part of the rugby.json file provided. We will assume the article is written

about that team (and only that team). (2 marks)

Note: Your implementation must make use of the list of teams in rugby.json. We

will run your program with a different rugby.json file and expect to find all the articles

that refer to the teams listed in the modified file. The file we use will follow the same

format, but may have different teams.

b) extract the largest match score identified in the article. You will need to use regular

expressions to accomplish this. We will assume this score relates to the first named

team in the article. (2 marks)

Produce a csv file containing the URL, headline, first team mentioned and first complete

match score of each the articles your crawler has found. The csv file should have four column

headings url, headline, team and score and be called task2.csv.

Note: Some articles may not contain a team name and/or a match score. These articles can

be discarded.

Task 3 (1 mark)

For each article used in Task 2, identify the absolute value of the game difference. E.g. a

14-6 score and a 5-13 score both have a game difference of 8. The value is referred to as the

game difference

Produce a csv file containing the team name and average game difference for each team that

at least one article has been written about. The csv file should have two column headings

team and avg game difference and be called task3.csv.

Page 2

COMP20008 2020 SM2

Task 4 (2 marks)

Generate a suitable plot showing five teams that articles are most frequently written about

and the number of times an article is written about that team.

Save this plot as a png file called task4.png

Task 5 (2 marks)

Generate a suitable plot comparing the average game difference for each team with their

game difference. Ignore any teams that have no articles written about them.

Save this plot as a png file called task5.png

Task 6 (14 marks)

Write a 3-4 page report to communicate the process and activities undertaken in the project,

the analysis, and some limitations. Specifically, the report should contain the following information:

A description of the crawling method and a brief summary the output for Task 1.

(2 marks)

A description of how you scraped data from each page, including any regular expressions

used for Task 2 and a brief summary of the output. (3 marks)

An analysis of the information shown in the two plots produced for Tasks 4 & 5, including

a brief summary of the data used. The plots are to be shown (included) along

with your analysis. (4 marks)

A discussion of the appropriateness of associating the first named team in the article

with the first match score. (2 marks)

At least two suggested methods for how you could figure out from the contents of the

article whether the first named team won or lost the match being reported on and a

comment on the advantages and disadvantages of each approach. (2 marks)

A discussion of what other information could be extracted from the articles to better

understand team performance and a brief suggestion for how this could be done.

(1 mark)

Submission instructions

Your responses to Tasks 1 - 5 must be contained in a single python script (.py) file. As the

output of this file will be verified automatically, it is essential that the program runs without

producing errors. For this assignment you may NOT install any additional packages that

aren’t present on the JupyterHub server, e.g. by using the pip install command. Doing so

will cause your submission to fail our marking scripts.

Submission is via the LMS. Two submission links will be provided, one for the .py file

Page 3

COMP20008 2020 SM2

containing your responses to Tasks 1 - 5 and a second for a .pdf or .docx file containing

your response to Task 6.

Extensions and late submission penalties

If requesting an extension due to illness, please submit a medical certificate to the lecturer.

If there are any other exceptional circumstances, please contact the lecturer with plenty of

notice. Late submissions without an approved extension will attract the following penalties

0 < hourslate <= 24 (2 marks deduction)

24 < hourslate <= 48 (4 marks deduction)

48 < hourslate <= 72: (6 marks deduction)

72 < hourslate <= 96: (8 marks deduction)

96 < hourslate <= 120: (10 marks deduction)

120 < hourslate <= 144: (12 marks deduction)

144 < hourslate: (25 marks deduction)

where hourslate is the elapsed time in hours (or fractions of hours).

This project is expected to require 30-35 hours work.

Academic honesty

You are expected to follow the academic honesty guidelines on the University website

https://academichonesty.unimelb.edu.au

Further information

A project discussion forum has also been created on the Ed forum. Please use this in the

first instance if you have questions, since it will allow discussion and responses to be seen by

everyone. There will also be a list of frequently asked questions on the project page.

Page 4