MATH6182代写、代写Python语言程序

日期：2022-12-15 04:59

MATH6182 Coursework (2022-23)

December 1, 2022

1 Football Performance Analysis

1.1 Purpose

The aim of this coursework is to extend your basic Python knowledge and analyse a large data set

consisting of thousands of entries. Within this framework, you will need to use your programming

and analysis skills to calculate some vital statistics for individual football teams or leagues and

visualise the results appropriately. The ultimate challenge is to solve a constrained optimisation

problem to predict the outcome of individual football matches based on the available statistical

data and using CVXPY. All exercises must be performed and presented in a Jupyter notebook.

1.2 Data

The required data set can be found on Blackboard, it is called understat_per_game.csv. In

general, it contains statistical quantities of individual teams per game of each season from 2014 to

2019 for 6 UEFA leagues:

In this framework, there are the following standard parameters:

league - Name of the league,

season - Starting year of the season,

team - Name of the team,

h_a - Flag that indicates a home (“h”) or an away game (“a”),

result - Flag that indicates a win (“w”), a draw (“d”) or a lose (“l”),

date - The date and time at which the game took place,

wins - Flag that indicates that the team won the game,

draws - Flag that indicates that the game was a draw,

loses - Flag that indicates that the team lost the game,

scored - Number of goals scored in the game by the team,

missed - Number of goals missed in the game by the team,

pts - Number of points collected for this game by the team.

Moreover, the following additional metrics are available:

1

xG - Expected goals metric (statistical measure for quality of chances created and conceded),

xG_diff - Difference between actual goals scored and expected goals,

npxG - Expected goals without penalties and own goals,

xGA - Expected goals against,

xGA_diff - Difference between actual goals missed and expected goals against,

npxGA - Expected goals against without penalties and own goals,

npxGD - Difference between “for” and “against” xG without penalties and own goals,

ppda_coef - Passes allowed per defensive action in the opposition half (power of pressure),

oppda_coef - Opponent ppda_coef (power of opponent’s pressure),

deep - Passes completed within an estimated 20 yards of goal (crosses excluded),

deep_allowed - Opponent deep,

xpts - Expected points,

xpts_diff - Difference between actual and expected points.

The data set can be loaded into the Jupyter notebook based on the following code.

[1]: import pandas as pd

understat_per_game_data = pd.read_csv('understat_per_game.csv')

understat_per_game_data.shape

[1]: (24580, 29)

The data set contains around 24,500 rows and 29 columns.

[2]: understat_per_game_data.head(4)

[2]: league season h_a xG xGA npxG npxGA deep \

0 Bundesliga 2014 h 2.57012 1.198420 2.57012 1.198420 5

1 Bundesliga 2014 a 1.50328 1.307950 1.50328 1.307950 10

2 Bundesliga 2014 h 1.22987 0.310166 1.22987 0.310166 13

3 Bundesliga 2014 a 1.03519 0.203118 1.03519 0.203118 6

deep_allowed scored … ppda_coef ppda_att ppda_def oppda_coef \

0 4 2 … 9.625000 231 24 21.850000

1 1 1 … 4.756098 195 41 17.695652

2 3 2 … 5.060606 167 33 16.961538

3 2 0 … 4.423077 115 26 9.446809

oppda_att oppda_def team xG_diff xGA_diff xpts_diff

0 437 20 Bayern Munich 0.57012 0.198420 -0.6514

1 407 23 Bayern Munich 0.50328 0.307950 0.5143

2 441 26 Bayern Munich -0.77013 0.310166 -0.8412

3 444 47 Bayern Munich 1.03519 0.203118 1.1367

[4 rows x 29 columns]

2 Tasks

The coursework is divided into 5 tasks:

2

1. Create your own Data Set.

2. Exploratory Data Analysis and Data Manipulation.

3. Performance Analysis for Individual Teams.

4. Performance Analysis for Individual Leagues.

5. Match Prediction.

Each of these tasks has a weighting of 20% for the final grade.

Within this framework, Jürgen Klopp, as an experienced team manager, has been hired to lead the

entire data analysis project. Therefore, he will give you further instructions for all tasks throughout

the coursework.

2.1 Task 1: Create your own Data Set

Since Jürgen Klopp has decided that only data analysts with certain first letters of the first name

can view the data of a team in a certain league, we will all analyse different subsets. Hence, we

first have to make sure that we have a suitable data set for our analyses. So, depending on the first

letter of your first name (1), you will be assigned to one of the following leagues, whose data you

will need to add to your own data set:

?{{{{

?{{{{?

Bundesliga, if 1 ∈ {,,,}

EPL, if 1 ∈ {, ,,}

La Liga, if 1 ∈ {, ,,}

Ligue 1, if 1 ∈ {,,, }

RFPL, if 1 ∈ {,, , }

Serie A, if 1 ∈ {, ,,, , }

However, since your team will also play internationally next season, Jürgen says that we should

add the data of another league to our data set in order to be able to analyse our future opponents

as well. According to Klopp’s calculations, the first letter of your surname (2) decides the league

of one of the teams you will play against in the group stage. So, depending on the first letter of

your surname, add the data of another league to your own data set:

Serie A, if 2 ∈ {,,,}

Bundesliga, if 2 ∈ {, ,,}

EPL, if 2 ∈ {, ,,}

La Liga, if 2 ∈ {,,, }

Ligue 1, if 2 ∈ {,, , }

RFPL, if 2 ∈ {, ,,, , }

It should be noted that Jürgen Klopp has found some errors in his calculations and it may therefore

happen that you receive the same league in the second allocation as in the first allocation. In this

case, Klopp cannot give you any further information about your future opponents and you will have

to randomly select one of the remaining leagues instead, based on a draw with equal probabilities.

3

When doing so, do not forget to set a seed that corresponds to the last three digits of your student

ID before the random selection.

In addition, Jürgen believes that your personal preferences will also have a great influence on the

division of the teams for the group stage. Therefore, arrange the leagues according to your personal

preferences and save them as a list of strings. In first place should be the league you like best, and

so it goes on to the last league. In this context, check whether your first preference is already part

of the data set. If not, add it to your record. Otherwise, continue with the same check for your

second and third preference. This ensures that your data set now contains the matches of football

clubs from three different leagues. Next, also check whether your last preference is part of the data

set. If so, draw a random number in the interval [0, 1] and round it to zero decimals. Do not forget

to set a seed based on the first three digits of your student ID beforehand. If the rounded number

is a “1”, Klopp says that it is very unlikely that you will have to play against a team from that

league. Therefore, you should remove the data from your record and add the data from one of the

remaining unselected leagues based on a random selection with equal probabilities.

Finally, Klopp believes that some outsiders have manipulated the data in an unfavourable way.

Therefore, he proposes to remove a certain number of rows from the data set to get the final subset

with unmanipulated data entries. In this context, you need to draw a number from the interval

[0, 500] based on a random selection with equal probabilities that tells you how many manipulated

rows you have in your dataset and should therefore remove them. Again, do not forget to set a

seed beforehand that corresponds to your entire student ID. Note that you must also randomise

the indices of the entries you are removing.

Jürgen Klopp expects you to define a function that outputs your final data set. Your function

should have four arguments: The first is a pandas DataFrame, the second is your student ID, the

third and fourth are the first letters of your first and last name respectively and the last is the list

of your ordered preferences. At the beginning of the function, please also check that all the entries

in the function are in the correct format, i.e. check that

df - is a non-epmty pandas DataFrame.

student_id - is a string or an integer consisting of 8 letters or individual numbers.

first_name_start_letter - is a string consisting of 1 letter.

last_name_start_letter - is a string consistig of 1 letter.

ordered_preferences - is a list of six strings.

If at least one of the previously defined checks fails, issue an error message, exist the function and

correct the inputs so that you can successfully apply the function. In this framework, you might

find it helpful to inform yourself about the possibility to raise a TypeError. The final dataset can

be named as mydataset or with a name of your choice.

Please use the structure shown below to set up your function. If you do not comment your code,

Jürgen will be mad at you and you will not get full points for this task. Comments in the code are

a very important part of programming so that other programmers can understand your code.

[3]: import numpy as np

def new_dataset(df,

student_id,

first_name_start_letter,

4

last_name_start_letter,

ordered_preferences):

"""

Outputs your individual data set.

:param df: A pandas DataFrame that contains the

"understat_per_game_data.csv" data.

:param student_id: An integer or a string (whatever is

easier for you) that corresponds to

your student ID.

:param: first_name_start_letter: A string that contains the first

letter of your first name.

:param: last_name_start_letter: A string that contains the first

letter of your last name.

:param: ordered_preferences: A list that contains your ordered

preferences regarding the different

leagues.

:return: A pandas DataFrame containing an individual subset of the

df DataFrame.

"""

# 1. Check if function inputs are valid

# [...]

# 2. Subset data based on the starting letter of your first name

# [...]

# 3. Subset data based on the starting letter of your last name

np.random.seed(XXX)

# [...]

# 4. Check your first three preferences

# [...]

# 5. Check your last preference

np.random.seed(XXX)

# [...]

# 6. Remove random amount of rows

np.random.seed(XXXXXXXX)

# [...]

return my_dataset # output the new data set.

5

2.2 Task 2: Exploratory Data Analysis & Data Manipulations

2.2.1 Subtask 2.1

After you have received the relevant data set for your work, the first thing Jürgen Klopp wants to

do is to carry out a detailed review of the data properties. He is still a little afraid that outsiders

might have falsified the data so that our analyses lead to false conclusions. Therefore, as good data

analysts, we check whether we have read in the data correctly and whether there are any suspicious

observations. In this regard, write a function that checks the following properties:

1. All columns are formatted as indicated in the data description file (data_description.csv).

2. The data set does not contain any missing values.

If you find missing values, remove the corresponding rows from the data set. Jürgen says that these

instances have most likely been manipulated by an outsider. In general, your function should have

two inputs corresponding to the data set and the data description file, and it should also have two

outputs: A Pandas DataFrame with three columns [ID of the check, short description of the check

and a flag indicating whether the check failed (1) or passed (0)] and the cleaned version of your

data set.

Do not forget to check the function inputs as already done in the previous task.

Please use the structure shown below to set up your function and print out the head of both

outputs. If you do not comment your code, Jürgen will be mad at you and you will not get full

points for this task. Comments in the code are a very important part of programming so that other

programmers can understand your code.

[4]: def cleaned_dataset(df, data_descrip):

"""

Outputs the cleaned version of your individual data set.

:param df: A pandas DataFrame with your individual data.

:param data_descrip: A pandas DataFrame with the data description.

:return: A pandas DataFrame with a summary of the performed checks and

a pandas DataFrame with the cleaned input data.

"""

# 1. Check if function inputs are valid

# [...]

# 2. Creates an empty table for the summary of your checks

# [...]

# 3. Checks if all columns are in the correct format

# [...]

# 4. Checks if the data contains missing values

# [...]

return check_summary, my_dataset # output.

6

2.2.2 Subtask 2.2

Jürgen Klopp is now sure that we have good data quality, but he still thinks that we need to

do more data preparation to facilitate our later analyses. Therefore, he asks you to also include

transformed versions of certain columns in the data set. Within this framework, please do the

following data manipulations:

1. Add another variable to the data set called “h_a_transformed” which corresponds to a nu-

merical representation of the “h_a” variable, i.e. an “h” is decoded as 1 and an “a” as 0.

2. Based on the “date” column, add three more variables to the data set called “year”, “month”

and “match_time”. The first two should correspond to the year and month when the match

took place and the third should represent the time when the match started.

3. Based on the “wins”, “draws”, “loses” and “pts” columns, for each team calculate the total

number of previous wins, draws, loses and points during the respective season and add each

of these variables to the data set. The columns should be named based on the following

structure: “cumulative_wins”, “cumulative_pts”, […].

Please use the structure shown below to set up your code and at the end print out the head of

your data set. If you do not comment your code, Jürgen will be mad at you and you will not get

full points for this task. Comments in the code are a very important part of programming so that

other programmers can understand your code.

[5]: # Transformation of "h_a" column

# [...]

# Transformation of "date" column

# [...]

# Transformation of "wins", "draws", "loses" and "pts" column

# [...]

# Print out the first five rows of your data set

# [...]

2.2.3 Subtask 2.3

Finally, to get a first impression of the distributions of the different numerical variables, Klopp would

like to see their summary statistics. Therefore, print the following information for all numerical

columns in a combined table: Count, Mean, Standard Deviation, Minimum, 25th Quantile, 50th

Quantile, 75th Quantile and Maximum.

At the end print out the head of your data set. If you do not comment your code, Jürgen will

be mad at you and you will not get full points for this task. Comments in the code are a very

important part of programming so that other programmers can understand your code.

7

2.3 Task 3: Performance Analysis on Individual Teams

2.3.1 Subtask 3.1

Jürgen Klopp now thinks that you are ready for your first performance analysis. Because you are

such a talented data analyst, you have offers from all the clubs that are in your data set and you

are in the comfortable position of deciding which club you want to sign with.

After good negotiations with your chosen club, you are happy to start the analysis. The first thing

Jürgen wants to know is how well your team has done in recent seasons, depending on whether

they have played at home or away. He also asks you to assess whether there are some trends in the

summarised data.

Recall that each row of the data set represents the game statistics of a certain football team. Hence,

you have to write a function that subsets the data based on the team you picked and creates a

DataFrame that summarizes the following information grouped by the season and the type of the

game (i.e., home or away):

The count of wins, draws and loses and,

The average goals, expected goals and expected goals against.

Then the function should also create a plot with four line graphs as subplots to visualise the results

and to simplify later interpretation:

The subplots on the left side should contain three lines showing the evolution of the number

of wins, draws and defeats in home and away matches (y-axis) over the seasons (x-axis).

The subplots on the right side should also contain three lines showing the development of

average goals, expected goals and expected goals conceded for home and away matches (y-

axis) over the seasons (x-axis).

Do not forget to include a suitable legend explaining the individual lines.

At the end the plot should look like this (you can vary the colors if you want to):

8

Do not forget to check the function inputs as already done in the previous tasks. Interpret the

results for Jürgen, e.g., the total number of wins at home has increased over the seasons and […].

Please use the structure shown below to set up your function and print out the head of the output.

If you do not comment your code, Jürgen will be mad at you and you will not get full points for this

task. Comments in the code are a very important part of programming so that other programmers

can understand your code.

[6]: import matplotlib.pyplot as plt

def team_performance(df, team_name):

"""

Outputs the summary table for subtask 3.1.

:param df: A pandas DataFrame with your individual data.

:param own_team: A string corresponding to the name of your team.

:return: A pandas DataFrame with the summary table for your team.

"""

# 1. Check if function inputs are valid

#[...]

9

# 2. Compute statistics

# [...]

# 3. Plot the results

# [...]

return summary_tab

# Print out head of summary tab

2.3.2 Subtask 3.2

Jürgen Klopp now has an idea of how well our team played during last seasons, but he also wants

to be prepared for the next game. It will be a regular league match, so you have to pick another

team from the same league. In this context, he asks you to analyse the tactical behaviour of our

next opponent. Based on his experience, Klopp knows that a team usually changes its tactics over

the course of a season, so you need to group the data by the year and month in which the matches

took place. Jürgen also knows that the team managers of our opponent have changed frequently

in the past and therefore only the data of the last available season shows the tactical behaviour of

the current coach. Based on the previous information, write a function that subsets the data and

groups it accordingly, then it should calculate the following quantities:

The average power of pressure, power of opponent’s pressure, deep and deep allowed.

Then create a line plot to visualise the results and to simplify later interpretation:

The plot should contain four lines showing the evolution of the average power of pressure,

power of opponent’s pressure, deep and deep allowed (y-axis) over the season (x-axis).

At the end the plot should look like this (you can pick different colors if you want to):

10

Do not forget to check the function inputs as already done in the previous tasks. Interpret the

results for Jürgen, e.g., over the season the opponent team has increased its power of pressure and

[…].

Please use the structure shown below to set up your function and print out the head of the output.

If you do not comment your code, Jürgen will be mad at you and you will not get full points for this

task. Comments in the code are a very important part of programming so that other programmers

can understand your code.

[7]: def opponent_performance(df, opponent_name, season):

"""

Outputs the summary table for subtask 3.2.

:param df: A pandas DataFrame with your individual data.

:param opponent_name: A string corresponding to the name of your opponent?

?team.

:param season: An integer corresponding to the starting year of the?

?desired season.

:return: A pandas DataFrame with the summary table for the opponent team.

"""

# 1. Check if function inputs are valid

# [...]

# 2. Summarize and prepare the data for plotting

# [...]

# 3. Plot the results

# [...]

return summary_tab

# Print out head of summary table

2.3.3 Subtask 3.3

Finally, Jürgen Klopp asks you for one more favour. He feels that in recent years our team’s

performance and that of our opponents has depended on the starting time of the game, i.e. our

team has won more games in the evening than at noon. For this purpose, you need to write a

function that creates a plot that contains two different bar charts as subplots:

One for our team and

One for our next opponent.

The two plots should present the following information:

On the x-axis should be the distinct starting times of the games and

On the y-axis the count of wins, draws and loses.

11

At the end the plot should look like this (you can pick different colors if you want to):

Do not forget to check the function inputs as already done in the previous tasks. Interpret the

results and check whether Jürgen was right with his feeling, e.g., our opponent normally wins more

games at noon and […].

Please use the structure shown below to set up your function and print out the head of the output.

If you do not comment your code, Jürgen will be mad at you and you will not get full points for this

task. Comments in the code are a very important part of programming so that other programmers

can understand your code.

[8]: def game_time_performance(df, team_name, opponent_name, season):

"""

Outputs the two summary tables for subtask 3.3.

:param df: A pandas DataFrame with your individual data.

:param team_name: A string corresponding to the name of your own team.

:param opponent_name: A string corresponding to the name of your opponent?

team.

:param season: An integer corresponding to the starting year of the?

desired season.

:return: A pandas DataFrame with the summary table for your own team,

a pandas DataFrame with the summary table for the opponent team.

"""

# 1. Check if function inputs are valid

# [...]

# 2. Prepare the data for plotting

# [...]

# 3. Plot the results

12

# [...]

return summary_tab_own, summary_tab_opp

# Print out head of summary tables

2.4 Task 4: Performance Analysis on Individual Leagues

Since our team will play in the Champions League next season, Jürgen Klopp also wants to be

prepared for international matches. Therefore, he is interested in a tool that gives him the same

analysis as in subtasks 3.1 and 3.2, but for a specific league rather than for an individual team. He

wants to use the tool to evaluate the overall strength of the league and to get an idea of what kind

of football is played in this country.

Therefore, you need to write a function for Klopp that receives a data frame and a league name as

inputs and outputs the two summary tables as in subtasks 3.1 and 3.2 for the appropriate league.

Finally, apply the function to all the leagues in your data set and interpret the results. In particular,

try to find differences in the tactical behaviour of the teams in the individual leagues, e.g. in the

Bundesliga the home teams have on average a higher pressure strength than in the other available

leagues.

Do not forget to check the function inputs as already done in the previous tasks. Please use the

structure shown below to set up your function and print out the head of the output. If you do

not comment your code, Jürgen will be mad at you and you will not get full points for this task.

Comments in the code are a very important part of programming so that other programmers can

understand your code.

[9]: def performance_analysis(df, league_name, season):

"""

Outputs the summary tables of subtasks 3.2 and 3.3 for an individual league?

and season.

In the framework of this function also the corresponding plots are printed.

:param df: A pandas DataFrame with your individual data.

:param league_name: A string corresponding to a league name.

:param season: An integer corresponding to the starting year of the

desired season.

:return: A pandas DataFrame with the summary table of subtask 3.2,

a pandas DataFrame with the summary table of subtask 3.3.

"""

# 1. Check if function inputs are valid

# [...]

# 2. Prepare the data for plotting

# [...]

13

# 3. Plot the results

# [...]

return summary_tab_1, summary_tab_2

# Compute and print out results

# [...]

2.5 Task 5: Game Prediction

Ultimately, Jürgen Klopp wants to have a model that predicts whether a team will win a match

based on the available match statistics. His idea is to apply this model during a match so that he

can change the tactical behaviour of our team in the right way. He knows that you are an expert

in linear programming and that you have used the library PuLP a lot. He has heard about the

linear Support Vector Machine and wants to challenge your data analyst skills a little. Therefore,

he gives you the task of calculating such a model for the given data set using CVXPY. CVXPY is

another optimisation library in Python, similar to PuLP, but it is designed for solving general

convex optimisation problems. You might find helpful information about the new library on this

webpage: https://www.cvxpy.org/.

The basic idea of a linear Support Vector Machine (SVM) is to find the maximum-margin hyper-

plane that separates two clases, i.e., wins and no wins, while allwoing for some violations controlled

by a parameter . A graphical representation based on two independet variables can be found

below (Source: Chapter 5 of the Python Data Science Handbook by Jake VanderPlas):

This general idea then can be formulated as a soft-margin classification problem with the following

objective function that we want to minimize:

max(0, 1 (wx )) + ‖w‖2

where ∈ ? represents the total number of games, x ∈ ? a vector containing the ∈ ?

independent variables available for a single game ∈ {1,… , }, ∈ {1,?1} the decoded target

variable (i.e., 1 for a win and -1 for not a win), w ∈ ? the vector containing the parameters for

the independent variables, ∈ ? is the intercept and > 0 is the previously mentioned penalty

14

parameter controlling the trade-off between a larger margin and the correct classification of the

instances.

By introducing individual slack variables = max(0, 1? (wx?)) for each instance the pre-

vioulsy defined objective function can be transformed into the following constrainted minimization

problem

Read the general documentation of CVXPY and solve the optimisation task in such a way that the

team leader gets his tool. Print your solution and the optimal unconstrainted objective function

value. Compare the predictions with the actual values and print the percentage of correctly classified

matches.

Note that predictions can be made by using the sign function:

sign(wx ? )

You should use the following columns as independet variables: h_a_transformed, xG, npxG,

npxGA, deep, deep_allowed, missed, npxGD, ppda_coef, ppda_att, ppda_def, oppda_coef,

oppda_att, oppda_def, xGA_diff, cumulative_wins, cumulative_draws, cumulative_loses and

cumulative_pts. Further, to train your model you should use 80% of the available data that you

randomly select. In addition, you must standardise all variables beforehand, as the Support Vector

Machine is sensitive to the scaling of the data. Therefore, subtract the empirical mean from each

variable and then divide it by the empirical standard deviation. The remaining data should be

used as testing data to check the out-of-sample performance. Do not forget to apply the same

standardization to the testing data.

If you are interested in getting a deeper insight into how the Linear Support Vector Machine works,

take a look at Chapter 5 of the Python Data Science Handbook by Jake VanderPlas.

Please use the structure shown below to set up your code and print out the value of objective

function, the percentage of correctly classified training and testing observations. If you do not

comment your code, Jürgen will be mad at you and you will not get full points for this task.

Comments in the code are a very important part of programming so that other programmers can

understand your code.

Hint: CVXPY needs all inputs as numpy.array. For the C parameter you can try different values

and see which gives you the best objective function value, a good starting point might be values

between (0, 1].