WebCMS3代写、代做Data Service、c/c++语言代写、Java，Python程序代做

日期：2020-03-30 09:20

Assignment-2 | COMP9321 20T1 | WebCMS3

Resources / Assignments (/COMP9321/20T1/resources/41975)

/ Week 7 (/COMP9321/20T1/resources/41978) / Assignment-2

Assignment-2

Data Service for World Bank Economic Indicators

In this assignment, you are asked to develop a Flask-Restplus data service that allows a client to read and

store some publicly available economic indicator data for countries around the world, and allow the consumers

to access the data through a REST API.

IMPORTANT : For this assignment , you will be asked to access the given Web content programmatically.

Some Web hosts do not allow their web pages to be accessed programmatically, some hosts may block your

IP if you access their pages too many times. During the implementation, download a few test pages and

access the content locally - try not to perform too many programmatically.

The source URL: http://api.worldbank.org/v2/ (http://api.worldbank.org/v2/)

(http://api.worldbank.org/v2/) Documentations on API Call Structure:

(https://datahelpdesk.worldbank.org/knowledgebase/articles/898581-api-basic-call-structure)

https://datahelpdesk.worldbank.org/knowledgebase/articles/898581-api-basic-call-structure

(https://datahelpdesk.worldbank.org/knowledgebase/articles/898581-api-basic-call-structure)

(https://datahelpdesk.worldbank.org/knowledgebase/articles/898581-api-basic-call-structure)

The World Bank indicators API provides 50 year of data over more than 1500 indicators for countries around

the world.

List of indicators can be found at: http://api.w o rldbank.org/v2/indicators

(http://api.worldbank.org/v2/indicators)

List of countries can be found at: (http://api.worldbank.org/v2/countries)

http://api.worldbank.org/v2/countries (http://api.worldbank.org/v2/countries)

As the documentation shows , you can construct URLs specifying different parameters to acquire necessary

data. Use a custom URL to get information about a specific indicator. For example, the data on the GDP of all

countries from 2012 to 2017 can be acquired via this URL:

(http://api.worldbank.org/v2/countries/all/indicators/NY.GDP.MKTP.CD?date=2012:2017)

http://api.worldbank.org/v2/countries/all/indicators/NY.GDP.MKTP.CD?

date=2012:2017&format=json&per_page=1000

(http://api.worldbank.org/v2/countries/all/indicators/NY.GDP.MKTP.CD?

date=2012:2017&format=json&per_page=1000)

For this assignment we are interested in annual values of a user specified indicator from 2012 to 2017 for one

or all countries. The content of the file is quite straightforward. The data service specification will ask you to

'import' this data into a 'data storage'. You should inspect the content of the file carefully and decide on a data

model and storage method. You will find that a JSON format is suitable to accessing data and publishing it.

Specification Make Submission Check Submission Collect Submission

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 2/10

Assignment Specification

The data service should use JSON format to publish its data and implement following operations.

Question-1: Import a collection from the data service (2.5 marks)

This operation can be considered as an on-demand 'import' operation. The service will download the JSON

data for all countries (http://api.worldbank.org/v2/countries/all/indicators/NY.GDP.MKTP.CD?

date=2012:2017&format=json&per_page=1000) respective to the year 2012 to 2017 and identified by the

indicator id given by the user and process the content into an internal data format and store it in the database;

use sqlite for storing the data (the name of the database should be YOURZID.db ) .

For example, the following link can be used to obtain data for the indicator called: NY.GDP.MKTP.CD

(http://api.worldbank.org/v2/countries/all/indicators/NY.GDP.MKTP.CD?

date=2012:2017&format=json&per_page=1000)

http://api.worldbank.org/v2/countries/all/indicators/ NY.GDP.MKTP.CD ?

date=2012:2017&format=json&per_page=1000

(http://api.worldbank.org/v2/countries/all/indicators/NY.GDP.MKTP.CD?

date=2012:2017&format=json&per_page=1000)

To import data your require an indicator as a parameter given by the API user. The parameter should be a

query parameter named:

indicator_id : an indicator (http://api.worldbank.org/v2/indicators) http://api.worldbank.org/v2/indicators

(http://api.worldbank.org/v2/indicators)

After importing the collection, the service should return a response containing at least the following information:

uri : the URL with which the imported collection can be retrieved

id : a unique integer identifier automatically generated

creation_time : the time the collection stored in the database

Example:

HTTP operation: POST /collections?indicator_id=NY.GDP.MKTP.CD

Returns: 201 Created

{

"uri" : "/collections/1",

"id" : 1,

"creation_time": "2019-04-08T12:06:11Z",

"indicator_id" : "NY.GDP.MKTP.CD"

}

You should return appropriate responses (JSON formatted) in case of already imported collections,

invalid indicators or any invalid attempts to use the endpoint ( e.g. If the input indicator id doesn't exist in

the data source, return error 404 )

UPDATE: Please remove those entries where the value is NONE

Question 2- Deleting a collection with the data service (2.5 marks)

This operation deletes an existing collection from the database. The interface should look like as below:

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 3/10

HTTP operation: DELETE /collections/{id}

Returns: 200 OK

{

"message" :"The collection 1 was removed from the database!",

"id": 1

}

Question 3 - Retrieve the list of available collections (2.5 marks)

This operation retrieves all available collections. The interface should look like as like below:

"order_by" is a comma separated string value to sort the collection based on the given criteria. Each segment

of this value indicates how the collection should be sorted, and it consists of two parts (+ or -, and the name of

column e.g., id). In each segment, + indicates ascending order, and - indicates descending order. Here are

some samples:

+id,+creation_time order by "id ascending" and then "creation_time ascending"

In this case sorting by "id" has priority over "creation_time "

-creation_time order by "creation_time descending"

Returns: 200 OK

[

{

"uri" : "/collections/1",

"id" : 1,

"creation_time": "2019-04-08T12:06:11Z",

"indicator" : "NY.GDP.MKTP.CD"

},

{

"uri" : "/collections/2",

"id" : 2,

"creation_time": "2019-05-08T12:16:11Z",

"indicator" : "2.0.cov.Math.pl_3.all"

},

...

Question 4 - Retrieve a collection (2.5 marks)

This operation retrieves a collection by its ID . The response of this operation will show the imported content

from world bank API for all 6 years. That is, the data model that you have designed is visible here inside a

JSON entry's content part.

The interface should look like as like below:

HTTP operation: GET /collections?order_by={+id,+creation_time,+indicator,-id,-creation_time,-in

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 4/10

HTTP operation: GET /collections/{id}

Returns: 200 OK

{

"id" : 1,

"indicator": "NY.GDP.MKTP.CD",

"indicator_value": "GDP (current US$)",

"creation_time" : "2019-04-08T12:06:11Z"

"entries" : [

{"country": "Arab World", "date": 2016, "value": 2500164034395.78 },

{"country": "Australia", "date": 2016, "value": 780016444034.00 },

...

]

}

Question 5 - Retrieve economic indicator value for given country and a year (2.5

marks)

The interface should look like as like below:

HTTP operation: GET /collections/{id}/{year}/{country}

Returns: 200 OK

{

"id": 1,

"indicator" : "NY.GDP.MKTP.CD",

"country": "Arab World",

"year": 2016,

"value": 780016444034.00

}

Question 6 - Retrieve top/bottom economic indicator values for a given year

(2.5 marks)

The interface should look like as like below:

HTTP operation: GET /collections/{id}/{year}?q=<query>

Returns: 200 OK

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 5/10

Resource created 29 days ago (Thursday 27 February 2020, 12:54:52 PM), last modified a day ago (Thursday 26 March 2020,

08:04:12 AM).

{

"indicator": "NY.GDP.MKTP.CD",

"indicator_value": "GDP (current US$)",

"entries" : [

{

"country": "Arab World",

"value": 2500164034395.78

},

...

]

}

The <query> is an optional integer parameter which can be either of following:

+N (or simply N) : Returns top N countries sorted by indicator value (highest first)

-N : Returns bottom N countries sorted by indicator value

where N can be an integer value between 1 and 100. For example, a request like " GET /collections/1 /2015?

query=+10 " should returns top 10 countries according to the indicator value.

Notice:

Your code must implemented in flask-restplus and automatically generate swagger doc for testing the

endpoints.

Your code must be executable in CSE machines

Your code must not require installing any software (python libraries are fine)

Your code must be written in Python 3.5 or newer versions.

Your operations (API methods) must return appropriate responses in JSON format, and appropriate

HTTP response code! e.g., 500 Internal Server Error is inappropriate response!

Make sure you are using right datatypes in the database and in you API methods (e.g., not string for

years '2017')

Install flask_restplus on cse machines by pip3 command, and make sure you install pip3 install

werkzeug~=0.16.1

Submission:

One and only one Python script file named " YOUR_ZID .py" which contains your code

How I can submit my assignment?

Go to the assignment page click on the "Make Submission" tab; pick your files which must be named

"YOUR_ZID.py". Make sure that the files are not empty, and submit the files together.

Can I submit my file after deadline?

Yes, you can. But 25% of your assignment will be deducted as a late penalty per day. In other words, if

you be late for more than 3 days, you will not be marked.

Comments

  (/COMP9321/20T1/forums/search?forum_choice=resource/43071)

 (/COMP9321/20T1/forums/resource/43071)

 Add a comment

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 6/10

Shunyang Li (/users/z5139935) 15 minutes ago (Fri Mar 27 2020 18:06:26 GMT+1100 (澳大利亚东部

夏令时间))

For q2, q4, q5 and q6 if there does not have any records in the database according to id, it

should return a empty list or 404 error message?

Reply

Shunyang Li (/users/z5139935) about an hour ago (Fri Mar 27 2020 17:23:28 GMT+1100 (澳大利亚东

部夏令时间))

hi, do we need to consider block the user users send requests too frequently for Q1, like

record the IP and calculate time difference (now_date - last_date)?

Reply

Zac Parsons (/users/z5165177) about 3 hours ago (Fri Mar 27 2020 15:02:17 GMT+1100 (澳大利亚东

部夏令时间))

are we required to implement some form of threading to account for database write times.

adding a collection takes a few seconds with my implementation.

Reply

Shichao Zhang (/users/z5178127) about 4 hours ago (Fri Mar 27 2020 14:44:34 GMT+1100 (澳大利亚

东部夏令时间))

In the last question, Q6, I use the inputs.int_range(self, low, high) to set the keyword

argument 'type' in RequestParser.add_argument(). This cause the Swagger UI not allow '+N'

to be a input but 'N' is ok. However, in other test tools such as Postman, I can pass '+N'

through the url and my code works well.

My question is 'Will this affect the evluation?'

Reply

Kanadech Jirapongtanavech (/users/z5176970) about an hour ago (Fri Mar 27 2020 17:08:05

GMT+1100 (澳大利亚东部夏令时间))

Make it str type. Works for me.

Reply

Andrew Lei (/users/z5207966) about an hour ago (Fri Mar 27 2020 17:12:13 GMT+1100 (澳大

利亚东部夏令时间)), last modified about an hour ago (Fri Mar 27 2020 17:15:23 GMT+1100 (澳大利

亚东部夏令时间))

But the spec states that it is an integer parameter. Can we assume that only N will be

entered and not +N? Also, will testing be done only on swagger.io?

Reply

Shichao Zhang (/users/z5178127) about an hour ago (Fri Mar 27 2020 17:28:48

GMT+1100 (澳大利亚东部夏令时间))

The reason of this issue is that Swagger UI will check the input fields according to

the parameter of add_argument and '+N' will not be considered to a integer.

However, '+N' can be converted into an integer in backend.

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 7/10

If we bypass the Swagger UI, the behaviour of this API actually conforms the

spec.

Reply

Shichao Zhang (/users/z5178127) about 4 hours ago (Fri Mar 27 2020 14:28:38 GMT+1100 (澳大利亚

东部夏令时间))

In the Question 1, I notice when I request DT.AMT.DECT.CD.03.US

(http://api.worldbank.org/v2/countries/all/indicators/DT.AMT.DECT.CD.03.US?

page=1&per_page=1000&date=2012:2017&format=json) , I got a json object whose second

entity is 'null'. This means that client posted a valid indicator ID and my server also

understand it. However, I actually do not need to store any data in database. So, which

status code should I return client? 204 or 404?

Reply

Adam El Mohamad (/users/z5205906) about 7 hours ago (Fri Mar 27 2020 11:48:58 GMT+1100 (澳大

利亚东部夏令时间))

for question 6, how do I specify that the <query> parameter is optional, I have searched

online and cannot find anything. I am really stuck.

Reply

Marvin Chui (/users/z5230341) about 6 hours ago (Fri Mar 27 2020 12:13:50 GMT+1100 (澳大利

亚东部夏令时间))

When they say its optional they mean you just have to make sure your code has a way to

handle the case where the parameter isn't provided and the case where it is. Refer to

Activity 1 of the Week 6 lab.

Reply

Adam El Mohamad (/users/z5205906) about 6 hours ago (Fri Mar 27 2020 12:19:51

GMT+1100 (澳大利亚东部夏令时间))

I know what it means. I have seen the activity and used those methods but my query

parameter is still not appearing when I try to test my code at the endpoint

Reply

Chen Ling (/users/z5206193) 43 minutes ago (Fri Mar 27 2020 17:37:37 GMT+1100 (澳大

利亚东部夏令时间))

Do you have @api.expect(parser) inside your class?

Reply

Joseph Aczel (/users/z5194935) about 7 hours ago (Fri Mar 27 2020 11:33:10 GMT+1100 (澳大利亚东

部夏令时间))

Does the datetime object need to be formatted exactly as you have shown? Is the following

acceptable:

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 8/10

{

"uri": "/collection/1",

"id": 1,

"creation_time": "2020-03-27 11:31:17.607330",

"indicator_id": "NY.GDP.MKTP.CD"

}

Reply

Alex Pham (/users/z5209652) about 19 hours ago (Thu Mar 26 2020 22:55:49 GMT+1100 (澳大利亚东

部夏令时间)), last modified about 18 hours ago (Fri Mar 27 2020 00:04:11 GMT+1100 (澳大利亚东部夏令时

间))

Do we have to create the database in the code or will it be assumed it exists?

Edit: Nevermind about this question I just realised the database gets created

automatically when I connect to it

Also can we assume that the returned JSON from the API will follow the same format? I.e.

Have indicator.id, indicator.value, country.id, country.value, countryiso3code, date, value,

unit, obs_status and decimal if the response from the API is 200?

Reply

Mohammadali Yaghoubzadehfard (/users/z5138589) about 11 hours ago (Fri Mar 27 2020

07:38:20 GMT+1100 (澳大利亚东部夏令时间)), last modified about 11 hours ago (Fri Mar 27 2020

07:39:07 GMT+1100 (澳大利亚东部夏令时间))

The database should be automatically by the code created if does not exist.The

responses should follow the same format specified in the specs.

Reply

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 9/10

Alex Pham (/users/z5209652) about 9 hours ago (Fri Mar 27 2020 09:12:38 GMT+1100 (澳大

利亚东部夏令时间))

Sorry I should’ve clarified. This is the output from the world bank API (not mine), can

we assume it will always look like this

Reply

Andrew Lei (/users/z5207966) about 18 hours ago (Thu Mar 26 2020 23:57:11 GMT+1100 (澳大利

亚东部夏令时间)), last modified about 18 hours ago (Fri Mar 27 2020 00:01:45 GMT+1100 (澳大利亚东

部夏令时间))

I just followed the same structure (exact same output) as the samples but we should wait

for confirmation.

Reply

Mohammadali Yaghoubzadehfard (/users/z5138589) about 11 hours ago (Fri Mar 27 2020

07:40:00 GMT+1100 (澳大利亚东部夏令时间))

Follow the same structure as specified in the specs above

Reply

Andrew Lei (/users/z5207966) about 20 hours ago (Thu Mar 26 2020 22:13:01 GMT+1100 (澳大利亚东

部夏令时间)), last modified about 13 hours ago (Fri Mar 27 2020 04:51:49 GMT+1100 (澳大利亚东部夏令时

间))

1. I'm using sqlite3 instead of sqlite. Is this ok?

2. Is the query parameter (indicator_id) in Q1 optional?

3. Are the only valid parameters in Q3 are id, creation_time and indicator?

4. If I limit my variable inputs to only allow a specific type (e.g. string) in the api, do I still

need to account for entering data type errors?

5. Can I store datetimes as a string type (text) in sqlite3 and output it as a string type as well

(or does it have to be another type?) - if you are allowed to tell us, what is the type for the

creation_time given in the samples (I don't know how to get the T and Z in the output)?

6. Correct output would be 'content-type: application/json' in the swagger.io response header

location?

Reply

Mohammadali Yaghoubzadehfard (/users/z5138589) about 11 hours ago (Fri Mar 27 2020

Reply

2020/3/27 Assignment-2 | COMP9321 20T1 | WebCMS3

https://webcms3.cse.unsw.edu.au/COMP9321/20T1/resources/43071 10/10

Load More Comments

Wenbin Liu (/users/z5199359) about 20 hours ago (Thu Mar 26 2020 21:57:47 GMT+1100 (澳大利亚东

部夏令时间))

Should I set debug=False before I submite the code.py ?

Reply

Mohammadali Yaghoubzadehfard (/users/z5138589) about 11 hours ago (Fri Mar 27 2020

07:35:42 GMT+1100 (澳大利亚东部夏令时间))

Makes no difference

Reply