BERNARD BARUCH COLLEGE

MTH 4130 MATHEMATICS OF DATA ANALYSIS (FALL 2020)

Final

The aim of this take-home final is to illustrate the techniques discussed during the first lecture to predict

temperature as a function of time. The data to be used are historical records of temperatures across the United

States. Data are available on the Noaa website1

and we will focus on the Hourly02 directory. The time is

specified by UTC DATE, UTC TIME, LST DATE and the temperature by T HR AVG.

Part one. As you can see data refer to multiple weather stations across the United States. The data set is not

consistent, for some of the stations there are recordings going back to the year 2000 while for some other weather

stations data recording starts later in time. In addition, it may happen that some of the data are missing only

for few days of the year or few hours of a given day. We will focus on studying temperature relative to the

weather station in Asheville (North Carolina) as a function of time.

(1) Plot the data over different time intervals (e.g. one day, one week, and one month) and realize that

there are periodic variations happening at different time scales. Temperatures variations of night and

day cycles happen with a period of 24 hours while there are seasonality variations happening on the

time scale of months.

(2) Identify a good set of features (all of them have to be a function of time at this point of the project)

to be used with linear regression to describe the time series. Clearly, good candidates for this task are

periodic sine and cosine functions of time with periods that take into account the variations observed

at the previous point. Specify wich features you are using.

(3) By using linear regrassion, build a first predictive model that is valid only for the month of May. In

order to do that you can isolate the month of May from each year of the time series and use hourly

temperature relative to that month only to predict temperatures in May.

Blind cross validation does not really make sense in the context of time series. A better idea is to

train your model using all the data except the one referring to the last year available (that should be

2020) on which you can test your model. Focusing on the prediction of a given month should make

features selection a bit easier since seasonal variations of the temperature should not play such a big role.

The more features you use the more the complexity of your model increases. Show the phenomenon of

overfitting when too many features are included in the model.

(4) Try to build a model that works for the entire year. Now you should include features (again functions

of time) that take into account seasonal variation of the temperature. Include all the data up to 3

days before the day/time you want to predict. So if you are predicting the temperature on May 5th at

4pm. Use all the temperature available up to May 2 at 4pm. Compare the performances of these two

models (the one using only data from May and the one using data over all the year) to predict hourly

temperatures during the month of May.

(5) Build another model using KNN. Here overfitting happens as you decrease the number of points K used

to compute the local averages used in KNN. Explain how you choose K in this context.

(6) How do the linear regression model and KNN compare in terms of predictions?

Part two. Include in your model the features as T MAX and T MIN and P CALC. Do they help to predict the

temperature?

In addition, in this part you should analyze your model from the statistical point of view. How many

coefficient are significantly different from zero in your linear regression model? How big is their variance?

Part three - optional for extra credits. By looking at the data you may realize that there are temperatures

variations that are not regular in time. These are weather systems passing over the weather station. They are

hard to predict and, for instance, cannot be captured with the linear regression model used at the previous

point. Come up with a strategy to predict these events for a given weather stations. This strategy should be

based on the information that can be captured from nearby weather stations by considering that a weather

system over a given region it is likely to move to nearby regions with a pattern that most of the time is regular

(from west to east).

In this case it is useful to see if there is a correlation between signals of different stations. The correlation

analysis should be done taking into account times lag of the signals due to the fact that it takes time for a

weather system to move from a region to another.

1https://www.ncdc.noaa.gov/crn/qcdatasets.html

1

版权所有：留学生编程辅导网 2018 All Rights Reserved 联系方式：QQ:99515681 电子信箱：99515681@qq.com

免责声明：本站部分内容从网络整理而来，只供参考！如有版权问题可联系本站删除。