In the time series analysis experiment, we learned how to use ARMA and ARIMA to model stationary and non-stationary time series, and fully understood the stationary test and pure randomness test involved in the time series analysis process. In this article, we will learn about another commonly used time series modeling method, and at the same time learn the use of Prophet, Facebook’s open source time series modeling tool.
- seasonal trend series
- Prophet tool introduction
- Prophet Tools Quick Start
- trend change point
- multiplicative model
seasonal trend series
Earlier we learned about modeling stationary and non-stationary time series using ARMA and ARIMA. In practice, the sequences we deal with exhibit some degree of randomness. Of course, this randomness doesn’t get to the point where it’s not worth analyzing. In the forecast analysis of time series, there is also a very common series, which show very significant seasonal characteristics.
For example, a dataset of surface air temperatures over the years (2000-2016) for the City of London is given below. You will find that the change of temperature will show obvious regularity with time.
import pandas as pd from matplotlib import pyplot as plt % matplotlib inline london = pd . read_csv ( "http://labfile.oss.aliyuncs.com/courses/1176/surface-temperature-observations.csv" , index_col = 0 ) london . index = pd . to_datetime ( london . index ) # convert time index print ( "Amount of data: " , len ( london )) london.plot ( figsize = ( 10 , 5 ) ) # Plotting
Introduction to factorization
Then, for such sequence data, we often use a method called factor decomposition to analyze. The factorization method was first proposed by the British statistician WMPersons in 1919. To put it simply, we believe that although the fluctuations of time series are various, they can all be summarized as being affected by the following four types of factors:
- Long-term trend Trend: The influence of the $T_t$ factor will cause the sequence star to show an obvious long-term trend (increasing, decreasing, etc.).
- Cyclic fluctuation Circle: The $C_t$ factor will cause the sequence to show repeated cyclical fluctuations from low to high and then from high to low.
- Seasonal change Season: The $S_t$ factor will cause the sequence star to show stable periodic fluctuations related to seasonal changes.
- Stochastic volatility immediate $I_t$: In addition to long-term trends, cyclic fluctuations and seasonal changes, the sequence is also affected by various other factors, and these effects cause the sequence to show certain random fluctuations.
Therefore, when performing time series analysis, we assume that the sequence will be affected by all or part of these four factors, showing different fluctuation characteristics. Two models are thus derived: the additive model and the multiplicative model.
The additive model, as the name implies, is to add and combine the above four types of factors:
x_t = T_t + C_t + S_t + I_t$$
The multiplicative model is also the multiplication and combination of the above four types of factors:
x_t = T_t * C_t * S_t * I_t$$
The idea of factorization has two main advantages:
- Overcome the interference of other factors and simply measure the impact of a certain deterministic factor (season, trend, etc.) on the sequence.
- According to the deterministic characteristics of the sequence, the interaction relationship between various deterministic factors and their comprehensive influence on the sequence are deduced.
Prophet tool introduction
Prophet is a time series analysis tool open sourced by Facebook in 2017, which provides interfaces supporting R and Python languages. The theoretical basis developed by Prophet relies on the additive model in the factorization method, so it is very suitable for the analysis of seasonal time series data. At the same time, Prophet has a built-in missing value and outlier processing mechanism, which can reduce the burden of sequence preprocessing to a certain extent.
Prophet was originally an internal tool of Facebook’s core data analysis team. After being open sourced, it did help more people complete deterministic analysis of seasonal time series. Although the additive model sounds simple, its derivation and implementation are very complicated. Prophet lowers the threshold for use and analysis to a certain extent.
Next, we will use the London city surface air temperature dataset provided above to learn how to use the Prophet tool.
Prophet stipulates that the incoming data must follow a certain data structure, and it can only contain two columns of data, namely: time
ds and value
y . So, we are now processing the
"""This cell can only be executed once, repeated execution needs to restart the kernel """ london = london . reset_index () # reset index london . columns = [ 'ds' , 'y' ] london . head ()
The upper cell can only be executed once, because the operation of resetting the index
reset_index() will have a cumulative effect and an error will be reported if repeated operations are performed.
Next, we need to check the format of
ds . According to Prophet regulations, the recommended format is
YYYY-MM-DD HH:MM:SS . Also,
y column must be numeric. Since
london meets the requirements, there is no need to make any changes.
Prophet provides an API structure similar to scikit-learn. After we use it to create an example of an additive model, we can call
predict methods to complete the training and prediction.
from fbprophet import Prophet import warnings warnings . filterwarnings ( 'ignore' ) # ignore warnings m = Prophet () # create an additive model m .fit ( london ) # training
Next, we create a prediction sequence, here we can use
make_future_dataframe method directly, or we can create a DataFrame of the corresponding format through Pandas.
make_future_dataframe method supports all
freq supported by
pd.date_range method. The frequency of the original data is days
freq='D' , so here create the sequence time index to be predicted for the next 365 days.
future = m . make_future_dataframe ( periods = 365 , freq = 'D' ) # generate forecast sequence future . tail () # Display the last 5 data of the sequence
Prediction is of course calling
predict method. Note that
predict returns a DataFrame with
19 columns. The DataFrame contains seasonality indicator data and corresponding confidence intervals. Here, we only take out
'yhat', 'yhat_lower', 'yhat_upper' 3 columns of data, which represent the predicted value $\hat y$, and the corresponding confidence interval.
forecast = m . predict ( future ) # forecast forecast [[ 'ds' , 'yhat' , 'yhat_lower' , 'yhat_upper' ]] . head () # Keep only the predicted values and corresponding confidence intervals
Finally, we can plot the predicted data.
fig = m .plot ( forecast ) # plot
As shown above, black dots represent true values, blue lines represent predicted values, and blue intervals represent confidence intervals. Since
make_future_dataframe will also include the time of the original sequence when generating the prediction sequence, the original sequence + prediction sequence is actually passed in during prediction. So in the picture above you can see that the confidence interval extends from 2000 to 2018.
In fact, we can also generate forecast sequences and plots by ourselves without using the corresponding methods provided by Prophet. This can ensure that the predicted value and confidence interval are drawn on the premise that the original sequence remains unchanged.
"""This cell can only be executed once, repeated execution needs to restart the kernel """ future_ = future [ len ( london ):] # Get the time index of the forecast sequence that does not contain the original sequence forecast_ = m . predict ( future_ )[[ 'ds' , 'yhat' , 'yhat_lower' , 'yhat_upper' ]] # prediction london . columns = [ 'ds' , 'yhat' ] # Modify the original data column name forecast_ = pd . concat ([ london , forecast_ ], sort = False ) # merge original data and forecast data DataFrame forecast_ = forecast_ . reset_index ( drop = True ) # Reset the index number of the merged DataFrame fig , axes = plt . subplots ( figsize = ( 12 , 7 )) axes . plot ( forecast_ . index , forecast_ [ 'yhat' ]) # Draw the original data graph axes . fill_between ( forecast_ . index , forecast_ [ 'yhat_lower' ], forecast_ [ 'yhat_upper' ], color = 'orange' , alpha =. 3 ) # draw forecast series and confidence intervals
It can be seen that the above guarantees that the original data image remains unchanged, and the predicted value and the confidence interval for it are directly added at the end. It can be seen from the figure that the change trend of the predicted data is similar to that of the original data, continuing the seasonal variation characteristics of the sequence.
trend change point
In the above section, we used the real data set to learn Prophet to quickly build an additive model. In fact, Prophet also provides some other useful methods to help us use in time series forecasting, one of which is to draw trend change points.
Trend change point, that is, the sudden change caused by the time series, for example, from a certain moment, the overall trend of the series suddenly changes from decline to growth. Marking trend change points is a very good way to observe the sequence cycle and change trend. Prophet can automatically detect and mark these change points, we need to use
from fbprophet.plot import add_changepoints_to_plot fig_base = m .plot ( forecast ) # draw forecast results fig = add_changepoints_to_plot ( fig_base . gca (), m , forecast ) # Add change points to the forecast result plot
Please note that the change point here is not a mark when the sequence value changes from increase to decrease or from decrease to increase, but the overall trend of sequence change. As shown in the figure above, the horizontal red line marks the overall change trend of the sequence, and the change point of this trend line is marked as the trend change point.
In the above example of temperature changes in London, an additive model works well because the forecast only needs to add the seasonal trend to the series, similar to replicating the changing trend. However, in some series, the additive model cannot accurately reflect the changing trend, and the seasonal trend may increase exponentially. At this point, the multiplicative model needs to be used.
For example, the following provides a series of changes in passenger numbers for an airline between 1949 and 1960.
air = pd . read_csv ( 'http://labfile.oss.aliyuncs.com/courses/1176/example_air_passengers.csv' ) air.plot ( figsize = ( 10 , 5 ) ) # Plotting
It can be clearly seen that the number of passengers is reflected in the seasonality, showing a trend of increasing year by year. At this point, if we apply the additive model prediction.
m_additive = Prophet () # additive model m_additive . fit ( air ) # training future = m_additive . make_future_dataframe ( 50 , freq = 'MS' ) # generate forecast time series forecast = m_additive . predict ( future ) # forecast fig = m_additive .plot ( forecast ) # drawing
As shown in the figure above, the additive model is stationary on the seasonal cycle variation. Although there is an overall growth trend, the series value forecast in the subsequent cycle obviously cannot accurately reflect the changes in the original data. At this point, we introduce the multiplicative model.
m_multiplicative = Prophet ( seasonality_mode = 'multiplicative' ) # multiplicative model m_multiplicative.fit ( air ) # training _ future = m_multiplicative . make_future_dataframe ( 50 , freq = 'MS' ) # generate forecast time series forecast = m_multiplicative . predict ( future ) # forecast fig = m_multiplicative .plot ( forecast ) # drawing
As shown in the figure above, the multiplicative model can reflect the characteristics that the number of passengers multiplies with the period. Compared with the additive model, the prediction result will definitely be more accurate.
In this article, we have been exposed to series data that exhibits seasonal characteristics, and learned to use the Prophet tool to build additive and multiplicative models. Simply put, if the series does not vary significantly from season to season, then an additive model can be used. Conversely, if the series increases over time, with an increasing or decreasing trend within each seasonal period, a multiplicative model is preferred.
series of articles
- Common data file storage and reading
- Data visualization and preliminary exploration
- Data cleaning for data preprocessing
- Data integration for data preprocessing
- Data conversion for data preprocessing
- Data specification for data preprocessing
- Apriori association rule learning method
- Time series data analysis and processing
- Time Series Data Modeling and Analysis
- Prophet factorization tool practice
This article is transferred from: https://huhuhang.com/post/machine-learning/dm-tutorials/prophet-factor-decomposition-tool-practice
This site is only for collection, and the copyright belongs to the original author.