.. _10min_tut_09_timeseries: {{ header }} .. ipython:: python import pandas as pd import matplotlib.pyplot as plt .. raw:: html
Data used for this tutorial:
How to handle time series data with ease ---------------------------------------- .. _10min_tut_09_timeseries.properties: Using pandas datetime properties ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. raw:: html .. note:: As many data sets do contain datetime information in one of the columns, pandas input function like :func:`pandas.read_csv` and :func:`pandas.read_json` can do the transformation to dates when reading the data using the ``parse_dates`` parameter with a list of the columns to read as Timestamp: :: pd.read_csv("../data/air_quality_no2_long.csv", parse_dates=["datetime"]) Why are these :class:`pandas.Timestamp` objects useful? Let’s illustrate the added value with some example cases. What is the start and end date of the time series data set we are working with? .. ipython:: python air_quality["datetime"].min(), air_quality["datetime"].max() Using :class:`pandas.Timestamp` for datetimes enables us to calculate with date information and make them comparable. Hence, we can use this to get the length of our time series: .. ipython:: python air_quality["datetime"].max() - air_quality["datetime"].min() The result is a :class:`pandas.Timedelta` object, similar to ``datetime.timedelta`` from the standard Python library and defining a time duration. .. raw:: html
To user guide The various time concepts supported by pandas are explained in the user guide section on :ref:`time related concepts `. .. raw:: html
.. raw:: html .. raw:: html
To user guide An overview of the existing date properties is given in the :ref:`time and date components overview table `. More details about the ``dt`` accessor to return datetime like properties are explained in a dedicated section on the :ref:`dt accessor `. .. raw:: html
.. raw:: html .. raw:: html Datetime as index ~~~~~~~~~~~~~~~~~ In the :ref:`tutorial on reshaping <10min_tut_07_reshape>`, :meth:`~pandas.pivot` was introduced to reshape the data table with each of the measurements locations as a separate column: .. ipython:: python no_2 = air_quality.pivot(index="datetime", columns="location", values="value") no_2.head() .. note:: By pivoting the data, the datetime information became the index of the table. In general, setting a column as an index can be achieved by the ``set_index`` function. Working with a datetime index (i.e. ``DatetimeIndex``) provides powerful functionalities. For example, we do not need the ``dt`` accessor to get the time series properties, but have these properties available on the index directly: .. ipython:: python no_2.index.year, no_2.index.weekday Some other advantages are the convenient subsetting of time period or the adapted time scale on plots. Let’s apply this on our data. .. raw:: html .. raw:: html
To user guide More information on the ``DatetimeIndex`` and the slicing by using strings is provided in the section on :ref:`time series indexing `. .. raw:: html
Resample a time series to another frequency ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. raw:: html The :meth:`~Series.resample` method is similar to a groupby operation: - it provides a time-based grouping, by using a string (e.g. ``M``, ``5H``,…) that defines the target frequency - it requires an aggregation function such as ``mean``, ``max``,… .. raw:: html
To user guide An overview of the aliases used to define time series frequencies is given in the :ref:`offset aliases overview table `. .. raw:: html
When defined, the frequency of the time series is provided by the ``freq`` attribute: .. ipython:: python monthly_max.index.freq .. raw:: html .. raw:: html
To user guide More details on the power of time series ``resampling`` is provided in the user guide section on :ref:`resampling `. .. raw:: html
.. raw:: html

REMEMBER

- Valid date strings can be converted to datetime objects using ``to_datetime`` function or as part of read functions. - Datetime objects in pandas support calculations, logical operations and convenient date-related properties using the ``dt`` accessor. - A ``DatetimeIndex`` contains these date-related properties and supports convenient slicing. - ``Resample`` is a powerful method to change the frequency of a time series. .. raw:: html
.. raw:: html
To user guide A full overview on time series is given on the pages on :ref:`time series and date functionality `. .. raw:: html