Skip to main content
`Ctrl`+`K`

# Time series / date functionality

pandas contains extensive capabilities and features for working with time series data for all domains. Using the NumPy `datetime64` and `timedelta64` dtypes, pandas has consolidated a large number of features from other Python libraries like `scikits.timeseries` as well as created a tremendous amount of new functionality for manipulating time series data.

For example, pandas supports:

Parsing time series information from various sources and formats

    In [1]: import datetime

    In [2]: dti = pd.to_datetime(
       ...:     ["1/1/2018", np.datetime64("2018-01-01"), datetime.datetime(2018, 1, 1)]
       ...: )
       ...:

    In [3]: dti
    Out[3]:
    DatetimeIndex(['2018-01-01', '2018-01-01', '2018-01-01'],
                  dtype='datetime64[us]', freq=None)

Generate sequences of fixed-frequency dates and time spans

    In [4]: dti = pd.date_range("2018-01-01", periods=3, freq="h")

    In [5]: dti
    Out[5]:
    DatetimeIndex(['2018-01-01 00:00:00', '2018-01-01 01:00:00',
                   '2018-01-01 02:00:00'],
                  dtype='datetime64[us]', freq='h')

Manipulating and converting date times with timezone information

    In [6]: dti = dti.tz_localize("UTC")

    In [7]: dti
    Out[7]:
    DatetimeIndex(['2018-01-01 00:00:00+00:00', '2018-01-01 01:00:00+00:00',
                   '2018-01-01 02:00:00+00:00'],
                  dtype='datetime64[us, UTC]', freq='h')

    In [8]: dti.tz_convert("US/Pacific")
    Out[8]:
    DatetimeIndex(['2017-12-31 16:00:00-08:00', '2017-12-31 17:00:00-08:00',
                   '2017-12-31 18:00:00-08:00'],
                  dtype='datetime64[us, US/Pacific]', freq='h')

Resampling or converting a time series to a particular frequency

    In [9]: idx = pd.date_range("2018-01-01", periods=5, freq="h")

    In [10]: ts = pd.Series(range(len(idx)), index=idx)

    In [11]: ts
    Out[11]:
    2018-01-01 00:00:00    0
    2018-01-01 01:00:00    1
    2018-01-01 02:00:00    2
    2018-01-01 03:00:00    3
    2018-01-01 04:00:00    4
    Freq: h, dtype: int64

    In [12]: ts.resample("2h").mean()
    Out[12]:
    2018-01-01 00:00:00    0.5
    2018-01-01 02:00:00    2.5
    2018-01-01 04:00:00    4.0
    Freq: 2h, dtype: float64

Performing date and time arithmetic with absolute or relative time increments

    In [13]: friday = pd.Timestamp("2018-01-05")

    In [14]: friday.day_name()
    Out[14]: 'Friday'

    # Add 1 day
    In [15]: saturday = friday + pd.Timedelta("1 day")

    In [16]: saturday.day_name()
    Out[16]: 'Saturday'

    # Add 1 business day (Friday --> Monday)
    In [17]: monday = friday + pd.offsets.BDay()

    In [18]: monday.day_name()
    Out[18]: 'Monday'

pandas provides a relatively compact and self-contained set of tools for performing the above tasks and more.

## Overview

pandas captures 4 general time related concepts:

1. Date times: A specific date and time with timezone support. Similar to `datetime.datetime` from the standard library.

2. Time deltas: An absolute time duration. Similar to `datetime.timedelta` from the standard library.

3. Time spans: A span of time defined by a point in time and its associated frequency.

4. Date offsets: A relative time duration that respects calendar arithmetic. Similar to `dateutil.relativedelta.relativedelta` from the `dateutil` package.

Concept | Scalar Class | Array Class | pandas Data Type | Primary Creation Method
--- | --- | --- | --- | ---
Date times | `Timestamp` | `DatetimeIndex` | `datetime64[us]` or `datetime64[us, tz]` | `to_datetime` or `date_range`
Time deltas | `Timedelta` | `TimedeltaIndex` | `timedelta64[us]` | `to_timedelta` or `timedelta_range`
Time spans | `Period` | `PeriodIndex` | `period[freq]` | `Period` or `period_range`
Date offsets | `DateOffset` | `None` | `None` | `DateOffset`

The default resolution for date times and time deltas is microsecond (“us”), though second (“s”), millisecond (“ms”), and nanosecond (“ns”) are also supported. The resolution can be changed using `as_unit()`.

For time series data, it’s conventional to represent the time component in the index of a `Series` or `DataFrame` so manipulations can be performed with respect to the time element.

    In [19]: pd.Series(range(3), index=pd.date_range("2000", freq="D", periods=3))
    Out[19]:
    2000-01-01    0
    2000-01-02    1
    2000-01-03    2
    Freq: D, dtype: int64

However, `Series` and `DataFrame` can directly also support the time component as data itself.

    In [20]: pd.Series(pd.date_range("2000", freq="D", periods=3))
    Out[20]:
    0   2000-01-01
    1   2000-01-02
    2   2000-01-03
    dtype: datetime64[us]

`Series` and `DataFrame` have extended data type support and functionality for `datetime`, `timedelta` and `Period` data when passed into those constructors. `DateOffset` data however will be stored as `object` data.

    In [21]: pd.Series(pd.period_range("1/1/2011", freq="M", periods=3))
    Out[21]:
    0    2011-01
    1    2011-02
    2    2011-03
    dtype: period[M]

    In [22]: pd.Series([pd.DateOffset(1), pd.DateOffset(2)])
    Out[22]:
    0         <DateOffset>
    1    <2 * DateOffsets>
    dtype: object

    In [23]: pd.Series(pd.date_range("1/1/2011", freq="ME", periods=3))
    Out[23]:
    0   2011-01-31
    1   2011-02-28
    2   2011-03-31
    dtype: datetime64[us]

Lastly, pandas represents null date times, time deltas, and time spans as `NaT` which is useful for representing missing or null date like values and behaves similar as `np.nan` does for float data.

    In [24]: pd.Timestamp(pd.NaT)
    Out[24]: NaT

    In [25]: pd.Timedelta(pd.NaT)
    Out[25]: NaT

    In [26]: pd.Period(pd.NaT)
    Out[26]: NaT

    # Equality acts as np.nan would
    In [27]: pd.NaT == pd.NaT
    Out[27]: False

## Timestamps vs. time spans

Timestamped data is the most basic type of time series data that associates values with points in time. For pandas objects it means using the points in time.

    In [28]: import datetime

    In [29]: pd.Timestamp(datetime.datetime(2012, 5, 1))
    Out[29]: Timestamp('2012-05-01 00:00:00')

    In [30]: pd.Timestamp("2012-05-01")
    Out[30]: Timestamp('2012-05-01 00:00:00')

    In [31]: pd.Timestamp(2012, 5, 1)
    Out[31]: Timestamp('2012-05-01 00:00:00')

However, in many cases it is more natural to associate things like change variables with a time span instead. The span represented by `Period` can be specified explicitly, or inferred from datetime string format.

For example:

    In [32]: pd.Period("2011-01")
    Out[32]: Period('2011-01', 'M')

    In [33]: pd.Period("2012-05", freq="D")
    Out[33]: Period('2012-05-01', 'D')

`Timestamp` and `Period` can serve as an index. Lists of `Timestamp` and `Period` are automatically coerced to `DatetimeIndex` and `PeriodIndex` respectively.

    In [34]: dates = [
       ....:     pd.Timestamp("2012-05-01"),
       ....:     pd.Timestamp("2012-05-02"),
       ....:     pd.Timestamp("2012-05-03"),
       ....: ]
       ....:

    In [35]: ts = pd.Series(np.random.randn(3), dates)

    In [36]: type(ts.index)
    Out[36]: pandas.DatetimeIndex

    In [37]: ts.index
    Out[37]:
    DatetimeIndex(['2012-05-01', '2012-05-02', '2012-05-03'],
                  dtype='datetime64[us]', freq=None)

    In [38]: ts
    Out[38]:
    2012-05-01    0.469112
    2012-05-02   -0.282863
    2012-05-03   -1.509059
    dtype: float64

    In [39]: periods = [pd.Period("2012-01"), pd.Period("2012-02"), pd.Period("2012-03")]

    In [40]: ts = pd.Series(np.random.randn(3), periods)

    In [41]: type(ts.index)
    Out[41]: pandas.PeriodIndex

    In [42]: ts.index
    Out[42]: PeriodIndex(['2012-01', '2012-02', '2012-03'], dtype='period[M]')

    In [43]: ts
    Out[43]:
    2012-01   -1.135632
    2012-02    1.212112
    2012-03   -0.173215
    Freq: M, dtype: float64

pandas allows you to capture both representations and convert between them. Under the hood, pandas represents timestamps using instances of `Timestamp` and sequences of timestamps using instances of `DatetimeIndex`. For regular time spans, pandas uses `Period` objects for scalar values and `PeriodIndex` for sequences of spans.

## Converting to timestamps

To convert a `Series` or list-like object of date-like objects e.g. strings, epochs, or a mixture, you can use the `to_datetime` function. When passed a `Series`, this returns a `Series` (with the same index), while a list-like is converted to a `DatetimeIndex`:

    In [44]: pd.to_datetime(pd.Series(["Jul 31, 2009", "Jan 10, 2010", None]))
    Out[44]:
    0   2009-07-31
    1   2010-01-10
    2          NaT
    dtype: datetime64[us]

    In [45]: pd.to_datetime(["2005/11/23", "2010/12/31"])
    Out[45]: DatetimeIndex(['2005-11-23', '2010-12-31'], dtype='datetime64[us]', freq=None)

If you use dates which start with the day first (i.e. European style), you can pass the `dayfirst` flag:

    In [46]: pd.to_datetime(["04-01-2012 10:00"], dayfirst=True)
    Out[46]: DatetimeIndex(['2012-01-04 10:00:00'], dtype='datetime64[us]', freq=None)

    In [47]: pd.to_datetime(["04-14-2012 10:00"], dayfirst=True)
    Out[47]: DatetimeIndex(['2012-04-14 10:00:00'], dtype='datetime64[us]', freq=None)

Warning

You see in the above example that `dayfirst` isn’t strict. If a date can’t be parsed with the day being first it will be parsed as if `dayfirst` were `False` and a warning will also be raised.

If you pass a single string to `to_datetime`, it returns a single `Timestamp`. `Timestamp` can also accept string input, but it doesn’t accept string parsing options like `dayfirst` or `format`, so use `to_datetime` if these are required.

    In [48]: pd.to_datetime("2010/11/12")
    Out[48]: Timestamp('2010-11-12 00:00:00')

    In [49]: pd.Timestamp("2010/11/12")
    Out[49]: Timestamp('2010-11-12 00:00:00')

You can also use the `DatetimeIndex` constructor directly:

    In [50]: pd.DatetimeIndex(["2018-01-01", "2018-01-03", "2018-01-05"])
    Out[50]:
    DatetimeIndex(['2018-01-01', '2018-01-03', '2018-01-05'],
                  dtype='datetime64[us]', freq=None)

The string ‘infer’ can be passed in order to set the frequency of the index as the inferred frequency upon creation:

    In [51]: pd.DatetimeIndex(["2018-01-01", "2018-01-03", "2018-01-05"], freq="infer")
    Out[51]:
    DatetimeIndex(['2018-01-01', '2018-01-03', '2018-01-05'],
                  dtype='datetime64[us]', freq='2D')

In most cases, parsing strings to datetimes (with any of `to_datetime()`, `DatetimeIndex`, or `Timestamp`) will produce objects with microsecond (“us”) unit. The exception to this rule is if your strings have nanosecond precision, in which case the result will have “ns” unit:

    In [52]: pd.to_datetime(["2016-01-01 02:03:04"]).unit
    Out[52]: 'us'

    In [53]: pd.to_datetime(["2016-01-01 02:03:04.123"]).unit
    Out[53]: 'us'

    In [54]: pd.to_datetime(["2016-01-01 02:03:04.123456"]).unit
    Out[54]: 'us'

    In [55]: pd.to_datetime(["2016-01-01 02:03:04.123456789"]).unit
    Out[55]: 'ns'

Changed in version 3.0.0: Previously, `to_datetime()` and `DatetimeIndex` would always parse strings to “ns” unit. During pandas 2.x, `Timestamp` could give any of “s”, “ms”, “us”, or “ns” depending on the specificity of the input string.

### Providing a format argument

In addition to the required datetime string, a `format` argument can be passed to ensure specific parsing. This could also potentially speed up the conversion considerably.

    In [56]: pd.to_datetime("2010/11/12", format="%Y/%m/%d")
    Out[56]: Timestamp('2010-11-12 00:00:00')

    In [57]: pd.to_datetime("12-11-2010 00:00", format="%d-%m-%Y %H:%M")
    Out[57]: Timestamp('2010-11-12 00:00:00')

For more information on the choices available when specifying the `format` option, see the Python datetime documentation.

### Assembling datetime from multiple DataFrame columns

You can also pass a `DataFrame` of integer or string columns to assemble into a `Series` of `Timestamps`.

    In [58]: df = pd.DataFrame(
       ....:     {"year": [2015, 2016], "month": [2, 3], "day": [4, 5], "hour": [2, 3]}
       ....: )
       ....:

    In [59]: pd.to_datetime(df)
    Out[59]:
    0   2015-02-04 02:00:00
    1   2016-03-05 03:00:00
    dtype: datetime64[us]

You can pass only the columns that you need to assemble.

    In [60]: pd.to_datetime(df[["year", "month", "day"]])
    Out[60]:
    0   2015-02-04
    1   2016-03-05
    dtype: datetime64[us]

`pd.to_datetime` looks for standard designations of the datetime component in the column names, including:

- required: `year`, `month`, `day`

- optional: `hour`, `minute`, `second`, `millisecond`, `microsecond`, `nanosecond`

### Invalid data

The default behavior, `errors='raise'`, is to raise when unparsable:

    In [61]: pd.to_datetime(['2009/07/31', 'asd'], errors='raise')
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    Cell In[61], line 1
    ----> 1 pd.to_datetime(['2009/07/31', 'asd'], errors='raise')

    File ~/work/pandas/pandas/pandas/core/tools/datetimes.py:1247, in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact, unit, origin, cache)
       1245         result = _convert_and_box_cache(argc, cache_array)
       1246     else:
    -> 1247         result = convert_listlike(argc, format)
       1248 else:
       1249     result = convert_listlike(np.array([arg]), format)[0]

    File ~/work/pandas/pandas/pandas/core/tools/datetimes.py:472, in _convert_listlike_datetimes(arg, format, name, utc, unit, errors, dayfirst, yearfirst, exact)
        470 # `format` could be inferred, or user didn't ask for mixed-format parsing.
        471 if format is not None and format != "mixed":
    --> 472     return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
        474 result, tz_parsed = objects_to_datetime64(
        475     arg,
        476     dayfirst=dayfirst,
       (...)    480     allow_object=True,
        481 )
        483 if tz_parsed is not None:
        484     # We can take a shortcut since the datetime64 numpy array
        485     # is in UTC

    File ~/work/pandas/pandas/pandas/core/tools/datetimes.py:507, in _array_strptime_with_fallback(arg, name, utc, fmt, exact, errors)
        496 def _array_strptime_with_fallback(
        497     arg,
        498     name,
       (...)    502     errors: str,
        503 ) -> Index:
        504     """
        505     Call array_strptime, with fallback behavior depending on 'errors'.
        506     """
    --> 507     result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
        508     if tz_out is not None:
        509         unit = np.datetime_data(result.dtype)[0]

    File pandas/_libs/tslibs/strptime.pyx:648, in pandas._libs.tslibs.strptime.array_strptime()
    --> 648 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/strptime.pyx:566, in pandas._libs.tslibs.strptime.array_strptime()
    --> 566 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/strptime.pyx:706, in pandas._libs.tslibs.strptime._parse_with_format()
    --> 706 'Could not get source, probably due dynamically evaluated source code.'
    ValueError: time data "asd" doesn't match format "%Y/%m/%d". You might want to try:
        - passing `format` if your strings have a consistent format;
        - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
        - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.

Pass `errors='coerce'` to convert unparsable data to `NaT` (not a time):

    In [62]: pd.to_datetime(["2009/07/31", "asd"], errors="coerce")
    Out[62]: DatetimeIndex(['2009-07-31', 'NaT'], dtype='datetime64[us]', freq=None)

### Epoch timestamps

pandas supports converting integer or float epoch times to `Timestamp` and `DatetimeIndex`. The default unit is nanoseconds when no `unit` is specified. However, epochs are often stored in another `unit` which can be specified. These are computed from the starting point specified by the `origin` parameter.

    In [63]: pd.to_datetime(
       ....:     [1349720105, 1349806505, 1349892905, 1349979305, 1350065705], unit="s"
       ....: )
       ....:
    Out[63]:
    DatetimeIndex(['2012-10-08 18:15:05', '2012-10-09 18:15:05',
                   '2012-10-10 18:15:05', '2012-10-11 18:15:05',
                   '2012-10-12 18:15:05'],
                  dtype='datetime64[s]', freq=None)

    In [64]: pd.to_datetime(
       ....:     [1349720105100, 1349720105200, 1349720105300, 1349720105400, 1349720105500],
       ....:     unit="ms",
       ....: )
       ....:
    Out[64]:
    DatetimeIndex(['2012-10-08 18:15:05.100000', '2012-10-08 18:15:05.200000',
                   '2012-10-08 18:15:05.300000', '2012-10-08 18:15:05.400000',
                   '2012-10-08 18:15:05.500000'],
                  dtype='datetime64[ms]', freq=None)

Note

The `unit` parameter does not use the same strings as the `format` parameter that was discussed above. The available units are listed on the documentation for `pandas.to_datetime()`.

Constructing a `Timestamp` or `DatetimeIndex` with an epoch timestamp with the `tz` argument specified will raise a ValueError. If you have epochs in wall time in another timezone, you can read the epochs as timezone-naive timestamps and then localize to the appropriate timezone:

    In [65]: pd.Timestamp(1262347200000000000).tz_localize("US/Pacific")
    Out[65]: Timestamp('2010-01-01 12:00:00-0800', tz='US/Pacific')

    In [66]: pd.DatetimeIndex([1262347200000000000]).tz_localize("US/Pacific")
    Out[66]:
    DatetimeIndex(['2010-01-01 12:00:00-08:00'], dtype='datetime64[ns, US/Pacific]',
                  freq=None)

Note

Epoch times will be rounded to the nearest nanosecond.

Warning

Conversion of float epoch times can lead to inaccurate and unexpected results. Python floats have about 15 digits precision in decimal. Rounding during conversion from float to high precision `Timestamp` is unavoidable. The only way to achieve exact precision is to use a fixed-width types (e.g. an int64).

    In [67]: pd.to_datetime([1490195805.433, 1490195805.433502912], unit="s")
    Out[67]:
    DatetimeIndex(['2017-03-22 15:16:45.433000088', '2017-03-22 15:16:45.433502913'],
                  dtype='datetime64[ns]', freq=None)

    In [68]: pd.to_datetime(1490195805433502912, unit="ns")
    Out[68]: Timestamp('2017-03-22 15:16:45.433502912')

See also

Using the origin parameter

### From timestamps to epoch

To invert the operation from above, namely, to convert from a `Timestamp` to a ‘unix’ epoch:

    In [69]: stamps = pd.date_range("2012-10-08 18:15:05", periods=4, freq="D")

    In [70]: stamps
    Out[70]:
    DatetimeIndex(['2012-10-08 18:15:05', '2012-10-09 18:15:05',
                   '2012-10-10 18:15:05', '2012-10-11 18:15:05'],
                  dtype='datetime64[us]', freq='D')

We subtract the epoch (midnight at January 1, 1970 UTC) and then floor divide by the “unit” (1 second).

    In [71]: (stamps - pd.Timestamp("1970-01-01")) // pd.Timedelta("1s")
    Out[71]: Index([1349720105, 1349806505, 1349892905, 1349979305], dtype='int64')

Note

You may also see `Timestamp.timestamp()` used for this conversion. The explicit subtraction above is recommended because it is unambiguous about treating a timezone-naive value as UTC: `Timestamp.timestamp()` treats a naive `Timestamp` as UTC, which does not match the standard library `datetime.datetime.timestamp`, where a naive value is interpreted as local time.

Another common way to perform this conversion is to convert directly to an integer dtype. Note that the exact integers this produces will depend on the specific unit or resolution of the datetime64 dtype:

    In [72]: stamps.astype(np.int64)
    Out[72]:
    Index([1349720105000000, 1349806505000000, 1349892905000000, 1349979305000000],
          dtype='int64')

    In [73]: stamps.as_unit("s").astype(np.int64)
    Out[73]: Index([1349720105, 1349806505, 1349892905, 1349979305], dtype='int64')

    In [74]: stamps.as_unit("ns").astype(np.int64)
    Out[74]:
    Index([1349720105000000000, 1349806505000000000, 1349892905000000000,
           1349979305000000000],
          dtype='int64')

### Using the `origin` parameter

Using the `origin` parameter, one can specify an alternative starting point for creation of a `DatetimeIndex`. For example, to use 1960-01-01 as the starting date:

    In [75]: pd.to_datetime([1, 2, 3], unit="D", origin=pd.Timestamp("1960-01-01"))
    Out[75]:
    DatetimeIndex(['1960-01-02', '1960-01-03', '1960-01-04'],
                  dtype='datetime64[us]', freq=None)

The default is set at `origin='unix'`, which defaults to `1970-01-01 00:00:00`. Commonly called ‘unix epoch’ or POSIX time.

    In [76]: pd.to_datetime([1, 2, 3], unit="D")
    Out[76]:
    DatetimeIndex(['1970-01-02', '1970-01-03', '1970-01-04'], dtype='datetime64[s]',
                  freq=None)

## Generating ranges of timestamps

To generate an index with timestamps, you can use either the `DatetimeIndex` or `Index` constructor and pass in a list of datetime objects:

    In [77]: dates = [
       ....:     datetime.datetime(2012, 5, 1),
       ....:     datetime.datetime(2012, 5, 2),
       ....:     datetime.datetime(2012, 5, 3),
       ....: ]
       ....:

    # Note the frequency information
    In [78]: index = pd.DatetimeIndex(dates)

    In [79]: index
    Out[79]:
    DatetimeIndex(['2012-05-01', '2012-05-02', '2012-05-03'],
                  dtype='datetime64[us]', freq=None)

    # Automatically converted to DatetimeIndex
    In [80]: index = pd.Index(dates)

    In [81]: index
    Out[81]:
    DatetimeIndex(['2012-05-01', '2012-05-02', '2012-05-03'],
                  dtype='datetime64[us]', freq=None)

In practice this becomes very cumbersome because we often need a very long index with a large number of timestamps. If we need timestamps on a regular frequency, we can use the `date_range()` and `bdate_range()` functions to create a `DatetimeIndex`. The default frequency for `date_range` is a calendar day while the default for `bdate_range` is a business day:

    In [82]: start = datetime.datetime(2011, 1, 1)

    In [83]: end = datetime.datetime(2012, 1, 1)

    In [84]: index = pd.date_range(start, end)

    In [85]: index
    Out[85]:
    DatetimeIndex(['2011-01-01', '2011-01-02', '2011-01-03', '2011-01-04',
                   '2011-01-05', '2011-01-06', '2011-01-07', '2011-01-08',
                   '2011-01-09', '2011-01-10',
                   ...
                   '2011-12-23', '2011-12-24', '2011-12-25', '2011-12-26',
                   '2011-12-27', '2011-12-28', '2011-12-29', '2011-12-30',
                   '2011-12-31', '2012-01-01'],
                  dtype='datetime64[us]', length=366, freq='D')

    In [86]: index = pd.bdate_range(start, end)

    In [87]: index
    Out[87]:
    DatetimeIndex(['2011-01-03', '2011-01-04', '2011-01-05', '2011-01-06',
                   '2011-01-07', '2011-01-10', '2011-01-11', '2011-01-12',
                   '2011-01-13', '2011-01-14',
                   ...
                   '2011-12-19', '2011-12-20', '2011-12-21', '2011-12-22',
                   '2011-12-23', '2011-12-26', '2011-12-27', '2011-12-28',
                   '2011-12-29', '2011-12-30'],
                  dtype='datetime64[us]', length=260, freq='B')

Convenience functions like `date_range` and `bdate_range` can utilize a variety of frequency aliases:

    In [88]: pd.date_range(start, periods=1000, freq="ME")
    Out[88]:
    DatetimeIndex(['2011-01-31', '2011-02-28', '2011-03-31', '2011-04-30',
                   '2011-05-31', '2011-06-30', '2011-07-31', '2011-08-31',
                   '2011-09-30', '2011-10-31',
                   ...
                   '2093-07-31', '2093-08-31', '2093-09-30', '2093-10-31',
                   '2093-11-30', '2093-12-31', '2094-01-31', '2094-02-28',
                   '2094-03-31', '2094-04-30'],
                  dtype='datetime64[us]', length=1000, freq='ME')

    In [89]: pd.bdate_range(start, periods=250, freq="BQS")
    Out[89]:
    DatetimeIndex(['2011-01-03', '2011-04-01', '2011-07-01', '2011-10-03',
                   '2012-01-02', '2012-04-02', '2012-07-02', '2012-10-01',
                   '2013-01-01', '2013-04-01',
                   ...
                   '2071-01-01', '2071-04-01', '2071-07-01', '2071-10-01',
                   '2072-01-01', '2072-04-01', '2072-07-01', '2072-10-03',
                   '2073-01-02', '2073-04-03'],
                  dtype='datetime64[us]', length=250, freq='BQS-JAN')

`date_range` and `bdate_range` make it easy to generate a range of dates using various combinations of parameters like `start`, `end`, `periods`, and `freq`. The start and end dates are strictly inclusive, so dates outside of those specified will not be generated:

    In [90]: pd.date_range(start, end, freq="BME")
    Out[90]:
    DatetimeIndex(['2011-01-31', '2011-02-28', '2011-03-31', '2011-04-29',
                   '2011-05-31', '2011-06-30', '2011-07-29', '2011-08-31',
                   '2011-09-30', '2011-10-31', '2011-11-30', '2011-12-30'],
                  dtype='datetime64[us]', freq='BME')

    In [91]: pd.date_range(start, end, freq="W")
    Out[91]:
    DatetimeIndex(['2011-01-02', '2011-01-09', '2011-01-16', '2011-01-23',
                   '2011-01-30', '2011-02-06', '2011-02-13', '2011-02-20',
                   '2011-02-27', '2011-03-06', '2011-03-13', '2011-03-20',
                   '2011-03-27', '2011-04-03', '2011-04-10', '2011-04-17',
                   '2011-04-24', '2011-05-01', '2011-05-08', '2011-05-15',
                   '2011-05-22', '2011-05-29', '2011-06-05', '2011-06-12',
                   '2011-06-19', '2011-06-26', '2011-07-03', '2011-07-10',
                   '2011-07-17', '2011-07-24', '2011-07-31', '2011-08-07',
                   '2011-08-14', '2011-08-21', '2011-08-28', '2011-09-04',
                   '2011-09-11', '2011-09-18', '2011-09-25', '2011-10-02',
                   '2011-10-09', '2011-10-16', '2011-10-23', '2011-10-30',
                   '2011-11-06', '2011-11-13', '2011-11-20', '2011-11-27',
                   '2011-12-04', '2011-12-11', '2011-12-18', '2011-12-25',
                   '2012-01-01'],
                  dtype='datetime64[us]', freq='W-SUN')

    In [92]: pd.bdate_range(end=end, periods=20)
    Out[92]:
    DatetimeIndex(['2011-12-05', '2011-12-06', '2011-12-07', '2011-12-08',
                   '2011-12-09', '2011-12-12', '2011-12-13', '2011-12-14',
                   '2011-12-15', '2011-12-16', '2011-12-19', '2011-12-20',
                   '2011-12-21', '2011-12-22', '2011-12-23', '2011-12-26',
                   '2011-12-27', '2011-12-28', '2011-12-29', '2011-12-30'],
                  dtype='datetime64[us]', freq='B')

    In [93]: pd.bdate_range(start=start, periods=20)
    Out[93]:
    DatetimeIndex(['2011-01-03', '2011-01-04', '2011-01-05', '2011-01-06',
                   '2011-01-07', '2011-01-10', '2011-01-11', '2011-01-12',
                   '2011-01-13', '2011-01-14', '2011-01-17', '2011-01-18',
                   '2011-01-19', '2011-01-20', '2011-01-21', '2011-01-24',
                   '2011-01-25', '2011-01-26', '2011-01-27', '2011-01-28'],
                  dtype='datetime64[us]', freq='B')

Specifying `start`, `end`, and `periods` will generate a range of evenly spaced dates from `start` to `end` inclusively, with `periods` number of elements in the resulting `DatetimeIndex`:

    In [94]: pd.date_range("2018-01-01", "2018-01-05", periods=5)
    Out[94]:
    DatetimeIndex(['2018-01-01', '2018-01-02', '2018-01-03', '2018-01-04',
                   '2018-01-05'],
                  dtype='datetime64[us]', freq=None)

    In [95]: pd.date_range("2018-01-01", "2018-01-05", periods=10)
    Out[95]:
    DatetimeIndex(['2018-01-01 00:00:00', '2018-01-01 10:40:00',
                   '2018-01-01 21:20:00', '2018-01-02 08:00:00',
                   '2018-01-02 18:40:00', '2018-01-03 05:20:00',
                   '2018-01-03 16:00:00', '2018-01-04 02:40:00',
                   '2018-01-04 13:20:00', '2018-01-05 00:00:00'],
                  dtype='datetime64[us]', freq=None)

### Custom frequency ranges

`bdate_range` can also generate a range of custom frequency dates by using the `weekmask` and `holidays` parameters. These parameters will only be used if a custom frequency string is passed.

    In [96]: weekmask = "Mon Wed Fri"

    In [97]: holidays = [datetime.datetime(2011, 1, 5), datetime.datetime(2011, 3, 14)]

    In [98]: pd.bdate_range(start, end, freq="C", weekmask=weekmask, holidays=holidays)
    Out[98]:
    DatetimeIndex(['2011-01-03', '2011-01-07', '2011-01-10', '2011-01-12',
                   '2011-01-14', '2011-01-17', '2011-01-19', '2011-01-21',
                   '2011-01-24', '2011-01-26',
                   ...
                   '2011-12-09', '2011-12-12', '2011-12-14', '2011-12-16',
                   '2011-12-19', '2011-12-21', '2011-12-23', '2011-12-26',
                   '2011-12-28', '2011-12-30'],
                  dtype='datetime64[us]', length=154, freq='C')

    In [99]: pd.bdate_range(start, end, freq="CBMS", weekmask=weekmask)
    Out[99]:
    DatetimeIndex(['2011-01-03', '2011-02-02', '2011-03-02', '2011-04-01',
                   '2011-05-02', '2011-06-01', '2011-07-01', '2011-08-01',
                   '2011-09-02', '2011-10-03', '2011-11-02', '2011-12-02'],
                  dtype='datetime64[us]', freq='CBMS')

See also

Custom business days

## Timestamp limitations

`Timestamp` uses a 64-bit integer to represent time, so the representable range depends on the chosen resolution (`unit`). For the default nanosecond resolution the span is limited to approximately 584 years; coarser resolutions such as second extend the range to roughly `+/- 2.9e11` years.

The class-level attributes `Timestamp.min`, `Timestamp.max`, and `Timestamp.resolution` default to nanosecond limits:

    In [100]: pd.Timestamp.min
    Out[100]: Timestamp('1677-09-21 00:12:43.145224193')

    In [101]: pd.Timestamp.max
    Out[101]: Timestamp('2262-04-11 23:47:16.854775807')

    In [102]: pd.Timestamp.resolution
    Out[102]: Timedelta('0 days 00:00:00.000000001')

On a `Timestamp` instance, the same attributes reflect the bounds and step size of that instance’s resolution:

    In [103]: ts = pd.Timestamp("2262-04-12").as_unit("s")

    In [104]: ts.min
    Out[104]: Timestamp('-292277022657-01-27 08:29:53')

    In [105]: ts.max
    Out[105]: Timestamp('292277026596-12-04 15:30:07')

    In [106]: ts.resolution
    Out[106]: Timedelta('0 days 00:00:01')

Because `Timestamp` construction automatically picks a resolution wide enough to represent the input, a value outside the nanosecond range (such as `"3000-06-10"`) is parsed using a coarser unit rather than raising `OutOfBoundsDatetime`:

    In [107]: far_future = pd.Timestamp("3000-06-10")

    In [108]: far_future
    Out[108]: Timestamp('3000-06-10 00:00:00')

    In [109]: far_future.unit
    Out[109]: 'us'

Different resolutions can be converted to each other through `Timestamp.as_unit()`.

See also

Representing out-of-bounds spans

## Indexing

One of the main uses for `DatetimeIndex` is as an index for pandas objects. The `DatetimeIndex` class contains many time series related optimizations:

- A large range of dates for various offsets are pre-computed and cached under the hood in order to make generating subsequent date ranges very fast (just have to grab a slice).

- Fast shifting using the `shift` method on pandas objects.

- Unioning of overlapping `DatetimeIndex` objects with the same frequency is very fast (important for fast data alignment).

- Quick access to date fields via properties such as `year`, `month`, etc.

- Regularization functions like `snap` and very fast `asof` logic.

`DatetimeIndex` objects have all the basic functionality of regular `Index` objects, and a smorgasbord of advanced time series specific methods for easy frequency processing.

See also

Reindexing methods

Note

While pandas does not force you to have a sorted date index, some of these methods may have unexpected or incorrect behavior if the dates are unsorted.

`DatetimeIndex` can be used like a regular index and offers all of its intelligent functionality like selection, slicing, etc.

    In [110]: rng = pd.date_range(start, end, freq="BME")

    In [111]: ts = pd.Series(np.random.randn(len(rng)), index=rng)

    In [112]: ts.index
    Out[112]:
    DatetimeIndex(['2011-01-31', '2011-02-28', '2011-03-31', '2011-04-29',
                   '2011-05-31', '2011-06-30', '2011-07-29', '2011-08-31',
                   '2011-09-30', '2011-10-31', '2011-11-30', '2011-12-30'],
                  dtype='datetime64[us]', freq='BME')

    In [113]: ts[:5].index
    Out[113]:
    DatetimeIndex(['2011-01-31', '2011-02-28', '2011-03-31', '2011-04-29',
                   '2011-05-31'],
                  dtype='datetime64[us]', freq='BME')

    In [114]: ts[::2].index
    Out[114]:
    DatetimeIndex(['2011-01-31', '2011-03-31', '2011-05-31', '2011-07-29',
                   '2011-09-30', '2011-11-30'],
                  dtype='datetime64[us]', freq='2BME')

### Partial string indexing

Dates and strings that parse to timestamps can be passed as indexing parameters:

    In [115]: ts["1/31/2011"]
    Out[115]: np.float64(0.11920871129693428)

    In [116]: ts[datetime.datetime(2011, 12, 25):]
    Out[116]:
    2011-12-30    0.56702
    Freq: BME, dtype: float64

    In [117]: ts["10/31/2011":"12/31/2011"]
    Out[117]:
    2011-10-31    0.271860
    2011-11-30   -0.424972
    2011-12-30    0.567020
    Freq: BME, dtype: float64

To provide convenience for accessing longer time series, you can also pass in the year or year and month as strings:

    In [118]: ts["2011"]
    Out[118]:
    2011-01-31    0.119209
    2011-02-28   -1.044236
    2011-03-31   -0.861849
    2011-04-29   -2.104569
    2011-05-31   -0.494929
    2011-06-30    1.071804
    2011-07-29    0.721555
    2011-08-31   -0.706771
    2011-09-30   -1.039575
    2011-10-31    0.271860
    2011-11-30   -0.424972
    2011-12-30    0.567020
    Freq: BME, dtype: float64

    In [119]: ts["2011-6"]
    Out[119]:
    2011-06-30    1.071804
    Freq: BME, dtype: float64

This type of slicing will work on a `DataFrame` with a `DatetimeIndex` as well. Since the partial string selection is a form of label slicing, the endpoints will be included. This would include matching times on an included date:

Note

Indexing `DataFrame` rows with a single string with getitem (e.g. `frame[dtstring]`) is no longer supported. Use `.loc` instead (e.g. `frame.loc[dtstring]`).

    In [120]: dft = pd.DataFrame(
       .....:     np.random.randn(100000, 1),
       .....:     columns=["A"],
       .....:     index=pd.date_range("20130101", periods=100000, freq="min"),
       .....: )
       .....:

    In [121]: dft
    Out[121]:
                                A
    2013-01-01 00:00:00  0.276232
    2013-01-01 00:01:00 -1.087401
    2013-01-01 00:02:00 -0.673690
    2013-01-01 00:03:00  0.113648
    2013-01-01 00:04:00 -1.478427
    ...                       ...
    2013-03-11 10:35:00 -0.747967
    2013-03-11 10:36:00 -0.034523
    2013-03-11 10:37:00 -0.201754
    2013-03-11 10:38:00 -1.509067
    2013-03-11 10:39:00 -1.693043

    [100000 rows x 1 columns]

    In [122]: dft.loc["2013"]
    Out[122]:
                                A
    2013-01-01 00:00:00  0.276232
    2013-01-01 00:01:00 -1.087401
    2013-01-01 00:02:00 -0.673690
    2013-01-01 00:03:00  0.113648
    2013-01-01 00:04:00 -1.478427
    ...                       ...
    2013-03-11 10:35:00 -0.747967
    2013-03-11 10:36:00 -0.034523
    2013-03-11 10:37:00 -0.201754
    2013-03-11 10:38:00 -1.509067
    2013-03-11 10:39:00 -1.693043

    [100000 rows x 1 columns]

This starts on the very first time in the month, and includes the last date and time for the month:

    In [123]: dft["2013-1":"2013-2"]
    Out[123]:
                                A
    2013-01-01 00:00:00  0.276232
    2013-01-01 00:01:00 -1.087401
    2013-01-01 00:02:00 -0.673690
    2013-01-01 00:03:00  0.113648
    2013-01-01 00:04:00 -1.478427
    ...                       ...
    2013-02-28 23:55:00  0.850929
    2013-02-28 23:56:00  0.976712
    2013-02-28 23:57:00 -2.693884
    2013-02-28 23:58:00 -1.575535
    2013-02-28 23:59:00 -1.573517

    [84960 rows x 1 columns]

This specifies a stop time that includes all of the times on the last day:

    In [124]: dft["2013-1":"2013-2-28"]
    Out[124]:
                                A
    2013-01-01 00:00:00  0.276232
    2013-01-01 00:01:00 -1.087401
    2013-01-01 00:02:00 -0.673690
    2013-01-01 00:03:00  0.113648
    2013-01-01 00:04:00 -1.478427
    ...                       ...
    2013-02-28 23:55:00  0.850929
    2013-02-28 23:56:00  0.976712
    2013-02-28 23:57:00 -2.693884
    2013-02-28 23:58:00 -1.575535
    2013-02-28 23:59:00 -1.573517

    [84960 rows x 1 columns]

This specifies an exact stop time (and is not the same as the above):

    In [125]: dft["2013-1":"2013-2-28 00:00:00"]
    Out[125]:
                                A
    2013-01-01 00:00:00  0.276232
    2013-01-01 00:01:00 -1.087401
    2013-01-01 00:02:00 -0.673690
    2013-01-01 00:03:00  0.113648
    2013-01-01 00:04:00 -1.478427
    ...                       ...
    2013-02-27 23:56:00  1.197749
    2013-02-27 23:57:00  0.720521
    2013-02-27 23:58:00 -0.072718
    2013-02-27 23:59:00 -0.681192
    2013-02-28 00:00:00 -0.557501

    [83521 rows x 1 columns]

We are stopping on the included end-point as it is part of the index:

    In [126]: dft["2013-1-15":"2013-1-15 12:30:00"]
    Out[126]:
                                A
    2013-01-15 00:00:00 -0.984810
    2013-01-15 00:01:00  0.941451
    2013-01-15 00:02:00  1.559365
    2013-01-15 00:03:00  1.034374
    2013-01-15 00:04:00 -1.480656
    ...                       ...
    2013-01-15 12:26:00  0.371454
    2013-01-15 12:27:00 -0.930806
    2013-01-15 12:28:00 -0.069177
    2013-01-15 12:29:00  0.066510
    2013-01-15 12:30:00 -0.003945

    [751 rows x 1 columns]

`DatetimeIndex` partial string indexing also works on a `DataFrame` with a `MultiIndex`:

    In [127]: dft2 = pd.DataFrame(
       .....:     np.random.randn(20, 1),
       .....:     columns=["A"],
       .....:     index=pd.MultiIndex.from_product(
       .....:         [pd.date_range("20130101", periods=10, freq="12h"), ["a", "b"]]
       .....:     ),
       .....: )
       .....:

    In [128]: dft2
    Out[128]:
                                  A
    2013-01-01 00:00:00 a -0.298694
                        b  0.823553
    2013-01-01 12:00:00 a  0.943285
                        b -1.479399
    2013-01-02 00:00:00 a -1.643342
    ...                         ...
    2013-01-04 12:00:00 b  0.069036
    2013-01-05 00:00:00 a  0.122297
                        b  1.422060
    2013-01-05 12:00:00 a  0.370079
                        b  1.016331

    [20 rows x 1 columns]

    In [129]: dft2.loc["2013-01-05"]
    Out[129]:
                                  A
    2013-01-05 00:00:00 a  0.122297
                        b  1.422060
    2013-01-05 12:00:00 a  0.370079
                        b  1.016331

    In [130]: idx = pd.IndexSlice

    In [131]: dft2 = dft2.swaplevel(0, 1).sort_index()

    In [132]: dft2.loc[idx[:, "2013-01-05"], :]
    Out[132]:
                                  A
    a 2013-01-05 00:00:00  0.122297
      2013-01-05 12:00:00  0.370079
    b 2013-01-05 00:00:00  1.422060
      2013-01-05 12:00:00  1.016331

Slicing with string indexing also honors UTC offset.

    In [133]: df = pd.DataFrame([0], index=pd.DatetimeIndex(["2019-01-01"], tz="US/Pacific"))

    In [134]: df
    Out[134]:
                               0
    2019-01-01 00:00:00-08:00  0

    In [135]: df["2019-01-01 12:00:00+04:00":"2019-01-01 13:00:00+04:00"]
    Out[135]:
                               0
    2019-01-01 00:00:00-08:00  0

### Slice vs. exact match

The same string used as an indexing parameter can be treated either as a slice or as an exact match depending on the resolution of the index. If the string is less accurate than the index, it will be treated as a slice, otherwise as an exact match.

Consider a `Series` object with a minute resolution index:

    In [136]: series_minute = pd.Series(
       .....:     [1, 2, 3],
       .....:     pd.DatetimeIndex(
       .....:         ["2011-12-31 23:59:00", "2012-01-01 00:00:00", "2012-01-01 00:02:00"]
       .....:     ),
       .....: )
       .....:

    In [137]: series_minute.index.resolution
    Out[137]: 'minute'

A timestamp string less accurate than a minute gives a `Series` object.

    In [138]: series_minute["2011-12-31 23"]
    Out[138]:
    2011-12-31 23:59:00    1
    dtype: int64

A timestamp string with minute resolution (or more accurate), gives a scalar instead, i.e. it is not casted to a slice.

    In [139]: series_minute["2011-12-31 23:59"]
    Out[139]: np.int64(1)

    In [140]: series_minute["2011-12-31 23:59:00"]
    Out[140]: np.int64(1)

If index resolution is second, then the minute-accurate timestamp gives a `Series`.

    In [141]: series_second = pd.Series(
       .....:     [1, 2, 3],
       .....:     pd.DatetimeIndex(
       .....:         ["2011-12-31 23:59:59", "2012-01-01 00:00:00", "2012-01-01 00:00:01"]
       .....:     ),
       .....: )
       .....:

    In [142]: series_second.index.resolution
    Out[142]: 'second'

    In [143]: series_second["2011-12-31 23:59"]
    Out[143]:
    2011-12-31 23:59:59    1
    dtype: int64

If the timestamp string is treated as a slice, it can be used to index `DataFrame` with `.loc[]` as well.

    In [144]: dft_minute = pd.DataFrame(
       .....:     {"a": [1, 2, 3], "b": [4, 5, 6]}, index=series_minute.index
       .....: )
       .....:

    In [145]: dft_minute.loc["2011-12-31 23"]
    Out[145]:
                         a  b
    2011-12-31 23:59:00  1  4

Note

As noted above, indexing `DataFrame` rows with a single string via `[]` is no longer supported; use `.loc` instead.

    In [146]: dft_minute.loc["2011-12-31 23:59"]
    Out[146]:
    a    1
    b    4
    Name: 2011-12-31 23:59:00, dtype: int64

Note also that `DatetimeIndex` resolution cannot be less precise than day.

    In [147]: series_monthly = pd.Series(
       .....:     [1, 2, 3], pd.DatetimeIndex(["2011-12", "2012-01", "2012-02"])
       .....: )
       .....:

    In [148]: series_monthly.index.resolution
    Out[148]: 'day'

    In [149]: series_monthly["2011-12"]  # returns Series
    Out[149]:
    2011-12-01    1
    dtype: int64

### Exact indexing

As discussed in previous section, indexing a `DatetimeIndex` with a partial string depends on the “accuracy” of the period, in other words how specific the interval is in relation to the resolution of the index. In contrast, indexing with `Timestamp` or `datetime` objects is exact, because the objects have exact meaning. These also follow the semantics of including both endpoints.

These `Timestamp` and `datetime` objects have exact `hours, minutes,` and `seconds`, even though they were not explicitly specified (they are `0`).

    In [150]: dft[datetime.datetime(2013, 1, 1): datetime.datetime(2013, 2, 28)]
    Out[150]:
                                A
    2013-01-01 00:00:00  0.276232
    2013-01-01 00:01:00 -1.087401
    2013-01-01 00:02:00 -0.673690
    2013-01-01 00:03:00  0.113648
    2013-01-01 00:04:00 -1.478427
    ...                       ...
    2013-02-27 23:56:00  1.197749
    2013-02-27 23:57:00  0.720521
    2013-02-27 23:58:00 -0.072718
    2013-02-27 23:59:00 -0.681192
    2013-02-28 00:00:00 -0.557501

    [83521 rows x 1 columns]

With no defaults.

    In [151]: dft[
       .....:     datetime.datetime(2013, 1, 1, 10, 12, 0): datetime.datetime(
       .....:         2013, 2, 28, 10, 12, 0
       .....:     )
       .....: ]
       .....:
    Out[151]:
                                A
    2013-01-01 10:12:00  0.565375
    2013-01-01 10:13:00  0.068184
    2013-01-01 10:14:00  0.788871
    2013-01-01 10:15:00 -0.280343
    2013-01-01 10:16:00  0.931536
    ...                       ...
    2013-02-28 10:08:00  0.148098
    2013-02-28 10:09:00 -0.388138
    2013-02-28 10:10:00  0.139348
    2013-02-28 10:11:00  0.085288
    2013-02-28 10:12:00  0.950146

    [83521 rows x 1 columns]

### Non-inclusive slicing

As noted in the sections above, label-based slicing with a `DatetimeIndex` includes both endpoints. This differs from standard Python sequence slicing where the stop endpoint is excluded.

If you need to select data using open or half-open intervals (for example, `start < t < end` or `start <= t < end`), you can use boolean indexing:

    In [152]: ts_example = pd.Series(
       .....:     range(5),
       .....:     index=pd.date_range("2000-01-01", periods=5, freq="D"),
       .....: )
       .....:

    In [153]: ts_example
    Out[153]:
    2000-01-01    0
    2000-01-02    1
    2000-01-03    2
    2000-01-04    3
    2000-01-05    4
    Freq: D, dtype: int64

Select all entries strictly between `"2000-01-02"` and `"2000-01-04"` (excluding both endpoints):

    In [154]: ts_example[(ts_example.index > "2000-01-02") & (ts_example.index < "2000-01-04")]
    Out[154]:
    2000-01-03    2
    Freq: D, dtype: int64

Select entries with a half-open interval (include left, exclude right):

    In [155]: ts_example[(ts_example.index >= "2000-01-02") & (ts_example.index < "2000-01-04")]
    Out[155]:
    2000-01-02    1
    2000-01-03    2
    Freq: D, dtype: int64

For large time series where boolean indexing may be less efficient, you can use `get_slice_bound()` together with `.iloc` to achieve non-inclusive slicing on a sorted index:

    In [156]: lo = ts_example.index.get_slice_bound("2000-01-02", "right")

    In [157]: hi = ts_example.index.get_slice_bound("2000-01-04", "left")

    In [158]: ts_example.iloc[lo:hi]
    Out[158]:
    2000-01-03    2
    Freq: D, dtype: int64

Here, `side="right"` returns the position just past `"2000-01-02"`, and `side="left"` returns the position of `"2000-01-04"` itself. Since `.iloc` uses standard Python half-open slicing, the result excludes both endpoints.

### Selecting a time of day

Like partial string indexing, the methods `at_time()` and `between_time()` operate on the object’s index, selecting rows by their time of day regardless of the date. `at_time()` selects all timestamps at a particular time:

    In [159]: rng_time = pd.date_range("2018-01-01", periods=10, freq="6h")

    In [160]: ts_time = pd.Series(range(10), index=rng_time)

    In [161]: ts_time
    Out[161]:
    2018-01-01 00:00:00    0
    2018-01-01 06:00:00    1
    2018-01-01 12:00:00    2
    2018-01-01 18:00:00    3
    2018-01-02 00:00:00    4
    2018-01-02 06:00:00    5
    2018-01-02 12:00:00    6
    2018-01-02 18:00:00    7
    2018-01-03 00:00:00    8
    2018-01-03 06:00:00    9
    Freq: 6h, dtype: int64

    In [162]: ts_time.at_time("12:00")
    Out[162]:
    2018-01-01 12:00:00    2
    2018-01-02 12:00:00    6
    Freq: 24h, dtype: int64

`between_time()` selects the rows whose time of day falls between two times, including both endpoints by default; the `inclusive` argument controls whether each endpoint is included:

    In [163]: ts_time.between_time("00:00", "12:00")
    Out[163]:
    2018-01-01 00:00:00    0
    2018-01-01 06:00:00    1
    2018-01-01 12:00:00    2
    2018-01-02 00:00:00    4
    2018-01-02 06:00:00    5
    2018-01-02 12:00:00    6
    2018-01-03 00:00:00    8
    2018-01-03 06:00:00    9
    dtype: int64

    In [164]: ts_time.between_time("00:00", "12:00", inclusive="left")
    Out[164]:
    2018-01-01 00:00:00    0
    2018-01-01 06:00:00    1
    2018-01-02 00:00:00    4
    2018-01-02 06:00:00    5
    2018-01-03 00:00:00    8
    2018-01-03 06:00:00    9
    dtype: int64

Both methods also accept `datetime.time` objects:

    In [165]: import datetime

    In [166]: ts_time.at_time(datetime.time(12, 0))
    Out[166]:
    2018-01-01 12:00:00    2
    2018-01-02 12:00:00    6
    Freq: 24h, dtype: int64

### Truncating & fancy indexing

A `truncate()` convenience function is provided that is similar to slicing. Note that `truncate` assumes a 0 value for any unspecified date component in a `DatetimeIndex` in contrast to slicing which returns any partially matching dates:

    In [167]: rng2 = pd.date_range("2011-01-01", "2012-01-01", freq="W")

    In [168]: ts2 = pd.Series(np.random.randn(len(rng2)), index=rng2)

    In [169]: ts2.truncate(before="2011-11", after="2011-12")
    Out[169]:
    2011-11-06    0.437823
    2011-11-13   -0.293083
    2011-11-20   -0.059881
    2011-11-27    1.252450
    Freq: W-SUN, dtype: float64

    In [170]: ts2["2011-11":"2011-12"]
    Out[170]:
    2011-11-06    0.437823
    2011-11-13   -0.293083
    2011-11-20   -0.059881
    2011-11-27    1.252450
    2011-12-04    0.046611
    2011-12-11    0.059478
    2011-12-18   -0.286539
    2011-12-25    0.841669
    Freq: W-SUN, dtype: float64

Even complicated fancy indexing that breaks the `DatetimeIndex` frequency regularity will result in a `DatetimeIndex`, although frequency is lost:

    In [171]: ts2.iloc[[0, 2, 6]].index
    Out[171]:
    DatetimeIndex(['2011-01-02', '2011-01-16', '2011-02-13'],
                  dtype='datetime64[us]', freq=None)

## Time/date components

There are several time/date properties that one can access from `Timestamp` or a collection of timestamps like a `DatetimeIndex`.

Property | Description
--- | ---
year | The year of the datetime
month | The month of the datetime
day | The days of the datetime
hour | The hour of the datetime
minute | The minutes of the datetime
second | The seconds of the datetime
microsecond | The microseconds of the datetime
nanosecond | The nanoseconds of the datetime
date | Returns datetime.date (does not contain timezone information)
time | Returns datetime.time (does not contain timezone information)
timetz | Returns datetime.time as local time with timezone information
dayofyear | The ordinal day of year
day_of_year | The ordinal day of year
dayofweek | The number of the day of the week with Monday=0, Sunday=6
day_of_week | The number of the day of the week with Monday=0, Sunday=6
weekday | The number of the day of the week with Monday=0, Sunday=6
quarter | Quarter of the date: Jan-Mar = 1, Apr-Jun = 2, etc.
days_in_month | The number of days in the month of the datetime
is_month_start | Logical indicating if first day of month (defined by frequency)
is_month_end | Logical indicating if last day of month (defined by frequency)
is_quarter_start | Logical indicating if first day of quarter (defined by frequency)
is_quarter_end | Logical indicating if last day of quarter (defined by frequency)
is_year_start | Logical indicating if first day of year (defined by frequency)
is_year_end | Logical indicating if last day of year (defined by frequency)
is_leap_year | Logical indicating if the date belongs to a leap year

Note

You can use `DatetimeIndex.isocalendar().week` to access week of year date information.

Furthermore, if you have a `Series` with datetimelike values, then you can access these properties via the `.dt` accessor, as detailed in the section on .dt accessors.

You may obtain the year, week and day components of the ISO year from the ISO 8601 standard:

    In [172]: idx = pd.date_range(start="2019-12-29", freq="D", periods=4)

    In [173]: idx.isocalendar()
    Out[173]:
                year  week  day
    2019-12-29  2019    52    7
    2019-12-30  2020     1    1
    2019-12-31  2020     1    2
    2020-01-01  2020     1    3

    In [174]: idx.to_series().dt.isocalendar()
    Out[174]:
                year  week  day
    2019-12-29  2019    52    7
    2019-12-30  2020     1    1
    2019-12-31  2020     1    2
    2020-01-01  2020     1    3

## DateOffset objects

In the preceding examples, frequency strings (e.g. `'D'`) were used to specify a frequency that defined:

- how the date times in `DatetimeIndex` were spaced when using `date_range()`

- the frequency of a `Period` or `PeriodIndex`

These frequency strings map to a `DateOffset` object and its subclasses. A `DateOffset` is similar to a `Timedelta` that represents a duration of time but follows specific calendar duration rules. For example, a `Timedelta` day will always increment `datetimes` by 24 hours, while a `DateOffset` day will increment `datetimes` to the same time the next day whether a day represents 23, 24 or 25 hours due to daylight savings time. However, all `DateOffset` subclasses that are an hour or smaller (`Hour`, `Minute`, `Second`, `Milli`, `Micro`, `Nano`) behave like `Timedelta` and respect absolute time.

The basic `DateOffset` acts similar to `dateutil.relativedelta` (relativedelta documentation) that shifts a date time by the corresponding calendar duration specified. The arithmetic operator (`+`) can be used to perform the shift.

    # This particular day contains a day light savings time transition
    In [175]: ts = pd.Timestamp("2016-10-30 00:00:00", tz="Europe/Helsinki")

    # Respects absolute time
    In [176]: ts + pd.Timedelta(days=1)
    Out[176]: Timestamp('2016-10-30 23:00:00+0200', tz='Europe/Helsinki')

    # Respects calendar time
    In [177]: ts + pd.DateOffset(days=1)
    Out[177]: Timestamp('2016-10-31 00:00:00+0200', tz='Europe/Helsinki')

    In [178]: friday = pd.Timestamp("2018-01-05")

    In [179]: friday.day_name()
    Out[179]: 'Friday'

    # Add 2 business days (Friday --> Tuesday)
    In [180]: two_business_days = 2 * pd.offsets.BDay()

    In [181]: friday + two_business_days
    Out[181]: Timestamp('2018-01-09 00:00:00')

    In [182]: (friday + two_business_days).day_name()
    Out[182]: 'Tuesday'

Most `DateOffsets` have associated frequencies strings, or offset aliases, that can be passed into `freq` keyword arguments. The available date offsets and associated frequency strings can be found below:

Date Offset | Frequency String | Description
--- | --- | ---
`DateOffset` | None | Generic offset class, defaults to absolute 24 hours
`BDay` or `BusinessDay` | `'B'` | business day (weekday)
`CDay` or `CustomBusinessDay` | `'C'` | custom business day
`Week` | `'W'` | one week, optionally anchored on a day of the week (the anchor day is the last day of the weekly period)
`WeekOfMonth` | `'WOM'` | the x-th day of the y-th week of each month
`LastWeekOfMonth` | `'LWOM'` | the x-th day of the last week of each month
`MonthEnd` | `'ME'` | calendar month end
`MonthBegin` | `'MS'` | calendar month begin
`BMonthEnd` or `BusinessMonthEnd` | `'BME'` | business month end
`BMonthBegin` or `BusinessMonthBegin` | `'BMS'` | business month begin
`CBMonthEnd` or `CustomBusinessMonthEnd` | `'CBME'` | custom business month end
`CBMonthBegin` or `CustomBusinessMonthBegin` | `'CBMS'` | custom business month begin
`SemiMonthEnd` | `'SME'` | 15th (or other day_of_month) and calendar month end
`SemiMonthBegin` | `'SMS'` | 15th (or other day_of_month) and calendar month begin
`QuarterEnd` | `'QE'` | calendar quarter end
`QuarterBegin` | `'QS'` | calendar quarter begin
`BQuarterEnd` | `'BQE` | business quarter end
`BQuarterBegin` | `'BQS'` | business quarter begin
`FY5253Quarter` | `'REQ'` | retail (aka 52-53 week) quarter
`HalfYearEnd` | `'HYE'` | calendar half year end
`HalfYearBegin` | `'HYS'` | calendar half year begin
`BHalfYearEnd` | `'BHYE` | business half year end
`BHalfYearBegin` | `'BHYS'` | business half year begin
`YearEnd` | `'YE'` | calendar year end
`YearBegin` | `'YS'` or `'BYS'` | calendar year begin
`BYearEnd` | `'BYE'` | business year end
`BYearBegin` | `'BYS'` | business year begin
`FY5253` | `'RE'` | retail (aka 52-53 week) year
`Easter` | None | Easter holiday
`BusinessHour` | `'bh'` | business hour
`CustomBusinessHour` | `'cbh'` | custom business hour
`Day` | `'D'` | one calendar day
`Hour` | `'h'` | one hour
`Minute` | `'min'` | one minute
`Second` | `'s'` | one second
`Milli` | `'ms'` | one millisecond
`Micro` | `'us'` | one microsecond
`Nano` | `'ns'` | one nanosecond

`DateOffsets` additionally have `rollforward()` and `rollback()` methods for moving a date forward or backward respectively to a valid offset date relative to the offset. For example, business offsets will roll dates that land on the weekends (Saturday and Sunday) forward to Monday since business offsets operate on the weekdays.

    In [183]: ts = pd.Timestamp("2018-01-06 00:00:00")

    In [184]: ts.day_name()
    Out[184]: 'Saturday'

    # BusinessHour's valid offset dates are Monday through Friday
    In [185]: offset = pd.offsets.BusinessHour(start="09:00")

    # Bring the date to the closest offset date (Monday)
    In [186]: offset.rollforward(ts)
    Out[186]: Timestamp('2018-01-08 09:00:00')

    # Date is brought to the closest offset date first and then the hour is added
    In [187]: ts + offset
    Out[187]: Timestamp('2018-01-08 10:00:00')

These operations preserve time (hour, minute, etc) information by default. To reset time to midnight, use `normalize()` before or after applying the operation (depending on whether you want the time information included in the operation).

    In [188]: ts = pd.Timestamp("2014-01-01 09:00")

    In [189]: day = pd.offsets.Day()

    In [190]: day + ts
    Out[190]: Timestamp('2014-01-02 09:00:00')

    In [191]: (day + ts).normalize()
    Out[191]: Timestamp('2014-01-02 00:00:00')

    In [192]: ts = pd.Timestamp("2014-01-01 22:00")

    In [193]: hour = pd.offsets.Hour()

    In [194]: hour + ts
    Out[194]: Timestamp('2014-01-01 23:00:00')

    In [195]: (hour + ts).normalize()
    Out[195]: Timestamp('2014-01-01 00:00:00')

    In [196]: (hour + pd.Timestamp("2014-01-01 23:30")).normalize()
    Out[196]: Timestamp('2014-01-02 00:00:00')

### Parametric offsets

Some of the offsets can be “parameterized” when created to result in different behaviors. For example, the `Week` offset for generating weekly data accepts a `weekday` parameter which results in the generated dates always lying on a particular day of the week:

    In [197]: d = datetime.datetime(2008, 8, 18, 9, 0)

    In [198]: d
    Out[198]: datetime.datetime(2008, 8, 18, 9, 0)

    In [199]: d + pd.offsets.Week()
    Out[199]: Timestamp('2008-08-25 09:00:00')

    In [200]: d + pd.offsets.Week(weekday=4)
    Out[200]: Timestamp('2008-08-22 09:00:00')

    In [201]: (d + pd.offsets.Week(weekday=4)).weekday()
    Out[201]: 4

    In [202]: d - pd.offsets.Week()
    Out[202]: Timestamp('2008-08-11 09:00:00')

The `normalize` option will be effective for addition and subtraction.

    In [203]: d + pd.offsets.Week(normalize=True)
    Out[203]: Timestamp('2008-08-25 00:00:00')

    In [204]: d - pd.offsets.Week(normalize=True)
    Out[204]: Timestamp('2008-08-11 00:00:00')

Another example is parameterizing `YearEnd` with the specific ending month:

    In [205]: d + pd.offsets.YearEnd()
    Out[205]: Timestamp('2008-12-31 09:00:00')

    In [206]: d + pd.offsets.YearEnd(month=6)
    Out[206]: Timestamp('2009-06-30 09:00:00')

### Using offsets with `Series` / `DatetimeIndex`

Offsets can be used with either a `Series` or `DatetimeIndex` to apply the offset to each element.

    In [207]: rng = pd.date_range("2012-01-01", "2012-01-03")

    In [208]: s = pd.Series(rng)

    In [209]: rng
    Out[209]:
    DatetimeIndex(['2012-01-01', '2012-01-02', '2012-01-03'],
                  dtype='datetime64[us]', freq='D')

    In [210]: rng + pd.DateOffset(months=2)
    Out[210]:
    DatetimeIndex(['2012-03-01', '2012-03-02', '2012-03-03'],
                  dtype='datetime64[us]', freq=None)

    In [211]: s + pd.DateOffset(months=2)
    Out[211]:
    0   2012-03-01
    1   2012-03-02
    2   2012-03-03
    dtype: datetime64[us]

    In [212]: s - pd.DateOffset(months=2)
    Out[212]:
    0   2011-11-01
    1   2011-11-02
    2   2011-11-03
    dtype: datetime64[us]

If the offset class maps directly to a `Timedelta` (`Hour`, `Minute`, `Second`, `Micro`, `Milli`, `Nano`) it can be used exactly like a `Timedelta` - see the Timedelta section for more examples.

    In [213]: s - pd.offsets.Day(2)
    Out[213]:
    0   2011-12-30
    1   2011-12-31
    2   2012-01-01
    dtype: datetime64[us]

    In [214]: td = s - pd.Series(pd.date_range("2011-12-29", "2011-12-31"))

    In [215]: td
    Out[215]:
    0   3 days
    1   3 days
    2   3 days
    dtype: timedelta64[us]

    In [216]: td + pd.offsets.Minute(15)
    Out[216]:
    0   3 days 00:15:00
    1   3 days 00:15:00
    2   3 days 00:15:00
    dtype: timedelta64[us]

Note that some offsets (such as `BQuarterEnd`) do not have a vectorized implementation. They can still be used but may calculate significantly slower and will show a `PerformanceWarning`

    In [217]: rng + pd.offsets.BQuarterEnd()
    Out[217]:
    DatetimeIndex(['2012-03-30', '2012-03-30', '2012-03-30'],
                  dtype='datetime64[us]', freq=None)

### Custom business days

The `CDay` or `CustomBusinessDay` class provides a parametric `BusinessDay` class which can be used to create customized business day calendars which account for local holidays and local weekend conventions.

As an interesting example, let’s look at Egypt where a Friday-Saturday weekend is observed.

    In [218]: weekmask_egypt = "Sun Mon Tue Wed Thu"

    # They also observe International Workers' Day so let's
    # add that for a couple of years
    In [219]: holidays = [
       .....:     "2012-05-01",
       .....:     datetime.datetime(2013, 5, 1),
       .....:     np.datetime64("2014-05-01"),
       .....: ]
       .....:

    In [220]: bday_egypt = pd.offsets.CustomBusinessDay(
       .....:     holidays=holidays,
       .....:     weekmask=weekmask_egypt,
       .....: )
       .....:

    In [221]: dt = datetime.datetime(2013, 4, 30)

    In [222]: dt + 2 * bday_egypt
    Out[222]: Timestamp('2013-05-05 00:00:00')

Let’s map to the weekday names:

    In [223]: dts = pd.date_range(dt, periods=5, freq=bday_egypt)

    In [224]: pd.Series(dts.day_of_week, dts).map(pd.Series("Mon Tue Wed Thu Fri Sat Sun".split()))
    Out[224]:
    2013-04-30    Tue
    2013-05-02    Thu
    2013-05-05    Sun
    2013-05-06    Mon
    2013-05-07    Tue
    Freq: C, dtype: str

Holiday calendars can be used to provide the list of holidays. See the holiday calendar section for more information.

    In [225]: from pandas.tseries.holiday import USFederalHolidayCalendar

    In [226]: bday_us = pd.offsets.CustomBusinessDay(calendar=USFederalHolidayCalendar())

    # Friday before MLK Day
    In [227]: dt = datetime.datetime(2014, 1, 17)

    # Tuesday after MLK Day (Monday is skipped because it's a holiday)
    In [228]: dt + bday_us
    Out[228]: Timestamp('2014-01-21 00:00:00')

Monthly offsets that respect a certain holiday calendar can be defined in the usual way.

    In [229]: bmth_us = pd.offsets.CustomBusinessMonthBegin(calendar=USFederalHolidayCalendar())

    # Skip new years
    In [230]: dt = datetime.datetime(2013, 12, 17)

    In [231]: dt + bmth_us
    Out[231]: Timestamp('2014-01-02 00:00:00')

    # Define date index with custom offset
    In [232]: pd.date_range(start="20100101", end="20120101", freq=bmth_us)
    Out[232]:
    DatetimeIndex(['2010-01-04', '2010-02-01', '2010-03-01', '2010-04-01',
                   '2010-05-03', '2010-06-01', '2010-07-01', '2010-08-02',
                   '2010-09-01', '2010-10-01', '2010-11-01', '2010-12-01',
                   '2011-01-03', '2011-02-01', '2011-03-01', '2011-04-01',
                   '2011-05-02', '2011-06-01', '2011-07-01', '2011-08-01',
                   '2011-09-01', '2011-10-03', '2011-11-01', '2011-12-01'],
                  dtype='datetime64[us]', freq='CBMS')

Note

The frequency string ‘C’ is used to indicate that a CustomBusinessDay DateOffset is used, it is important to note that since CustomBusinessDay is a parameterised type, instances of CustomBusinessDay may differ and this is not detectable from the ‘C’ frequency string. The user therefore needs to ensure that the ‘C’ frequency string is used consistently within the user’s application.

### Business hour

The `BusinessHour` class provides a business hour representation on `BusinessDay`, allowing to use specific start and end times.

By default, `BusinessHour` uses 9:00 - 17:00 as business hours. Adding `BusinessHour` will increment `Timestamp` by hourly frequency. If target `Timestamp` is out of business hours, move to the next business hour then increment it. If the result exceeds the business hours end, the remaining hours are added to the next business day.

    In [233]: bh = pd.offsets.BusinessHour()

    In [234]: bh
    Out[234]: <BusinessHour: bh=09:00-17:00>

    # 2014-08-01 is Friday
    In [235]: pd.Timestamp("2014-08-01 10:00").weekday()
    Out[235]: 4

    In [236]: pd.Timestamp("2014-08-01 10:00") + bh
    Out[236]: Timestamp('2014-08-01 11:00:00')

    # Below example is the same as: pd.Timestamp('2014-08-01 09:00') + bh
    In [237]: pd.Timestamp("2014-08-01 08:00") + bh
    Out[237]: Timestamp('2014-08-01 10:00:00')

    # If the results is on the end time, move to the next business day
    In [238]: pd.Timestamp("2014-08-01 16:00") + bh
    Out[238]: Timestamp('2014-08-04 09:00:00')

    # Remainings are added to the next day
    In [239]: pd.Timestamp("2014-08-01 16:30") + bh
    Out[239]: Timestamp('2014-08-04 09:30:00')

    # Adding 2 business hours
    In [240]: pd.Timestamp("2014-08-01 10:00") + pd.offsets.BusinessHour(2)
    Out[240]: Timestamp('2014-08-01 12:00:00')

    # Subtracting 3 business hours
    In [241]: pd.Timestamp("2014-08-01 10:00") + pd.offsets.BusinessHour(-3)
    Out[241]: Timestamp('2014-07-31 15:00:00')

You can also specify `start` and `end` time by keywords. The argument must be a `str` with an `hour:minute` representation or a `datetime.time` instance. Specifying seconds, microseconds and nanoseconds as business hour results in `ValueError`.

    In [242]: bh = pd.offsets.BusinessHour(start="11:00", end=datetime.time(20, 0))

    In [243]: bh
    Out[243]: <BusinessHour: bh=11:00-20:00>

    In [244]: pd.Timestamp("2014-08-01 13:00") + bh
    Out[244]: Timestamp('2014-08-01 14:00:00')

    In [245]: pd.Timestamp("2014-08-01 09:00") + bh
    Out[245]: Timestamp('2014-08-01 12:00:00')

    In [246]: pd.Timestamp("2014-08-01 18:00") + bh
    Out[246]: Timestamp('2014-08-01 19:00:00')

Passing `start` time later than `end` represents midnight business hour. In this case, business hour exceeds midnight and overlap to the next day. Valid business hours are distinguished by whether it started from valid `BusinessDay`.

    In [247]: bh = pd.offsets.BusinessHour(start="17:00", end="09:00")

    In [248]: bh
    Out[248]: <BusinessHour: bh=17:00-09:00>

    In [249]: pd.Timestamp("2014-08-01 17:00") + bh
    Out[249]: Timestamp('2014-08-01 18:00:00')

    In [250]: pd.Timestamp("2014-08-01 23:00") + bh
    Out[250]: Timestamp('2014-08-02 00:00:00')

    # Although 2014-08-02 is Saturday,
    # it is valid because it starts from 08-01 (Friday).
    In [251]: pd.Timestamp("2014-08-02 04:00") + bh
    Out[251]: Timestamp('2014-08-02 05:00:00')

    # Although 2014-08-04 is Monday,
    # it is out of business hours because it starts from 08-03 (Sunday).
    In [252]: pd.Timestamp("2014-08-04 04:00") + bh
    Out[252]: Timestamp('2014-08-04 18:00:00')

Applying `BusinessHour.rollforward` and `rollback` to out of business hours results in the next business hour start or previous day’s end. Different from other offsets, `BusinessHour.rollforward` may output different results from `apply` by definition.

This is because one day’s business hour end is equal to next day’s business hour start. For example, under the default business hours (9:00 - 17:00), there is no gap (0 minutes) between `2014-08-01 17:00` and `2014-08-04 09:00`.

    # This adjusts a Timestamp to business hour edge
    In [253]: pd.offsets.BusinessHour().rollback(pd.Timestamp("2014-08-02 15:00"))
    Out[253]: Timestamp('2014-08-01 17:00:00')

    In [254]: pd.offsets.BusinessHour().rollforward(pd.Timestamp("2014-08-02 15:00"))
    Out[254]: Timestamp('2014-08-04 09:00:00')

    # It is the same as BusinessHour() + pd.Timestamp('2014-08-01 17:00').
    # And it is the same as BusinessHour() + pd.Timestamp('2014-08-04 09:00')
    In [255]: pd.offsets.BusinessHour() + pd.Timestamp("2014-08-02 15:00")
    Out[255]: Timestamp('2014-08-04 10:00:00')

    # BusinessDay results (for reference)
    In [256]: pd.offsets.BusinessHour().rollforward(pd.Timestamp("2014-08-02"))
    Out[256]: Timestamp('2014-08-04 09:00:00')

    # It is the same as BusinessDay() + pd.Timestamp('2014-08-01')
    # The result is the same as rollworward because BusinessDay never overlap.
    In [257]: pd.offsets.BusinessHour() + pd.Timestamp("2014-08-02")
    Out[257]: Timestamp('2014-08-04 10:00:00')

`BusinessHour` regards Saturday and Sunday as holidays. To use arbitrary holidays, you can use `CustomBusinessHour` offset, as explained in the following subsection.

### Custom business hour

The `CustomBusinessHour` is a mixture of `BusinessHour` and `CustomBusinessDay` which allows you to specify arbitrary holidays. `CustomBusinessHour` works as the same as `BusinessHour` except that it skips specified custom holidays.

    In [258]: from pandas.tseries.holiday import USFederalHolidayCalendar

    In [259]: bhour_us = pd.offsets.CustomBusinessHour(calendar=USFederalHolidayCalendar())

    # Friday before MLK Day
    In [260]: dt = datetime.datetime(2014, 1, 17, 15)

    In [261]: dt + bhour_us
    Out[261]: Timestamp('2014-01-17 16:00:00')

    # Tuesday after MLK Day (Monday is skipped because it's a holiday)
    In [262]: dt + bhour_us * 2
    Out[262]: Timestamp('2014-01-21 09:00:00')

You can use keyword arguments supported by either `BusinessHour` and `CustomBusinessDay`.

    In [263]: bhour_mon = pd.offsets.CustomBusinessHour(start="10:00", weekmask="Tue Wed Thu Fri")

    # Monday is skipped because it's a holiday, business hour starts from 10:00
    In [264]: dt + bhour_mon * 2
    Out[264]: Timestamp('2014-01-21 10:00:00')

### Offset aliases

A number of string aliases are given to useful common time series frequencies. We will refer to these aliases as offset aliases.

Alias | Description
--- | ---
B | business day frequency
C | custom business day frequency
D | calendar day frequency
W | weekly frequency (default anchored to Sunday, i.e. W-SUN)
ME | month end frequency
SME | semi-month end frequency (15th and end of month)
BME | business month end frequency
CBME | custom business month end frequency
MS | month start frequency
SMS | semi-month start frequency (1st and 15th)
BMS | business month start frequency
CBMS | custom business month start frequency
QE | quarter end frequency
BQE | business quarter end frequency
QS | quarter start frequency
BQS | business quarter start frequency
YE | year end frequency
BYE | business year end frequency
YS | year start frequency
BYS | business year start frequency
h | hourly frequency
bh | business hour frequency
cbh | custom business hour frequency
min | minutely frequency
s | secondly frequency
ms | milliseconds
us | microseconds
ns | nanoseconds

Note

> When using the offset aliases above, it should be noted that functions such as `date_range()`, `bdate_range()`, will only return timestamps that are in the interval defined by `start_date` and `end_date`. If the `start_date` does not correspond to the frequency, the returned timestamps will start at the next valid timestamp, same for `end_date`, the returned timestamps will stop at the previous valid timestamp.

For example, for the offset `MS`, if the `start_date` is not the first of the month, the returned timestamps will start with the first day of the next month. If `end_date` is not the first day of a month, the last returned timestamp will be the first day of the corresponding month.

    In [265]: dates_lst_1 = pd.date_range("2020-01-06", "2020-04-03", freq="MS")

    In [266]: dates_lst_1
    Out[266]:
    DatetimeIndex(['2020-02-01', '2020-03-01', '2020-04-01'],
                  dtype='datetime64[us]', freq='MS')

    In [267]: dates_lst_2 = pd.date_range("2020-01-01", "2020-04-01", freq="MS")

    In [268]: dates_lst_2
    Out[268]:
    DatetimeIndex(['2020-01-01', '2020-02-01', '2020-03-01', '2020-04-01'],
                  dtype='datetime64[us]', freq='MS')

We can see in the above example `date_range()` and `bdate_range()` will only return the valid timestamps between the `start_date` and `end_date`. If these are not valid timestamps for the given frequency it will roll to the next value for `start_date` (respectively previous for the `end_date`)

### Period aliases

A number of string aliases are given to useful common time series frequencies. We will refer to these aliases as period aliases.

Alias | Description
--- | ---
B | business day frequency
D | calendar day frequency
W | weekly frequency (default anchored to Sunday, i.e. W-SUN)
M | monthly frequency
Q | quarterly frequency
Y | yearly frequency
h | hourly frequency
min | minutely frequency
s | secondly frequency
ms | milliseconds
us | microseconds
ns | nanoseconds

### Combining aliases

As we have seen previously, the alias and the offset instance are fungible in most functions:

    In [269]: pd.date_range(start, periods=5, freq="B")
    Out[269]:
    DatetimeIndex(['2011-01-03', '2011-01-04', '2011-01-05', '2011-01-06',
                   '2011-01-07'],
                  dtype='datetime64[us]', freq='B')

    In [270]: pd.date_range(start, periods=5, freq=pd.offsets.BDay())
    Out[270]:
    DatetimeIndex(['2011-01-03', '2011-01-04', '2011-01-05', '2011-01-06',
                   '2011-01-07'],
                  dtype='datetime64[us]', freq='B')

You can combine together day and intraday offsets:

    In [271]: pd.date_range(start, periods=10, freq="2h20min")
    Out[271]:
    DatetimeIndex(['2011-01-01 00:00:00', '2011-01-01 02:20:00',
                   '2011-01-01 04:40:00', '2011-01-01 07:00:00',
                   '2011-01-01 09:20:00', '2011-01-01 11:40:00',
                   '2011-01-01 14:00:00', '2011-01-01 16:20:00',
                   '2011-01-01 18:40:00', '2011-01-01 21:00:00'],
                  dtype='datetime64[us]', freq='140min')

    In [272]: pd.date_range(start, periods=10, freq="1D10us")
    Out[272]:
    DatetimeIndex([       '2011-01-01 00:00:00', '2011-01-02 00:00:00.000010',
                   '2011-01-03 00:00:00.000020', '2011-01-04 00:00:00.000030',
                   '2011-01-05 00:00:00.000040', '2011-01-06 00:00:00.000050',
                   '2011-01-07 00:00:00.000060', '2011-01-08 00:00:00.000070',
                   '2011-01-09 00:00:00.000080', '2011-01-10 00:00:00.000090'],
                  dtype='datetime64[us]', freq='86400000010us')

### Anchored offsets

For some frequencies you can specify an anchoring suffix:

Alias | Description
--- | ---
W-SUN | weekly frequency, week ends on Sunday. Same as ‘W’
W-MON | weekly frequency, week ends on Monday
W-TUE | weekly frequency, week ends on Tuesday
W-WED | weekly frequency, week ends on Wednesday
W-THU | weekly frequency, week ends on Thursday
W-FRI | weekly frequency, week ends on Friday
W-SAT | weekly frequency, week ends on Saturday
(B)QE-DEC | quarterly frequency, year ends in December. Same as ‘QE’
(B)QE-JAN | quarterly frequency, year ends in January
(B)QE-FEB | quarterly frequency, year ends in February
(B)QE-MAR | quarterly frequency, year ends in March
(B)QE-APR | quarterly frequency, year ends in April
(B)QE-MAY | quarterly frequency, year ends in May
(B)QE-JUN | quarterly frequency, year ends in June
(B)QE-JUL | quarterly frequency, year ends in July
(B)QE-AUG | quarterly frequency, year ends in August
(B)QE-SEP | quarterly frequency, year ends in September
(B)QE-OCT | quarterly frequency, year ends in October
(B)QE-NOV | quarterly frequency, year ends in November
(B)QS-JAN | quarterly frequency, year starts in January. Same as ‘QS’
(B)QS-FEB | quarterly frequency, year starts in February
(B)QS-MAR | quarterly frequency, year starts in March
(B)QS-APR | quarterly frequency, year starts in April
(B)QS-MAY | quarterly frequency, year starts in May
(B)QS-JUN | quarterly frequency, year starts in June
(B)QS-JUL | quarterly frequency, year starts in July
(B)QS-AUG | quarterly frequency, year starts in August
(B)QS-SEP | quarterly frequency, year starts in September
(B)QS-OCT | quarterly frequency, year starts in October
(B)QS-NOV | quarterly frequency, year starts in November
(B)QS-DEC | quarterly frequency, year starts in December
(B)YE-DEC | annual frequency, anchored end of December. Same as ‘YE’
(B)YE-JAN | annual frequency, anchored end of January
(B)YE-FEB | annual frequency, anchored end of February
(B)YE-MAR | annual frequency, anchored end of March
(B)YE-APR | annual frequency, anchored end of April
(B)YE-MAY | annual frequency, anchored end of May
(B)YE-JUN | annual frequency, anchored end of June
(B)YE-JUL | annual frequency, anchored end of July
(B)YE-AUG | annual frequency, anchored end of August
(B)YE-SEP | annual frequency, anchored end of September
(B)YE-OCT | annual frequency, anchored end of October
(B)YE-NOV | annual frequency, anchored end of November
(B)YS-JAN | annual frequency, anchored start of January. Same as ‘YS’
(B)YS-FEB | annual frequency, anchored start of February
(B)YS-MAR | annual frequency, anchored start of March
(B)YS-APR | annual frequency, anchored start of April
(B)YS-MAY | annual frequency, anchored start of May
(B)YS-JUN | annual frequency, anchored start of June
(B)YS-JUL | annual frequency, anchored start of July
(B)YS-AUG | annual frequency, anchored start of August
(B)YS-SEP | annual frequency, anchored start of September
(B)YS-OCT | annual frequency, anchored start of October
(B)YS-NOV | annual frequency, anchored start of November
(B)YS-DEC | annual frequency, anchored start of December

These can be used as arguments to `date_range`, `bdate_range`, constructors for `DatetimeIndex`, as well as various other timeseries-related functions in pandas.

### Anchored offset semantics

For those offsets that are anchored to the start or end of specific frequency (`MonthEnd`, `MonthBegin`, `WeekEnd`, etc), the following rules apply to rolling forward and backwards.

When `n` is not 0, if the given date is not on an anchor point, it snapped to the next(previous) anchor point, and moved `|n|-1` additional steps forwards or backwards.

    In [273]: pd.Timestamp("2014-01-02") + pd.offsets.MonthBegin(n=1)
    Out[273]: Timestamp('2014-02-01 00:00:00')

    In [274]: pd.Timestamp("2014-01-02") + pd.offsets.MonthEnd(n=1)
    Out[274]: Timestamp('2014-01-31 00:00:00')

    In [275]: pd.Timestamp("2014-01-02") - pd.offsets.MonthBegin(n=1)
    Out[275]: Timestamp('2014-01-01 00:00:00')

    In [276]: pd.Timestamp("2014-01-02") - pd.offsets.MonthEnd(n=1)
    Out[276]: Timestamp('2013-12-31 00:00:00')

    In [277]: pd.Timestamp("2014-01-02") + pd.offsets.MonthBegin(n=4)
    Out[277]: Timestamp('2014-05-01 00:00:00')

    In [278]: pd.Timestamp("2014-01-02") - pd.offsets.MonthBegin(n=4)
    Out[278]: Timestamp('2013-10-01 00:00:00')

If the given date is on an anchor point, it is moved `|n|` points forwards or backwards.

    In [279]: pd.Timestamp("2014-01-01") + pd.offsets.MonthBegin(n=1)
    Out[279]: Timestamp('2014-02-01 00:00:00')

    In [280]: pd.Timestamp("2014-01-31") + pd.offsets.MonthEnd(n=1)
    Out[280]: Timestamp('2014-02-28 00:00:00')

    In [281]: pd.Timestamp("2014-01-01") - pd.offsets.MonthBegin(n=1)
    Out[281]: Timestamp('2013-12-01 00:00:00')

    In [282]: pd.Timestamp("2014-01-31") - pd.offsets.MonthEnd(n=1)
    Out[282]: Timestamp('2013-12-31 00:00:00')

    In [283]: pd.Timestamp("2014-01-01") + pd.offsets.MonthBegin(n=4)
    Out[283]: Timestamp('2014-05-01 00:00:00')

    In [284]: pd.Timestamp("2014-01-31") - pd.offsets.MonthBegin(n=4)
    Out[284]: Timestamp('2013-10-01 00:00:00')

For the case when `n=0`, the date is not moved if on an anchor point, otherwise it is rolled forward to the next anchor point.

    In [285]: pd.Timestamp("2014-01-02") + pd.offsets.MonthBegin(n=0)
    Out[285]: Timestamp('2014-02-01 00:00:00')

    In [286]: pd.Timestamp("2014-01-02") + pd.offsets.MonthEnd(n=0)
    Out[286]: Timestamp('2014-01-31 00:00:00')

    In [287]: pd.Timestamp("2014-01-01") + pd.offsets.MonthBegin(n=0)
    Out[287]: Timestamp('2014-01-01 00:00:00')

    In [288]: pd.Timestamp("2014-01-31") + pd.offsets.MonthEnd(n=0)
    Out[288]: Timestamp('2014-01-31 00:00:00')

### Holidays / holiday calendars

Holidays and calendars provide a simple way to define holiday rules to be used with `CustomBusinessDay` or in other analysis that requires a predefined set of holidays. The `AbstractHolidayCalendar` class provides all the necessary methods to return a list of holidays and only `rules` need to be defined in a specific holiday calendar class. Furthermore, the `start_date` and `end_date` class attributes determine over what date range holidays are generated. These should be overwritten on the `AbstractHolidayCalendar` class to have the range apply to all calendar subclasses. `USFederalHolidayCalendar` is the only calendar that exists and primarily serves as an example for developing other calendars.

For holidays that occur on fixed dates (e.g., US Memorial Day or July 4th) an observance rule determines when that holiday is observed if it falls on a weekend or some other non-observed day. Defined observance rules are:

Rule | Description
--- | ---
next_workday | move to the next weekday, always advancing by at least one day
previous_workday | move to the previous weekday, always moving back by at least one day
nearest_workday | move Saturday to Friday and Sunday to Monday
before_nearest_workday | apply `nearest_workday` and then move to previous workday before that day
after_nearest_workday | apply `nearest_workday` and then move to next workday after that day
sunday_to_monday | move Sunday to following Monday
next_monday_or_tuesday | move Saturday to Monday and Sunday/Monday to Tuesday
previous_friday | move Saturday and Sunday to previous Friday
next_monday | move Saturday and Sunday to following Monday
weekend_to_monday | same as `next_monday`

An example of how holidays and holiday calendars are defined:

    In [289]: from pandas.tseries.holiday import (
       .....:     Holiday,
       .....:     USMemorialDay,
       .....:     AbstractHolidayCalendar,
       .....:     nearest_workday,
       .....:     MO,
       .....: )
       .....:

    In [290]: class ExampleCalendar(AbstractHolidayCalendar):
       .....:     rules = [
       .....:         USMemorialDay,
       .....:         Holiday("July 4th", month=7, day=4, observance=nearest_workday),
       .....:         Holiday(
       .....:             "Columbus Day",
       .....:             month=10,
       .....:             day=1,
       .....:             offset=pd.DateOffset(weekday=MO(2)),
       .....:         ),
       .....:     ]
       .....:

    In [291]: cal = ExampleCalendar()

    In [292]: cal.holidays(datetime.datetime(2012, 1, 1), datetime.datetime(2012, 12, 31))
    Out[292]:
    DatetimeIndex(['2012-05-28', '2012-07-04', '2012-10-08'],
                  dtype='datetime64[us]', freq=None)

hint:

  weekday=MO(2) is same as 2 * Week(weekday=2)

Using this calendar, creating an index or doing offset arithmetic skips weekends and holidays (i.e., Memorial Day/July 4th). For example, the below defines a custom business day offset using the `ExampleCalendar`. Like any other offset, it can be used to create a `DatetimeIndex` or added to `datetime` or `Timestamp` objects.

    In [293]: pd.date_range(
       .....:     start="7/1/2012", end="7/10/2012", freq=pd.offsets.CDay(calendar=cal)
       .....: ).to_pydatetime()
       .....:
    Out[293]:
    array([datetime.datetime(2012, 7, 2, 0, 0),
           datetime.datetime(2012, 7, 3, 0, 0),
           datetime.datetime(2012, 7, 5, 0, 0),
           datetime.datetime(2012, 7, 6, 0, 0),
           datetime.datetime(2012, 7, 9, 0, 0),
           datetime.datetime(2012, 7, 10, 0, 0)], dtype=object)

    In [294]: offset = pd.offsets.CustomBusinessDay(calendar=cal)

    In [295]: datetime.datetime(2012, 5, 25) + offset
    Out[295]: Timestamp('2012-05-29 00:00:00')

    In [296]: datetime.datetime(2012, 7, 3) + offset
    Out[296]: Timestamp('2012-07-05 00:00:00')

    In [297]: datetime.datetime(2012, 7, 3) + 2 * offset
    Out[297]: Timestamp('2012-07-06 00:00:00')

    In [298]: datetime.datetime(2012, 7, 6) + offset
    Out[298]: Timestamp('2012-07-09 00:00:00')

Ranges are defined by the `start_date` and `end_date` class attributes of `AbstractHolidayCalendar`. The defaults are shown below.

    In [299]: AbstractHolidayCalendar.start_date
    Out[299]: Timestamp('1970-01-01 00:00:00')

    In [300]: AbstractHolidayCalendar.end_date
    Out[300]: Timestamp('2200-12-31 00:00:00')

These dates can be overwritten by setting the attributes as datetime/Timestamp/string.

    In [301]: AbstractHolidayCalendar.start_date = datetime.datetime(2012, 1, 1)

    In [302]: AbstractHolidayCalendar.end_date = datetime.datetime(2012, 12, 31)

    In [303]: cal.holidays()
    Out[303]:
    DatetimeIndex(['2012-05-28', '2012-07-04', '2012-10-08'],
                  dtype='datetime64[us]', freq=None)

Every calendar class is accessible by name using the `get_calendar` function which returns a holiday class instance. Any imported calendar class will automatically be available by this function. Also, `HolidayCalendarFactory` provides an easy interface to create calendars that are combinations of calendars or calendars with additional rules.

    In [304]: from pandas.tseries.holiday import get_calendar, HolidayCalendarFactory, USLaborDay

    In [305]: cal = get_calendar("ExampleCalendar")

    In [306]: cal.rules
    Out[306]:
    [Holiday: Memorial Day (month=5, day=31, offset=<DateOffset: weekday=MO(-1)>),
     Holiday: July 4th (month=7, day=4, observance=<function nearest_workday at 0x7f9ead585620>),
     Holiday: Columbus Day (month=10, day=1, offset=<DateOffset: weekday=MO(+2)>)]

    In [307]: new_cal = HolidayCalendarFactory("NewExampleCalendar", cal, USLaborDay)

    In [308]: new_cal.rules
    Out[308]:
    [Holiday: Labor Day (month=9, day=1, offset=<DateOffset: weekday=MO(+1)>),
     Holiday: Memorial Day (month=5, day=31, offset=<DateOffset: weekday=MO(-1)>),
     Holiday: July 4th (month=7, day=4, observance=<function nearest_workday at 0x7f9ead585620>),
     Holiday: Columbus Day (month=10, day=1, offset=<DateOffset: weekday=MO(+2)>)]

## Time Series-related instance methods

### Shifting / lagging

One may want to shift or lag the values in a time series back and forward in time. The method for this is `shift()`, which is available on all of the pandas objects.

    In [309]: ts = pd.Series(range(len(rng)), index=rng)

    In [310]: ts = ts[:5]

    In [311]: ts.shift(1)
    Out[311]:
    2012-01-01    NaN
    2012-01-02    0.0
    2012-01-03    1.0
    Freq: D, dtype: float64

The `shift` method accepts a `freq` argument which can accept a `DateOffset` class or other `timedelta`-like object or also an offset alias.

When `freq` is specified, `shift` method changes all the dates in the index rather than changing the alignment of the data and the index:

    In [312]: ts.shift(5, freq="D")
    Out[312]:
    2012-01-06    0
    2012-01-07    1
    2012-01-08    2
    Freq: D, dtype: int64

    In [313]: ts.shift(5, freq=pd.offsets.BDay())
    Out[313]:
    2012-01-06    0
    2012-01-09    1
    2012-01-10    2
    dtype: int64

    In [314]: ts.shift(5, freq="BME")
    Out[314]:
    2012-05-31    0
    2012-05-31    1
    2012-05-31    2
    dtype: int64

Note that with when `freq` is specified, the leading entry is no longer NaN because the data is not being realigned.

### Frequency conversion

The primary function for changing frequencies is the `asfreq()` method. For a `DatetimeIndex`, this is basically just a thin, but convenient wrapper around `reindex()` which generates a `date_range` and calls `reindex`.

    In [315]: dr = pd.date_range("1/1/2010", periods=3, freq=3 * pd.offsets.BDay())

    In [316]: ts = pd.Series(np.random.randn(3), index=dr)

    In [317]: ts
    Out[317]:
    2010-01-01    1.494522
    2010-01-06   -0.778425
    2010-01-11   -0.253355
    Freq: 3B, dtype: float64

    In [318]: ts.asfreq(pd.offsets.BDay())
    Out[318]:
    2010-01-01    1.494522
    2010-01-04         NaN
    2010-01-05         NaN
    2010-01-06   -0.778425
    2010-01-07         NaN
    2010-01-08         NaN
    2010-01-11   -0.253355
    Freq: B, dtype: float64

`asfreq` provides a further convenience so you can specify an interpolation method for any gaps that may appear after the frequency conversion.

    In [319]: ts.asfreq(pd.offsets.BDay(), method="pad")
    Out[319]:
    2010-01-01    1.494522
    2010-01-04    1.494522
    2010-01-05    1.494522
    2010-01-06   -0.778425
    2010-01-07   -0.778425
    2010-01-08   -0.778425
    2010-01-11   -0.253355
    Freq: B, dtype: float64

### Filling forward / backward

Related to `asfreq` and `reindex` is `fillna()`, which is documented in the missing data section.

### Converting to Python datetimes

`DatetimeIndex` can be converted to an array of Python native `datetime.datetime` objects using the `to_pydatetime` method.

## Resampling

pandas has a simple, powerful, and efficient functionality for performing resampling operations during frequency conversion (e.g., converting secondly data into 5-minutely data). This is extremely common in, but not limited to, financial applications.

`resample()` is a time-based groupby, followed by a reduction method on each of its groups. See some cookbook examples for some advanced strategies.

The `resample()` method can be used directly from `DataFrameGroupBy` objects, see the groupby docs.

### Basics

    In [320]: rng = pd.date_range("1/1/2012", periods=100, freq="s")

    In [321]: ts = pd.Series(np.random.randint(0, 500, len(rng)), index=rng)

    In [322]: ts.resample("5Min").sum()
    Out[322]:
    2012-01-01    25103
    Freq: 5min, dtype: int64

The `resample` function is very flexible and allows you to specify many different parameters to control the frequency conversion and resampling operation.

Any built-in method available via GroupBy is available as a method of the returned object, including `sum`, `mean`, `std`, `sem`, `max`, `min`, `median`, `first`, `last`, `ohlc`:

    In [323]: ts.resample("5Min").mean()
    Out[323]:
    2012-01-01    251.03
    Freq: 5min, dtype: float64

    In [324]: ts.resample("5Min").ohlc()
    Out[324]:
                open  high  low  close
    2012-01-01   308   460    9    205

    In [325]: ts.resample("5Min").max()
    Out[325]:
    2012-01-01    460
    Freq: 5min, dtype: int64

For downsampling, `closed` can be set to ‘left’ or ‘right’ to specify which end of the interval is closed:

    In [326]: ts.resample("5Min", closed="right").mean()
    Out[326]:
    2011-12-31 23:55:00    308.000000
    2012-01-01 00:00:00    250.454545
    Freq: 5min, dtype: float64

    In [327]: ts.resample("5Min", closed="left").mean()
    Out[327]:
    2012-01-01    251.03
    Freq: 5min, dtype: float64

Parameters like `label` are used to manipulate the resulting labels. `label` specifies whether the result is labeled with the beginning or the end of the interval.

    In [328]: ts.resample("5Min").mean()  # by default label='left'
    Out[328]:
    2012-01-01    251.03
    Freq: 5min, dtype: float64

    In [329]: ts.resample("5Min", label="left").mean()
    Out[329]:
    2012-01-01    251.03
    Freq: 5min, dtype: float64

Warning

The default values for `label` and `closed` is ‘left’ for all frequency offsets except for ‘ME’, ‘YE’, ‘QE’, ‘BME’, ‘BYE’, ‘BQE’, and ‘W’ which all have a default of ‘right’.

This might unintendedly lead to looking ahead, where the value for a later time is pulled back to a previous time as in the following example with the `BusinessDay` frequency:

    In [330]: s = pd.date_range("2000-01-01", "2000-01-05").to_series()

    In [331]: s.iloc[2] = pd.NaT

    In [332]: s.dt.day_name()
    Out[332]:
    2000-01-01     Saturday
    2000-01-02       Sunday
    2000-01-03          NaN
    2000-01-04      Tuesday
    2000-01-05    Wednesday
    Freq: D, dtype: str

    # default: label='left', closed='left'
    In [333]: s.resample("B").last().dt.day_name()
    Out[333]:
    1999-12-31       Sunday
    2000-01-03          NaN
    2000-01-04      Tuesday
    2000-01-05    Wednesday
    Freq: B, dtype: str

Notice how the value for Sunday got pulled back to the previous Friday. To get the behavior where the value for Sunday is pushed to Monday, use instead

    In [334]: s.resample("B", label="right", closed="right").last().dt.day_name()
    Out[334]:
    2000-01-03       Sunday
    2000-01-04      Tuesday
    2000-01-05    Wednesday
    2000-01-06          NaN
    Freq: B, dtype: str

The `axis` parameter can be set to 0 or 1 and allows you to resample the specified axis for a `DataFrame`.

`kind` can be set to ‘timestamp’ or ‘period’ to convert the resulting index to/from timestamp and time span representations. By default `resample` retains the input representation.

`convention` can be set to ‘start’ or ‘end’ when resampling period data (detail below). It specifies how low frequency periods are converted to higher frequency periods.

### Upsampling

For upsampling, you can specify a way to upsample and the `limit` parameter to interpolate over the gaps that are created:

    # from secondly to every 250 milliseconds
    In [335]: ts[:2].resample("250ms").asfreq()
    Out[335]:
    2012-01-01 00:00:00.000    308.0
    2012-01-01 00:00:00.250      NaN
    2012-01-01 00:00:00.500      NaN
    2012-01-01 00:00:00.750      NaN
    2012-01-01 00:00:01.000    204.0
    Freq: 250ms, dtype: float64

    In [336]: ts[:2].resample("250ms").ffill()
    Out[336]:
    2012-01-01 00:00:00.000    308
    2012-01-01 00:00:00.250    308
    2012-01-01 00:00:00.500    308
    2012-01-01 00:00:00.750    308
    2012-01-01 00:00:01.000    204
    Freq: 250ms, dtype: int64

    In [337]: ts[:2].resample("250ms").ffill(limit=2)
    Out[337]:
    2012-01-01 00:00:00.000    308.0
    2012-01-01 00:00:00.250    308.0
    2012-01-01 00:00:00.500    308.0
    2012-01-01 00:00:00.750      NaN
    2012-01-01 00:00:01.000    204.0
    Freq: 250ms, dtype: float64

### Sparse resampling

Sparse timeseries are the ones where you have a lot fewer points relative to the amount of time you are looking to resample. Naively upsampling a sparse series can potentially generate lots of intermediate values. When you don’t want to use a method to fill these values, e.g. `fill_method` is `None`, then intermediate values will be filled with `NaN`.

Since `resample` is a time-based groupby, the following is a method to efficiently resample only the groups that are not all `NaN`.

    In [338]: rng = pd.date_range("2014-1-1", periods=100, freq="D") + pd.Timedelta("1s")

    In [339]: ts = pd.Series(range(100), index=rng)

If we want to resample to the full range of the series:

    In [340]: ts.resample("3min").sum()
    Out[340]:
    2014-01-01 00:00:00     0
    2014-01-01 00:03:00     0
    2014-01-01 00:06:00     0
    2014-01-01 00:09:00     0
    2014-01-01 00:12:00     0
                           ..
    2014-04-09 23:48:00     0
    2014-04-09 23:51:00     0
    2014-04-09 23:54:00     0
    2014-04-09 23:57:00     0
    2014-04-10 00:00:00    99
    Freq: 3min, Length: 47521, dtype: int64

We can instead only resample those groups where we have points as follows:

    In [341]: from functools import partial

    In [342]: from pandas.tseries.frequencies import to_offset

    In [343]: def round(t, freq):
       .....:     freq = to_offset(freq)
       .....:     td = pd.Timedelta(freq)
       .....:     return pd.Timestamp((t.value // td.value) * td.value)
       .....:

    In [344]: ts.groupby(partial(round, freq="3min")).sum()
    Out[344]:
    2014-01-01     0
    2014-01-02     1
    2014-01-03     2
    2014-01-04     3
    2014-01-05     4
                  ..
    2014-04-06    95
    2014-04-07    96
    2014-04-08    97
    2014-04-09    98
    2014-04-10    99
    Length: 100, dtype: int64

### Aggregation

The `resample()` method returns a `pandas.api.typing.Resampler` instance. Similar to the aggregating API, groupby API, and the window API, a `Resampler` can be selectively resampled.

Resampling a `DataFrame`, the default will be to act on all columns with the same function.

    In [345]: df = pd.DataFrame(
       .....:     np.random.randn(1000, 3),
       .....:     index=pd.date_range("1/1/2012", freq="s", periods=1000),
       .....:     columns=["A", "B", "C"],
       .....: )
       .....:

    In [346]: r = df.resample("3min")

    In [347]: r.mean()
    Out[347]:
                                A         B         C
    2012-01-01 00:00:00 -0.033823 -0.121514 -0.081447
    2012-01-01 00:03:00  0.056909  0.146731 -0.024320
    2012-01-01 00:06:00 -0.058837  0.047046 -0.052021
    2012-01-01 00:09:00  0.063123 -0.026158 -0.066533
    2012-01-01 00:12:00  0.186340 -0.003144  0.074752
    2012-01-01 00:15:00 -0.085954 -0.016287 -0.050046

We can select a specific column or columns using standard getitem.

    In [348]: r["A"].mean()
    Out[348]:
    2012-01-01 00:00:00   -0.033823
    2012-01-01 00:03:00    0.056909
    2012-01-01 00:06:00   -0.058837
    2012-01-01 00:09:00    0.063123
    2012-01-01 00:12:00    0.186340
    2012-01-01 00:15:00   -0.085954
    Freq: 3min, Name: A, dtype: float64

    In [349]: r[["A", "B"]].mean()
    Out[349]:
                                A         B
    2012-01-01 00:00:00 -0.033823 -0.121514
    2012-01-01 00:03:00  0.056909  0.146731
    2012-01-01 00:06:00 -0.058837  0.047046
    2012-01-01 00:09:00  0.063123 -0.026158
    2012-01-01 00:12:00  0.186340 -0.003144
    2012-01-01 00:15:00 -0.085954 -0.016287

You can pass a list or dict of functions to do aggregation with, outputting a `DataFrame`:

    In [350]: r["A"].agg(["sum", "mean", "std"])
    Out[350]:
                               sum      mean       std
    2012-01-01 00:00:00  -6.088060 -0.033823  1.043263
    2012-01-01 00:03:00  10.243678  0.056909  1.058534
    2012-01-01 00:06:00 -10.590584 -0.058837  0.949264
    2012-01-01 00:09:00  11.362228  0.063123  1.028096
    2012-01-01 00:12:00  33.541257  0.186340  0.884586
    2012-01-01 00:15:00  -8.595393 -0.085954  1.035476

On a resampled `DataFrame`, you can pass a list of functions to apply to each column, which produces an aggregated result with a hierarchical index:

    In [351]: r.agg(["sum", "mean"])
    Out[351]:
                                 A            ...          C
                               sum      mean  ...        sum      mean
    2012-01-01 00:00:00  -6.088060 -0.033823  ... -14.660515 -0.081447
    2012-01-01 00:03:00  10.243678  0.056909  ...  -4.377642 -0.024320
    2012-01-01 00:06:00 -10.590584 -0.058837  ...  -9.363825 -0.052021
    2012-01-01 00:09:00  11.362228  0.063123  ... -11.975895 -0.066533
    2012-01-01 00:12:00  33.541257  0.186340  ...  13.455299  0.074752
    2012-01-01 00:15:00  -8.595393 -0.085954  ...  -5.004580 -0.050046

    [6 rows x 6 columns]

By passing a dict to `aggregate` you can apply a different aggregation to the columns of a `DataFrame`:

    In [352]: r.agg({"A": "sum", "B": lambda x: np.std(x, ddof=1)})
    Out[352]:
                                 A         B
    2012-01-01 00:00:00  -6.088060  1.001294
    2012-01-01 00:03:00  10.243678  1.074597
    2012-01-01 00:06:00 -10.590584  0.987309
    2012-01-01 00:09:00  11.362228  0.944953
    2012-01-01 00:12:00  33.541257  1.095025
    2012-01-01 00:15:00  -8.595393  1.035312

The function names can also be strings. In order for a string to be valid it must be implemented on the resampled object:

    In [353]: r.agg({"A": "sum", "B": "std"})
    Out[353]:
                                 A         B
    2012-01-01 00:00:00  -6.088060  1.001294
    2012-01-01 00:03:00  10.243678  1.074597
    2012-01-01 00:06:00 -10.590584  0.987309
    2012-01-01 00:09:00  11.362228  0.944953
    2012-01-01 00:12:00  33.541257  1.095025
    2012-01-01 00:15:00  -8.595393  1.035312

Furthermore, you can also specify multiple aggregation functions for each column separately.

    In [354]: r.agg({"A": ["sum", "std"], "B": ["mean", "std"]})
    Out[354]:
                                 A                   B
                               sum       std      mean       std
    2012-01-01 00:00:00  -6.088060  1.043263 -0.121514  1.001294
    2012-01-01 00:03:00  10.243678  1.058534  0.146731  1.074597
    2012-01-01 00:06:00 -10.590584  0.949264  0.047046  0.987309
    2012-01-01 00:09:00  11.362228  1.028096 -0.026158  0.944953
    2012-01-01 00:12:00  33.541257  0.884586 -0.003144  1.095025
    2012-01-01 00:15:00  -8.595393  1.035476 -0.016287  1.035312

If a `DataFrame` does not have a datetimelike index, but instead you want to resample based on datetimelike column in the frame, it can passed to the `on` keyword.

    In [355]: df = pd.DataFrame(
       .....:     {"date": pd.date_range("2015-01-01", freq="W", periods=5), "a": np.arange(5)},
       .....:     index=pd.MultiIndex.from_arrays(
       .....:         [[1, 2, 3, 4, 5], pd.date_range("2015-01-01", freq="W", periods=5)],
       .....:         names=["v", "d"],
       .....:     ),
       .....: )
       .....:

    In [356]: df
    Out[356]:
                       date  a
    v d
    1 2015-01-04 2015-01-04  0
    2 2015-01-11 2015-01-11  1
    3 2015-01-18 2015-01-18  2
    4 2015-01-25 2015-01-25  3
    5 2015-02-01 2015-02-01  4

    In [357]: df.resample("MS", on="date")[["a"]].sum()
    Out[357]:
                a
    date
    2015-01-01  6
    2015-02-01  4

Similarly, if you instead want to resample by a datetimelike level of `MultiIndex`, its name or location can be passed to the `level` keyword.

    In [358]: df.resample("MS", level="d")[["a"]].sum()
    Out[358]:
                a
    d
    2015-01-01  6
    2015-02-01  4

### Iterating through groups

With the `Resampler` object in hand, iterating through the grouped data is very natural and functions similarly to `itertools.groupby()`:

    In [359]: small = pd.Series(
       .....:     range(6),
       .....:     index=pd.to_datetime(
       .....:         [
       .....:             "2017-01-01T00:00:00",
       .....:             "2017-01-01T00:30:00",
       .....:             "2017-01-01T00:31:00",
       .....:             "2017-01-01T01:00:00",
       .....:             "2017-01-01T03:00:00",
       .....:             "2017-01-01T03:05:00",
       .....:         ]
       .....:     ),
       .....: )
       .....:

    In [360]: resampled = small.resample("h")

    In [361]: for name, group in resampled:
       .....:     print("Group: ", name)
       .....:     print("-" * 27)
       .....:     print(group, end="\n\n")
       .....:
    Group:  2017-01-01 00:00:00
    ---------------------------
    2017-01-01 00:00:00    0
    2017-01-01 00:30:00    1
    2017-01-01 00:31:00    2
    dtype: int64

    Group:  2017-01-01 01:00:00
    ---------------------------
    2017-01-01 01:00:00    3
    dtype: int64

    Group:  2017-01-01 02:00:00
    ---------------------------
    Series([], dtype: int64)

    Group:  2017-01-01 03:00:00
    ---------------------------
    2017-01-01 03:00:00    4
    2017-01-01 03:05:00    5
    dtype: int64

See Iterating through groups or `Resampler.__iter__` for more.

### Use `origin` or `offset` to adjust the start of the bins

The bins of the grouping are adjusted based on the beginning of the day of the time series starting point. This works well with frequencies that are multiples of a day (like `30D`) or that divide a day evenly (like `90s` or `1min`). This can create inconsistencies with some frequencies that do not meet this criteria. To change this behavior you can specify a fixed Timestamp with the argument `origin`.

For example:

    In [362]: start, end = "2000-10-01 23:30:00", "2000-10-02 00:30:00"

    In [363]: middle = "2000-10-02 00:00:00"

    In [364]: rng = pd.date_range(start, end, freq="7min")

    In [365]: ts = pd.Series(np.arange(len(rng)) * 3, index=rng)

    In [366]: ts
    Out[366]:
    2000-10-01 23:30:00     0
    2000-10-01 23:37:00     3
    2000-10-01 23:44:00     6
    2000-10-01 23:51:00     9
    2000-10-01 23:58:00    12
    2000-10-02 00:05:00    15
    2000-10-02 00:12:00    18
    2000-10-02 00:19:00    21
    2000-10-02 00:26:00    24
    Freq: 7min, dtype: int64

Here we can see that, when using `origin` with its default value (`'start_day'`), the result after `'2000-10-02 00:00:00'` are not identical depending on the start of time series:

    In [367]: ts.resample("17min", origin="start_day").sum()
    Out[367]:
    2000-10-01 23:14:00     0
    2000-10-01 23:31:00     9
    2000-10-01 23:48:00    21
    2000-10-02 00:05:00    54
    2000-10-02 00:22:00    24
    Freq: 17min, dtype: int64

    In [368]: ts[middle:end].resample("17min", origin="start_day").sum()
    Out[368]:
    2000-10-02 00:00:00    33
    2000-10-02 00:17:00    45
    Freq: 17min, dtype: int64

Here we can see that, when setting `origin` to `'epoch'`, the result after `'2000-10-02 00:00:00'` are identical depending on the start of time series:

    In [369]: ts.resample("17min", origin="epoch").sum()
    Out[369]:
    2000-10-01 23:18:00     0
    2000-10-01 23:35:00    18
    2000-10-01 23:52:00    27
    2000-10-02 00:09:00    39
    2000-10-02 00:26:00    24
    Freq: 17min, dtype: int64

    In [370]: ts[middle:end].resample("17min", origin="epoch").sum()
    Out[370]:
    2000-10-01 23:52:00    15
    2000-10-02 00:09:00    39
    2000-10-02 00:26:00    24
    Freq: 17min, dtype: int64

If needed you can use a custom timestamp for `origin`:

    In [371]: ts.resample("17min", origin="2001-01-01").sum()
    Out[371]:
    2000-10-01 23:30:00     9
    2000-10-01 23:47:00    21
    2000-10-02 00:04:00    54
    2000-10-02 00:21:00    24
    Freq: 17min, dtype: int64

    In [372]: ts[middle:end].resample("17min", origin=pd.Timestamp("2001-01-01")).sum()
    Out[372]:
    2000-10-02 00:04:00    54
    2000-10-02 00:21:00    24
    Freq: 17min, dtype: int64

If needed you can just adjust the bins with an `offset` Timedelta that would be added to the default `origin`. Those two examples are equivalent for this time series:

    In [373]: ts.resample("17min", origin="start").sum()
    Out[373]:
    2000-10-01 23:30:00     9
    2000-10-01 23:47:00    21
    2000-10-02 00:04:00    54
    2000-10-02 00:21:00    24
    Freq: 17min, dtype: int64

    In [374]: ts.resample("17min", offset="23h30min").sum()
    Out[374]:
    2000-10-01 23:30:00     9
    2000-10-01 23:47:00    21
    2000-10-02 00:04:00    54
    2000-10-02 00:21:00    24
    Freq: 17min, dtype: int64

Note the use of `'start'` for `origin` on the last example. In that case, `origin` will be set to the first value of the timeseries.

### Backward resample

Instead of adjusting the beginning of bins, sometimes we need to fix the end of the bins to make a backward resample with a given `freq`. The backward resample sets `closed` to `'right'` by default since the last value should be considered as the edge point for the last bin.

We can set `origin` to `'end'`. The value for a specific `Timestamp` index stands for the resample result from the current `Timestamp` minus `freq` to the current `Timestamp` with a right close.

    In [375]: ts.resample('17min', origin='end').sum()
    Out[375]:
    2000-10-01 23:35:00     0
    2000-10-01 23:52:00    18
    2000-10-02 00:09:00    27
    2000-10-02 00:26:00    63
    Freq: 17min, dtype: int64

Besides, in contrast with the `'start_day'` option, `end_day` is supported. This will set the origin as the ceiling midnight of the largest `Timestamp`.

    In [376]: ts.resample('17min', origin='end_day').sum()
    Out[376]:
    2000-10-01 23:38:00     3
    2000-10-01 23:55:00    15
    2000-10-02 00:12:00    45
    2000-10-02 00:29:00    45
    Freq: 17min, dtype: int64

The above result uses `2000-10-02 00:29:00` as the last bin’s right edge since the following computation.

    In [377]: ceil_mid = rng.max().ceil('D')

    In [378]: freq = pd.offsets.Minute(17)

    In [379]: bin_res = ceil_mid - freq * ((ceil_mid - rng.max()) // freq)

    In [380]: bin_res
    Out[380]: Timestamp('2000-10-02 00:29:00')

## Time span representation

Regular intervals of time are represented by `Period` objects in pandas while sequences of `Period` objects are collected in a `PeriodIndex`, which can be created with the convenience function `period_range`.

### Period

A `Period` represents a span of time (e.g., a day, a month, a quarter, etc). You can specify the span via `freq` keyword using a frequency alias like below. Because `freq` represents a span of `Period`, it cannot be negative like “-3D”.

    In [381]: pd.Period("2012", freq="Y-DEC")
    Out[381]: Period('2012', 'Y-DEC')

    In [382]: pd.Period("2012-1-1", freq="D")
    Out[382]: Period('2012-01-01', 'D')

    In [383]: pd.Period("2012-1-1 19:00", freq="h")
    Out[383]: Period('2012-01-01 19:00', 'h')

    In [384]: pd.Period("2012-1-1 19:00", freq="5h")
    Out[384]: Period('2012-01-01 19:00', '5h')

Adding and subtracting integers from periods shifts the period by its own frequency. Arithmetic is not allowed between `Period` with different `freq` (span).

    In [385]: p = pd.Period("2012", freq="Y-DEC")

    In [386]: p + 1
    Out[386]: Period('2013', 'Y-DEC')

    In [387]: p - 3
    Out[387]: Period('2009', 'Y-DEC')

    In [388]: p = pd.Period("2012-01", freq="2M")

    In [389]: p + 2
    Out[389]: Period('2012-05', '2M')

    In [390]: p - 1
    Out[390]: Period('2011-11', '2M')

    In [391]: p == pd.Period("2012-01", freq="3M")
    Out[391]: False

If `Period` freq is daily or higher (`D`, `h`, `min`, `s`, `ms`, `us`, and `ns`), `offsets` and `timedelta`-like can be added if the result can have the same freq. Otherwise, `ValueError` will be raised.

    In [392]: p = pd.Period("2014-07-01 09:00", freq="h")

    In [393]: p + pd.offsets.Hour(2)
    Out[393]: Period('2014-07-01 11:00', 'h')

    In [394]: p + datetime.timedelta(minutes=120)
    Out[394]: Period('2014-07-01 11:00', 'h')

    In [395]: p + np.timedelta64(7200, "s")
    Out[395]: Period('2014-07-01 11:00', 'h')

    In [396]: p + pd.offsets.Minute(5)
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    File pandas/_libs/tslibs/period.pyx:2134, in pandas._libs.tslibs.period._Period._add_timedeltalike_scalar()
    -> 2134 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/timedeltas.pyx:295, in pandas._libs.tslibs.timedeltas.delta_to_nanoseconds()
    --> 295 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/np_datetime.pyx:796, in pandas._libs.tslibs.np_datetime.convert_reso()
    --> 796 'Could not get source, probably due dynamically evaluated source code.'
    ValueError: Cannot losslessly convert units

    The above exception was the direct cause of the following exception:

    IncompatibleFrequency                     Traceback (most recent call last)
    Cell In[396], line 1
    ----> 1 p + pd.offsets.Minute(5)

    File pandas/_libs/tslibs/period.pyx:2155, in pandas._libs.tslibs.period._Period.__add__()
    -> 2155 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/period.pyx:2136, in pandas._libs.tslibs.period._Period._add_timedeltalike_scalar()
    -> 2136 'Could not get source, probably due dynamically evaluated source code.'
    IncompatibleFrequency: Input cannot be converted to Period(freq=h)

If `Period` has other frequencies, only the same `offsets` can be added. Otherwise, `ValueError` will be raised.

    In [397]: p = pd.Period("2014-07", freq="M")

    In [398]: p + pd.offsets.MonthEnd(3)
    Out[398]: Period('2014-10', 'M')

    In [399]: p + pd.offsets.MonthBegin(3)
    ---------------------------------------------------------------------------
    IncompatibleFrequency                     Traceback (most recent call last)
    Cell In[399], line 1
    ----> 1 p + pd.offsets.MonthBegin(3)

    File pandas/_libs/tslibs/period.pyx:2157, in pandas._libs.tslibs.period._Period.__add__()
    -> 2157 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/period.pyx:2147, in pandas._libs.tslibs.period._Period._add_offset()
    -> 2147 'Could not get source, probably due dynamically evaluated source code.'
    File pandas/_libs/tslibs/period.pyx:1940, in pandas._libs.tslibs.period.PeriodMixin._require_matching_unit()
    -> 1940 'Could not get source, probably due dynamically evaluated source code.'
    IncompatibleFrequency: Input has different freq=3MS from Period(freq=M)

Taking the difference of `Period` instances with the same frequency will return the number of frequency units between them:

    In [400]: pd.Period("2012", freq="Y-DEC") - pd.Period("2002", freq="Y-DEC")
    Out[400]: <10 * YearEnds: month=12>

### PeriodIndex and period_range

Regular sequences of `Period` objects can be collected in a `PeriodIndex`, which can be constructed using the `period_range` convenience function:

    In [401]: prng = pd.period_range("1/1/2011", "1/1/2012", freq="M")

    In [402]: prng
    Out[402]:
    PeriodIndex(['2011-01', '2011-02', '2011-03', '2011-04', '2011-05', '2011-06',
                 '2011-07', '2011-08', '2011-09', '2011-10', '2011-11', '2011-12',
                 '2012-01'],
                dtype='period[M]')

The `PeriodIndex` constructor can also be used directly:

    In [403]: pd.PeriodIndex(["2011-1", "2011-2", "2011-3"], freq="M")
    Out[403]: PeriodIndex(['2011-01', '2011-02', '2011-03'], dtype='period[M]')

Passing multiplied frequency outputs a sequence of `Period` which has multiplied span.

    In [404]: pd.period_range(start="2014-01", freq="3M", periods=4)
    Out[404]: PeriodIndex(['2014-01', '2014-04', '2014-07', '2014-10'], dtype='period[3M]')

If `start` or `end` are `Period` objects, they will be used as anchor endpoints for a `PeriodIndex` with frequency matching that of the `PeriodIndex` constructor.

    In [405]: pd.period_range(
       .....:     start=pd.Period("2017Q1", freq="Q"), end=pd.Period("2017Q2", freq="Q"), freq="M"
       .....: )
       .....:
    Out[405]: PeriodIndex(['2017-03', '2017-04', '2017-05', '2017-06'], dtype='period[M]')

Just like `DatetimeIndex`, a `PeriodIndex` can also be used to index pandas objects:

    In [406]: ps = pd.Series(np.random.randn(len(prng)), prng)

    In [407]: ps
    Out[407]:
    2011-01   -2.916901
    2011-02    0.514474
    2011-03    1.346470
    2011-04    0.816397
    2011-05    2.258648
    2011-06    0.494789
    2011-07    0.301239
    2011-08    0.464776
    2011-09   -1.393581
    2011-10    0.056780
    2011-11    0.197035
    2011-12    2.261385
    2012-01   -0.329583
    Freq: M, dtype: float64

`PeriodIndex` supports addition and subtraction with the same rule as `Period`.

    In [408]: idx = pd.period_range("2014-07-01 09:00", periods=5, freq="h")

    In [409]: idx
    Out[409]:
    PeriodIndex(['2014-07-01 09:00', '2014-07-01 10:00', '2014-07-01 11:00',
                 '2014-07-01 12:00', '2014-07-01 13:00'],
                dtype='period[h]')

    In [410]: idx + pd.offsets.Hour(2)
    Out[410]:
    PeriodIndex(['2014-07-01 11:00', '2014-07-01 12:00', '2014-07-01 13:00',
                 '2014-07-01 14:00', '2014-07-01 15:00'],
                dtype='period[h]')

    In [411]: idx = pd.period_range("2014-07", periods=5, freq="M")

    In [412]: idx
    Out[412]:
    PeriodIndex(['2014-07', '2014-08', '2014-09', '2014-10', '2014-11'],
                dtype='period[M]')

    In [413]: idx + pd.offsets.MonthEnd(3)
    Out[413]:
    PeriodIndex(['2014-10', '2014-11', '2014-12', '2015-01', '2015-02'],
                dtype='period[M]')

`PeriodIndex` has its own dtype named `period`, refer to Period Dtypes.

### Period dtypes

`PeriodIndex` has a custom `period` dtype. This is a pandas extension dtype similar to the timezone aware dtype (`datetime64[ns, tz]`).

The `period` dtype holds the `freq` attribute and is represented with `period[freq]` like `period[D]` or `period[M]`, using frequency strings.

    In [414]: pi = pd.period_range("2016-01-01", periods=3, freq="M")

    In [415]: pi
    Out[415]: PeriodIndex(['2016-01', '2016-02', '2016-03'], dtype='period[M]')

    In [416]: pi.dtype
    Out[416]: period[M]

The `period` dtype can be used in `.astype(...)`. It allows one to change the `freq` of a `PeriodIndex` like `.asfreq()` and convert a `DatetimeIndex` to `PeriodIndex` like `to_period()`:

    # change monthly freq to daily freq
    In [417]: pi.astype("period[D]")
    Out[417]: PeriodIndex(['2016-01-31', '2016-02-29', '2016-03-31'], dtype='period[D]')

    # convert to DatetimeIndex
    In [418]: pi.astype("datetime64[ns]")
    Out[418]:
    DatetimeIndex(['2016-01-01', '2016-02-01', '2016-03-01'],
                  dtype='datetime64[ns]', freq=None)

    # convert to PeriodIndex
    In [419]: dti = pd.date_range("2011-01-01", freq="ME", periods=3)

    In [420]: dti
    Out[420]:
    DatetimeIndex(['2011-01-31', '2011-02-28', '2011-03-31'],
                  dtype='datetime64[us]', freq='ME')

    In [421]: dti.astype("period[M]")
    Out[421]: PeriodIndex(['2011-01', '2011-02', '2011-03'], dtype='period[M]')

### PeriodIndex partial string indexing

PeriodIndex now supports partial string slicing with non-monotonic indexes.

You can pass in dates and strings to `Series` and `DataFrame` with `PeriodIndex`, in the same manner as `DatetimeIndex`. For details, refer to DatetimeIndex Partial String Indexing.

    In [422]: ps["2011-01"]
    Out[422]: np.float64(-2.9169013294054507)

    In [423]: ps[datetime.datetime(2011, 12, 25):]
    Out[423]:
    2011-12    2.261385
    2012-01   -0.329583
    Freq: M, dtype: float64

    In [424]: ps["10/31/2011":"12/31/2011"]
    Out[424]:
    2011-10    0.056780
    2011-11    0.197035
    2011-12    2.261385
    Freq: M, dtype: float64

Passing a string representing a lower frequency than `PeriodIndex` returns partial sliced data.

    In [425]: ps["2011"]
    Out[425]:
    2011-01   -2.916901
    2011-02    0.514474
    2011-03    1.346470
    2011-04    0.816397
    2011-05    2.258648
    2011-06    0.494789
    2011-07    0.301239
    2011-08    0.464776
    2011-09   -1.393581
    2011-10    0.056780
    2011-11    0.197035
    2011-12    2.261385
    Freq: M, dtype: float64

    In [426]: dfp = pd.DataFrame(
       .....:     np.random.randn(600, 1),
       .....:     columns=["A"],
       .....:     index=pd.period_range("2013-01-01 9:00", periods=600, freq="min"),
       .....: )
       .....:

    In [427]: dfp
    Out[427]:
                             A
    2013-01-01 09:00 -0.538468
    2013-01-01 09:01 -1.365819
    2013-01-01 09:02 -0.969051
    2013-01-01 09:03 -0.331152
    2013-01-01 09:04 -0.245334
    ...                    ...
    2013-01-01 18:55  0.522460
    2013-01-01 18:56  0.118710
    2013-01-01 18:57  0.167517
    2013-01-01 18:58  0.922883
    2013-01-01 18:59  1.721104

    [600 rows x 1 columns]

    In [428]: dfp.loc["2013-01-01 10h"]
    Out[428]:
                             A
    2013-01-01 10:00 -0.308975
    2013-01-01 10:01  0.542520
    2013-01-01 10:02  1.061068
    2013-01-01 10:03  0.754005
    2013-01-01 10:04  0.352933
    ...                    ...
    2013-01-01 10:55 -0.865621
    2013-01-01 10:56 -1.167818
    2013-01-01 10:57 -2.081748
    2013-01-01 10:58 -0.527146
    2013-01-01 10:59  0.802298

    [60 rows x 1 columns]

As with `DatetimeIndex`, the endpoints will be included in the result. The example below slices data starting from 10:00 to 11:59.

    In [429]: dfp["2013-01-01 10h":"2013-01-01 11h"]
    Out[429]:
                             A
    2013-01-01 10:00 -0.308975
    2013-01-01 10:01  0.542520
    2013-01-01 10:02  1.061068
    2013-01-01 10:03  0.754005
    2013-01-01 10:04  0.352933
    ...                    ...
    2013-01-01 11:55 -0.590204
    2013-01-01 11:56  1.539990
    2013-01-01 11:57 -1.224826
    2013-01-01 11:58  0.578798
    2013-01-01 11:59 -0.685496

    [120 rows x 1 columns]

### Frequency conversion and resampling with PeriodIndex

The frequency of `Period` and `PeriodIndex` can be converted via the `asfreq` method. Let’s start with the fiscal year 2011, ending in December:

    In [430]: p = pd.Period("2011", freq="Y-DEC")

    In [431]: p
    Out[431]: Period('2011', 'Y-DEC')

We can convert it to a monthly frequency. Using the `how` parameter, we can specify whether to return the starting or ending month:

    In [432]: p.asfreq("M", how="start")
    Out[432]: Period('2011-01', 'M')

    In [433]: p.asfreq("M", how="end")
    Out[433]: Period('2011-12', 'M')

The shorthands ‘s’ and ‘e’ are provided for convenience:

    In [434]: p.asfreq("M", "s")
    Out[434]: Period('2011-01', 'M')

    In [435]: p.asfreq("M", "e")
    Out[435]: Period('2011-12', 'M')

Converting to a “super-period” (e.g., annual frequency is a super-period of quarterly frequency) automatically returns the super-period that includes the input period:

    In [436]: p = pd.Period("2011-12", freq="M")

    In [437]: p.asfreq("Y-NOV")
    Out[437]: Period('2012', 'Y-NOV')

Note that since we converted to an annual frequency that ends the year in November, the monthly period of December 2011 is actually in the 2012 Y-NOV period.

Period conversions with anchored frequencies are particularly useful for working with various quarterly data common to economics, business, and other fields. Many organizations define quarters relative to the month in which their fiscal year starts and ends. Thus, first quarter of 2011 could start in 2010 or a few months into 2011. Via anchored frequencies, pandas works for all quarterly frequencies `Q-JAN` through `Q-DEC`.

`Q-DEC` define regular calendar quarters:

    In [438]: p = pd.Period("2012Q1", freq="Q-DEC")

    In [439]: p.asfreq("D", "s")
    Out[439]: Period('2012-01-01', 'D')

    In [440]: p.asfreq("D", "e")
    Out[440]: Period('2012-03-31', 'D')

`Q-MAR` defines fiscal year end in March:

    In [441]: p = pd.Period("2011Q4", freq="Q-MAR")

    In [442]: p.asfreq("D", "s")
    Out[442]: Period('2011-01-01', 'D')

    In [443]: p.asfreq("D", "e")
    Out[443]: Period('2011-03-31', 'D')

## Converting between representations

Timestamped data can be converted to PeriodIndex-ed data using `to_period` and vice-versa using `to_timestamp`:

    In [444]: rng = pd.date_range("1/1/2012", periods=5, freq="ME")

    In [445]: ts = pd.Series(np.random.randn(len(rng)), index=rng)

    In [446]: ts
    Out[446]:
    2012-01-31    1.931253
    2012-02-29   -0.184594
    2012-03-31    0.249656
    2012-04-30   -0.978151
    2012-05-31   -0.873389
    Freq: ME, dtype: float64

    In [447]: ps = ts.to_period()

    In [448]: ps
    Out[448]:
    2012-01    1.931253
    2012-02   -0.184594
    2012-03    0.249656
    2012-04   -0.978151
    2012-05   -0.873389
    Freq: M, dtype: float64

    In [449]: ps.to_timestamp()
    Out[449]:
    2012-01-01    1.931253
    2012-02-01   -0.184594
    2012-03-01    0.249656
    2012-04-01   -0.978151
    2012-05-01   -0.873389
    Freq: MS, dtype: float64

Remember that ‘s’ and ‘e’ can be used to return the timestamps at the start or end of the period:

    In [450]: ps.to_timestamp("D", how="s")
    Out[450]:
    2012-01-01    1.931253
    2012-02-01   -0.184594
    2012-03-01    0.249656
    2012-04-01   -0.978151
    2012-05-01   -0.873389
    Freq: MS, dtype: float64

Converting between period and timestamp enables some convenient arithmetic functions to be used. In the following example, we convert a quarterly frequency with year ending in November to 9am of the end of the month following the quarter end:

    In [451]: prng = pd.period_range("1990Q1", "2000Q4", freq="Q-NOV")

    In [452]: ts = pd.Series(np.random.randn(len(prng)), prng)

    In [453]: ts.index = (prng.asfreq("M", "e") + 1).asfreq("h", "s") + 9

    In [454]: ts.head()
    Out[454]:
    1990-03-01 09:00   -0.109291
    1990-06-01 09:00   -0.637235
    1990-09-01 09:00   -1.735925
    1990-12-01 09:00    2.096946
    1991-03-01 09:00   -1.039926
    Freq: h, dtype: float64

## Representing out-of-bounds spans

If you have data that is outside of the `Timestamp` bounds, see Timestamp limitations, then you can use a `PeriodIndex` and/or `Series` of `Periods` to do computations.

    In [455]: span = pd.period_range("1215-01-01", "1381-01-01", freq="D")

    In [456]: span
    Out[456]:
    PeriodIndex(['1215-01-01', '1215-01-02', '1215-01-03', '1215-01-04',
                 '1215-01-05', '1215-01-06', '1215-01-07', '1215-01-08',
                 '1215-01-09', '1215-01-10',
                 ...
                 '1380-12-23', '1380-12-24', '1380-12-25', '1380-12-26',
                 '1380-12-27', '1380-12-28', '1380-12-29', '1380-12-30',
                 '1380-12-31', '1381-01-01'],
                dtype='period[D]', length=60632)

To convert from an `int64` based YYYYMMDD representation.

    In [457]: s = pd.Series([20121231, 20141130, 99991231])

    In [458]: s
    Out[458]:
    0    20121231
    1    20141130
    2    99991231
    dtype: int64

    In [459]: def conv(x):
       .....:     return pd.Period(year=x // 10000, month=x // 100 % 100, day=x % 100, freq="D")
       .....:

    In [460]: s.apply(conv)
    Out[460]:
    0    2012-12-31
    1    2014-11-30
    2    9999-12-31
    dtype: period[D]

    In [461]: s.apply(conv)[2]
    Out[461]: Period('9999-12-31', 'D')

These can easily be converted to a `PeriodIndex`:

    In [462]: span = pd.PeriodIndex(s.apply(conv))

    In [463]: span
    Out[463]: PeriodIndex(['2012-12-31', '2014-11-30', '9999-12-31'], dtype='period[D]')

## Time zone handling

pandas provides rich support for working with timestamps in different time zones using the `zoneinfo`, `pytz` and `dateutil` libraries or `datetime.timezone` objects from the standard library.

### Working with time zones

By default, pandas objects are time zone unaware:

    In [464]: rng = pd.date_range("3/6/2012 00:00", periods=15, freq="D")

    In [465]: rng.tz is None
    Out[465]: True

To localize these dates to a time zone (assign a particular time zone to a naive date), you can use the `tz_localize` method or the `tz` keyword argument in `date_range()`, `Timestamp`, or `DatetimeIndex`. You can either pass `zoneinfo`, `pytz` or `dateutil` time zone objects or Olson time zone database strings. Olson time zone strings will return `zoneinfo` time zone objects by default. To return `dateutil` time zone objects, append `dateutil/` before the string.

- For `zoneinfo`, a list of available timezones are available from `zoneinfo.available_timezones()`.

- If `pytz` is installed (optional), you can find a list of common (and less common) time zones using `pytz.all_timezones`.

- `dateutil` uses the OS time zones so there isn’t a fixed list available. For common zones, the names are the same as `zoneinfo`.

    In [466]: import dateutil

    In [467]: from zoneinfo import ZoneInfo

    # zoneinfo (default when using Olson strings)
    In [468]: rng_zi = pd.date_range("3/6/2012 00:00", periods=3, freq="D", tz="Europe/London")

    In [469]: rng_zi.tz
    Out[469]: zoneinfo.ZoneInfo(key='Europe/London')

    # dateutil
    In [470]: rng_dateutil = pd.date_range("3/6/2012 00:00", periods=3, freq="D")

    In [471]: rng_dateutil = rng_dateutil.tz_localize("dateutil/Europe/London")

    In [472]: rng_dateutil.tz
    Out[472]: tzfile('/usr/share/zoneinfo/Europe/London')

    # dateutil - utc special case
    In [473]: rng_utc = pd.date_range(
       .....:     "3/6/2012 00:00",
       .....:     periods=3,
       .....:     freq="D",
       .....:     tz=dateutil.tz.tzutc(),
       .....: )
       .....:

    In [474]: rng_utc.tz
    Out[474]: tzutc()

    # datetime.timezone
    In [475]: rng_utc = pd.date_range(
       .....:     "3/6/2012 00:00",
       .....:     periods=3,
       .....:     freq="D",
       .....:     tz=datetime.timezone.utc,
       .....: )
       .....:

    In [476]: rng_utc.tz
    Out[476]: datetime.timezone.utc

Note that the `UTC` time zone is a special case in `dateutil` and should be constructed explicitly as an instance of `dateutil.tz.tzutc`. You can also construct other time zones objects explicitly first.

    # zoneinfo
    In [477]: tz_zi = ZoneInfo("Europe/London")

    In [478]: rng_zi = pd.date_range("3/6/2012 00:00", periods=3, freq="D")

    In [479]: rng_zi = rng_zi.tz_localize(tz_zi)

    In [480]: rng_zi.tz == tz_zi
    Out[480]: True

    # dateutil
    In [481]: tz_dateutil = dateutil.tz.gettz("Europe/London")

    In [482]: rng_dateutil = pd.date_range("3/6/2012 00:00", periods=3, freq="D", tz=tz_dateutil)

    In [483]: rng_dateutil.tz == tz_dateutil
    Out[483]: True

To convert a time zone aware pandas object from one time zone to another, you can use the `tz_convert` method.

    In [484]: rng_zi.tz_convert("US/Eastern")
    Out[484]:
    DatetimeIndex(['2012-03-05 19:00:00-05:00', '2012-03-06 19:00:00-05:00',
                   '2012-03-07 19:00:00-05:00'],
                  dtype='datetime64[us, US/Eastern]', freq=None)

Warning

Be wary of conversions between time zone libraries. For some time zones, different libraries may have different definitions of the zone. This is more of a problem for unusual time zones than for ‘standard’ zones like `US/Eastern`.

Warning

Be aware that a time zone definition across versions of time zone libraries may not be considered equal. This may cause problems when working with stored data that is localized using one version and operated on with a different version. See here for how to handle such a situation.

Warning

Be aware that for times in the future, correct conversion between time zones (and UTC) cannot be guaranteed by any time zone library because a timezone’s offset from UTC may be changed by the respective government.

Under the hood, all timestamps are stored in UTC. Values from a time zone aware `DatetimeIndex` or `Timestamp` will have their fields (day, hour, minute, etc.) localized to the time zone. However, timestamps with the same UTC value are still considered to be equal even if they are in different time zones:

    In [485]: rng_eastern = rng_utc.tz_convert("US/Eastern")

    In [486]: rng_berlin = rng_utc.tz_convert("Europe/Berlin")

    In [487]: rng_eastern[2]
    Out[487]: Timestamp('2012-03-07 19:00:00-0500', tz='US/Eastern')

    In [488]: rng_berlin[2]
    Out[488]: Timestamp('2012-03-08 01:00:00+0100', tz='Europe/Berlin')

    In [489]: rng_eastern[2] == rng_berlin[2]
    Out[489]: True

Operations between `Series` in different time zones will yield UTC `Series`, aligning the data on the UTC timestamps:

    In [490]: ts_utc = pd.Series(range(3), pd.date_range("20130101", periods=3, tz="UTC"))

    In [491]: eastern = ts_utc.tz_convert("US/Eastern")

    In [492]: berlin = ts_utc.tz_convert("Europe/Berlin")

    In [493]: result = eastern + berlin

    In [494]: result
    Out[494]:
    2013-01-01 00:00:00+00:00    0
    2013-01-02 00:00:00+00:00    2
    2013-01-03 00:00:00+00:00    4
    dtype: int64

    In [495]: result.index
    Out[495]:
    DatetimeIndex(['2013-01-01 00:00:00+00:00', '2013-01-02 00:00:00+00:00',
                   '2013-01-03 00:00:00+00:00'],
                  dtype='datetime64[us, UTC]', freq=None)

To remove time zone information, use `tz_localize(None)` or `tz_convert(None)`. `tz_localize(None)` will remove the time zone yielding the local time representation. `tz_convert(None)` will remove the time zone after converting to UTC time.

    In [496]: didx = pd.date_range(start="2014-08-01 09:00", freq="h", periods=3, tz="US/Eastern")

    In [497]: didx
    Out[497]:
    DatetimeIndex(['2014-08-01 09:00:00-04:00', '2014-08-01 10:00:00-04:00',
                   '2014-08-01 11:00:00-04:00'],
                  dtype='datetime64[us, US/Eastern]', freq='h')

    In [498]: didx.tz_localize(None)
    Out[498]:
    DatetimeIndex(['2014-08-01 09:00:00', '2014-08-01 10:00:00',
                   '2014-08-01 11:00:00'],
                  dtype='datetime64[us]', freq=None)

    In [499]: didx.tz_convert(None)
    Out[499]:
    DatetimeIndex(['2014-08-01 13:00:00', '2014-08-01 14:00:00',
                   '2014-08-01 15:00:00'],
                  dtype='datetime64[us]', freq='h')

    # tz_convert(None) is identical to tz_convert('UTC').tz_localize(None)
    In [500]: didx.tz_convert("UTC").tz_localize(None)
    Out[500]:
    DatetimeIndex(['2014-08-01 13:00:00', '2014-08-01 14:00:00',
                   '2014-08-01 15:00:00'],
                  dtype='datetime64[us]', freq=None)

### Fold

For ambiguous times, pandas supports explicitly specifying the keyword-only fold argument. Due to daylight saving time, one wall clock time can occur twice when shifting from summer to winter time; fold describes whether the datetime-like corresponds to the first (0) or the second time (1) the wall clock hits the ambiguous time. Fold is supported only for constructing from naive `datetime.datetime` (see datetime documentation for details) or from `Timestamp` or for constructing from components (see below). Fold is supported with `zoneinfo` and `dateutil` timezones. We recommend using `Timestamp.tz_localize()` when localizing ambiguous datetimes if you need direct control over how they are handled.

    In [501]: pd.Timestamp(
       .....:     datetime.datetime(2019, 10, 27, 1, 30, 0, 0),
       .....:     tz="dateutil/Europe/London",
       .....:     fold=0,
       .....: )
       .....:
    Out[501]: Timestamp('2019-10-27 01:30:00+0100', tz='dateutil//usr/share/zoneinfo/Europe/London')

    In [502]: pd.Timestamp(
       .....:     year=2019,
       .....:     month=10,
       .....:     day=27,
       .....:     hour=1,
       .....:     minute=30,
       .....:     tz="dateutil/Europe/London",
       .....:     fold=1,
       .....: )
       .....:
    Out[502]: Timestamp('2019-10-27 01:30:00+0000', tz='dateutil//usr/share/zoneinfo/Europe/London')

### Ambiguous times when localizing

`tz_localize` may not be able to determine the UTC offset of a timestamp because daylight savings time (DST) in a local time zone causes some times to occur twice within one day (“clocks fall back”). The following options are available:

- `'raise'`: Raises a `ValueError` (the default behavior)

- `'infer'`: Attempt to determine the correct offset based on the monotonicity of the timestamps

- `'NaT'`: Replaces ambiguous times with `NaT`

- `bool`: `True` represents a DST time, `False` represents non-DST time. An array-like of `bool` values is supported for a sequence of times.

    In [503]: rng_hourly = pd.DatetimeIndex(
       .....:     ["11/06/2011 00:00", "11/06/2011 01:00", "11/06/2011 01:00", "11/06/2011 02:00"]
       .....: )
       .....:

This will fail as there are ambiguous times (`'11/06/2011 01:00'`)

    In [504]: rng_hourly.tz_localize('US/Eastern')
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    Cell In[504], line 1
    ----> 1 rng_hourly.tz_localize('US/Eastern')

    File ~/work/pandas/pandas/pandas/core/indexes/datetimes.py:743, in DatetimeIndex.tz_localize(self, tz, ambiguous, nonexistent)
        601 """
        602 Localize tz-naive Datetime Array/Index to tz-aware Datetime Array/Index.
        603
       (...)    740 dtype: datetime64[ns, Europe/Warsaw]
        741 """  # noqa: E501
        742 freq = self._freq
    --> 743 arr = self._data.tz_localize(tz, ambiguous, nonexistent)
        744 result = type(self)._simple_new(arr, name=self.name)
        745 if timezones.is_utc(arr.tz) or (len(arr) == 1 and arr[0] is not NaT):
        746     # we can preserve freq
        747     # TODO: Also for fixed-offsets

    File ~/work/pandas/pandas/pandas/core/arrays/_mixins.py:82, in ravel_compat.<locals>.method(self, *args, **kwargs)
         79 @wraps(meth)
         80 def method(self, *args, **kwargs):
         81     if self.ndim == 1:
    ---> 82         return meth(self, *args, **kwargs)
         84     flags = self._ndarray.flags
         85     flat = self.ravel("K")

    File ~/work/pandas/pandas/pandas/core/arrays/datetimes.py:1118, in DatetimeArray.tz_localize(self, tz, ambiguous, nonexistent)
       1115     tz = timezones.maybe_get_tz(tz)
       1116     # Convert to UTC
    -> 1118     new_dates = tzconversion.tz_localize_to_utc(
       1119         self.asi8,
       1120         tz,
       1121         ambiguous=ambiguous,
       1122         nonexistent=nonexistent,
       1123         creso=self._creso,
       1124     )
       1125 new_dates_dt64 = new_dates.view(f"M8[{self.unit}]")
       1126 dtype = tz_to_dtype(tz, unit=self.unit)

    File pandas/_libs/tslibs/tzconversion.pyx:433, in pandas._libs.tslibs.tzconversion.tz_localize_to_utc()
    --> 433 'Could not get source, probably due dynamically evaluated source code.'
    ValueError: Cannot infer dst time from 2011-11-06 01:00:00, try using the 'ambiguous' argument

Handle these ambiguous times by specifying the following.

    In [505]: rng_hourly.tz_localize("US/Eastern", ambiguous="infer")
    Out[505]:
    DatetimeIndex(['2011-11-06 00:00:00-04:00', '2011-11-06 01:00:00-04:00',
                   '2011-11-06 01:00:00-05:00', '2011-11-06 02:00:00-05:00'],
                  dtype='datetime64[us, US/Eastern]', freq=None)

    In [506]: rng_hourly.tz_localize("US/Eastern", ambiguous="NaT")
    Out[506]:
    DatetimeIndex(['2011-11-06 00:00:00-04:00', 'NaT', 'NaT',
                   '2011-11-06 02:00:00-05:00'],
                  dtype='datetime64[us, US/Eastern]', freq=None)

    In [507]: rng_hourly.tz_localize("US/Eastern", ambiguous=[True, True, False, False])
    Out[507]:
    DatetimeIndex(['2011-11-06 00:00:00-04:00', '2011-11-06 01:00:00-04:00',
                   '2011-11-06 01:00:00-05:00', '2011-11-06 02:00:00-05:00'],
                  dtype='datetime64[us, US/Eastern]', freq=None)

### Nonexistent times when localizing

A DST transition may also shift the local time ahead by 1 hour creating nonexistent local times (“clocks spring forward”). The behavior of localizing a timeseries with nonexistent times can be controlled by the `nonexistent` argument. The following options are available:

- `'raise'`: Raises a `ValueError` (the default behavior)

- `'NaT'`: Replaces nonexistent times with `NaT`

- `'shift_forward'`: Shifts nonexistent times forward to the closest real time

- `'shift_backward'`: Shifts nonexistent times backward to the closest real time

- timedelta object: Shifts nonexistent times by the timedelta duration

    In [508]: dti = pd.date_range(start="2015-03-29 02:30:00", periods=3, freq="h")

    # 2:30 is a nonexistent time

Localization of nonexistent times will raise an error by default.

    In [509]: dti.tz_localize('Europe/Warsaw')
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    Cell In[509], line 1
    ----> 1 dti.tz_localize('Europe/Warsaw')

    File ~/work/pandas/pandas/pandas/core/indexes/datetimes.py:743, in DatetimeIndex.tz_localize(self, tz, ambiguous, nonexistent)
        601 """
        602 Localize tz-naive Datetime Array/Index to tz-aware Datetime Array/Index.
        603
       (...)    740 dtype: datetime64[ns, Europe/Warsaw]
        741 """  # noqa: E501
        742 freq = self._freq
    --> 743 arr = self._data.tz_localize(tz, ambiguous, nonexistent)
        744 result = type(self)._simple_new(arr, name=self.name)
        745 if timezones.is_utc(arr.tz) or (len(arr) == 1 and arr[0] is not NaT):
        746     # we can preserve freq
        747     # TODO: Also for fixed-offsets

    File ~/work/pandas/pandas/pandas/core/arrays/_mixins.py:82, in ravel_compat.<locals>.method(self, *args, **kwargs)
         79 @wraps(meth)
         80 def method(self, *args, **kwargs):
         81     if self.ndim == 1:
    ---> 82         return meth(self, *args, **kwargs)
         84     flags = self._ndarray.flags
         85     flat = self.ravel("K")

    File ~/work/pandas/pandas/pandas/core/arrays/datetimes.py:1118, in DatetimeArray.tz_localize(self, tz, ambiguous, nonexistent)
       1115     tz = timezones.maybe_get_tz(tz)
       1116     # Convert to UTC
    -> 1118     new_dates = tzconversion.tz_localize_to_utc(
       1119         self.asi8,
       1120         tz,
       1121         ambiguous=ambiguous,
       1122         nonexistent=nonexistent,
       1123         creso=self._creso,
       1124     )
       1125 new_dates_dt64 = new_dates.view(f"M8[{self.unit}]")
       1126 dtype = tz_to_dtype(tz, unit=self.unit)

    File pandas/_libs/tslibs/tzconversion.pyx:519, in pandas._libs.tslibs.tzconversion.tz_localize_to_utc()
    --> 519 'Could not get source, probably due dynamically evaluated source code.'
    ValueError: 2015-03-29 02:30:00 is a nonexistent time due to daylight savings time. Try using the 'nonexistent' argument.

Transform nonexistent times to `NaT` or shift the times.

    In [510]: dti
    Out[510]:
    DatetimeIndex(['2015-03-29 02:30:00', '2015-03-29 03:30:00',
                   '2015-03-29 04:30:00'],
                  dtype='datetime64[us]', freq='h')

    In [511]: dti.tz_localize("Europe/Warsaw", nonexistent="shift_forward")
    Out[511]:
    DatetimeIndex(['2015-03-29 03:00:00+02:00', '2015-03-29 03:30:00+02:00',
                   '2015-03-29 04:30:00+02:00'],
                  dtype='datetime64[us, Europe/Warsaw]', freq=None)

    In [512]: dti.tz_localize("Europe/Warsaw", nonexistent="shift_backward")
    Out[512]:
    DatetimeIndex(['2015-03-29 01:59:59.999999+01:00',
                          '2015-03-29 03:30:00+02:00',
                          '2015-03-29 04:30:00+02:00'],
                  dtype='datetime64[us, Europe/Warsaw]', freq=None)

    In [513]: dti.tz_localize("Europe/Warsaw", nonexistent=pd.Timedelta(1, unit="h"))
    Out[513]:
    DatetimeIndex(['2015-03-29 03:30:00+02:00', '2015-03-29 03:30:00+02:00',
                   '2015-03-29 04:30:00+02:00'],
                  dtype='datetime64[us, Europe/Warsaw]', freq=None)

    In [514]: dti.tz_localize("Europe/Warsaw", nonexistent="NaT")
    Out[514]:
    DatetimeIndex(['NaT', '2015-03-29 03:30:00+02:00',
                   '2015-03-29 04:30:00+02:00'],
                  dtype='datetime64[us, Europe/Warsaw]', freq=None)

### Time zone Series operations

A `Series` with time zone naive values is represented with a dtype of `datetime64[us]`.

    In [515]: s_naive = pd.Series(pd.date_range("20130101", periods=3))

    In [516]: s_naive
    Out[516]:
    0   2013-01-01
    1   2013-01-02
    2   2013-01-03
    dtype: datetime64[us]

A `Series` with a time zone aware values is represented with a dtype of `datetime64[us, tz]` where `tz` is the time zone

    In [517]: s_aware = pd.Series(pd.date_range("20130101", periods=3, tz="US/Eastern"))

    In [518]: s_aware
    Out[518]:
    0   2013-01-01 00:00:00-05:00
    1   2013-01-02 00:00:00-05:00
    2   2013-01-03 00:00:00-05:00
    dtype: datetime64[us, US/Eastern]

Both of these `Series` time zone information can be manipulated via the `.dt` accessor, see the dt accessor section.

For example, to localize and convert a naive stamp to time zone aware.

    In [519]: s_naive.dt.tz_localize("UTC").dt.tz_convert("US/Eastern")
    Out[519]:
    0   2012-12-31 19:00:00-05:00
    1   2013-01-01 19:00:00-05:00
    2   2013-01-02 19:00:00-05:00
    dtype: datetime64[us, US/Eastern]

Time zone information can also be manipulated using the `astype` method. This method can convert between different timezone-aware dtypes.

    # convert to a new time zone
    In [520]: s_aware.astype("datetime64[us, CET]")
    Out[520]:
    0   2013-01-01 06:00:00+01:00
    1   2013-01-02 06:00:00+01:00
    2   2013-01-03 06:00:00+01:00
    dtype: datetime64[us, CET]

Note

Using `Series.to_numpy()` on a `Series`, returns a NumPy array of the data. NumPy does not currently support time zones (even though it is printing in the local time zone!), therefore an object array of Timestamps is returned for time zone aware data:

    In [521]: s_naive.to_numpy()
    Out[521]:
    array(['2013-01-01T00:00:00.000000', '2013-01-02T00:00:00.000000',
           '2013-01-03T00:00:00.000000'], dtype='datetime64[us]')

    In [522]: s_aware.to_numpy()
    Out[522]:
    array([Timestamp('2013-01-01 00:00:00-0500', tz='US/Eastern'),
           Timestamp('2013-01-02 00:00:00-0500', tz='US/Eastern'),
           Timestamp('2013-01-03 00:00:00-0500', tz='US/Eastern')],
          dtype=object)

This object-dtype default is significantly slower and more memory-intensive than a native `datetime64` array. For better performance, pass an explicit `dtype`:

    In [523]: s_aware.to_numpy(dtype="datetime64[us]")
    Out[523]:
    array(['2013-01-01T05:00:00.000000', '2013-01-02T05:00:00.000000',
           '2013-01-03T05:00:00.000000'], dtype='datetime64[us]')

This converts to UTC and drops the timezone, returning a native `datetime64` array. If you need to preserve per-element timezone information (e.g. when converting back to a Series), use `dtype=object` explicitly:

    In [524]: pd.Series(s_aware.to_numpy(dtype=object))
    Out[524]:
    0   2013-01-01 00:00:00-05:00
    1   2013-01-02 00:00:00-05:00
    2   2013-01-03 00:00:00-05:00
    dtype: datetime64[us, US/Eastern]