Skip to main content
`Ctrl`+`K`

# Version 0.18.0 (March 13, 2016)

This is a major release from 0.17.1 and includes a small number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. We recommend that all users upgrade to this version.

Warning

pandas >= 0.18.0 no longer supports compatibility with Python version 2.6 and 3.3 (GH 7718, GH 11273)

Warning

`numexpr` version 2.4.4 will now show a warning and not be used as a computation back-end for pandas because of some buggy behavior. This does not affect other versions (>= 2.1 and >= 2.4.6). (GH 12489)

Highlights include:

- Moving and expanding window functions are now methods on Series and DataFrame, similar to `.groupby`, see here.

- Adding support for a `RangeIndex` as a specialized form of the `Int64Index` for memory savings, see here.

- API breaking change to the `.resample` method to make it more `.groupby` like, see here.

- Removal of support for positional indexing with floats, which was deprecated since 0.14.0. This will now raise a `TypeError`, see here.

- The `.to_xarray()` function has been added for compatibility with the xarray package, see here.

- The `read_sas` function has been enhanced to read `sas7bdat` files, see here.

- Addition of the .str.extractall() method, and API changes to the .str.extract() method and .str.cat() method.

- `pd.test()` top-level nose test runner is available (GH 4327).

Check the API Changes and deprecations before updating.

## New features

### Window functions are now methods

Window functions have been refactored to be methods on `Series/DataFrame` objects, rather than top-level functions, which are now deprecated. This allows these window-type functions, to have a similar API to that of `.groupby`. See the full documentation here (GH 11603, GH 12373)

    In [1]: np.random.seed(1234)

    In [2]: df = pd.DataFrame({'A': range(10), 'B': np.random.randn(10)})

    In [3]: df
    Out[3]:
       A         B
    0  0  0.471435
    1  1 -1.190976
    2  2  1.432707
    3  3 -0.312652
    4  4 -0.720589
    5  5  0.887163
    6  6  0.859588
    7  7 -0.636524
    8  8  0.015696
    9  9 -2.242685

Previous behavior:

    In [8]: pd.rolling_mean(df, window=3)
            FutureWarning: pd.rolling_mean is deprecated for DataFrame and will be removed in a future version, replace with
                           DataFrame.rolling(window=3,center=False).mean()
    Out[8]:
        A         B
    0 NaN       NaN
    1 NaN       NaN
    2   1  0.237722
    3   2 -0.023640
    4   3  0.133155
    5   4 -0.048693
    6   5  0.342054
    7   6  0.370076
    8   7  0.079587
    9   8 -0.954504

New behavior:

    In [4]: r = df.rolling(window=3)

These show a descriptive repr

    In [5]: r
    Out[5]: Rolling [window=3,center=False,method=single]

with tab-completion of available methods and properties.

    In [9]: r.<TAB>  # noqa E225, E999
    r.A           r.agg         r.apply       r.count       r.exclusions  r.max         r.median      r.name        r.skew        r.sum
    r.B           r.aggregate   r.corr        r.cov         r.kurt        r.mean        r.min         r.quantile    r.std         r.var

The methods operate on the `Rolling` object itself

    In [6]: r.mean()
    Out[6]:
         A         B
    0  NaN       NaN
    1  NaN       NaN
    2  1.0  0.237722
    3  2.0 -0.023640
    4  3.0  0.133155
    5  4.0 -0.048693
    6  5.0  0.342054
    7  6.0  0.370076
    8  7.0  0.079587
    9  8.0 -0.954504

They provide getitem accessors

    In [7]: r['A'].mean()
    Out[7]:
    0    NaN
    1    NaN
    2    1.0
    3    2.0
    4    3.0
    5    4.0
    6    5.0
    7    6.0
    8    7.0
    9    8.0
    Name: A, dtype: float64

And multiple aggregations

    In [8]: r.agg({'A': ['mean', 'std'],
       ...:        'B': ['mean', 'std']})
       ...:
    Out[8]:
         A              B
      mean  std      mean       std
    0  NaN  NaN       NaN       NaN
    1  NaN  NaN       NaN       NaN
    2  1.0  1.0  0.237722  1.327364
    3  2.0  1.0 -0.023640  1.335505
    4  3.0  1.0  0.133155  1.143778
    5  4.0  1.0 -0.048693  0.835747
    6  5.0  1.0  0.342054  0.920379
    7  6.0  1.0  0.370076  0.871850
    8  7.0  1.0  0.079587  0.750099
    9  8.0  1.0 -0.954504  1.162285

### Changes to rename

`Series.rename` and `NDFrame.rename_axis` can now take a scalar or list-like argument for altering the Series or axis name, in addition to their old behaviors of altering labels. (GH 9494, GH 11965)

    In [9]: s = pd.Series(np.random.randn(5))

    In [10]: s.rename('newname')
    Out[10]:
    0    1.150036
    1    0.991946
    2    0.953324
    3   -2.021255
    4   -0.334077
    Name: newname, dtype: float64

    In [11]: df = pd.DataFrame(np.random.randn(5, 2))

    In [12]: (df.rename_axis("indexname")
       ....:    .rename_axis("columns_name", axis="columns"))
       ....:
    Out[12]:
    columns_name         0         1
    indexname
    0             0.002118  0.405453
    1             0.289092  1.321158
    2            -1.546906 -0.202646
    3            -0.655969  0.193421
    4             0.553439  1.318152

The new functionality works well in method chains. Previously these methods only accepted functions or dicts mapping a label to a new label. This continues to work as before for function or dict-like values.

### Range Index

A `RangeIndex` has been added to the `Int64Index` sub-classes to support a memory saving alternative for common use cases. This has a similar implementation to the python `range` object (`xrange` in python 2), in that it only stores the start, stop, and step values for the index. It will transparently interact with the user API, converting to `Int64Index` if needed.

This will now be the default constructed index for `NDFrame` objects, rather than previous an `Int64Index`. (GH 939, GH 12070, GH 12071, GH 12109, GH 12888)

Previous behavior:

    In [3]: s = pd.Series(range(1000))

    In [4]: s.index
    Out[4]:
    Int64Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,
                ...
                990, 991, 992, 993, 994, 995, 996, 997, 998, 999], dtype='int64', length=1000)

    In [6]: s.index.nbytes
    Out[6]: 8000

New behavior:

    In [13]: s = pd.Series(range(1000))

    In [14]: s.index
    Out[14]: RangeIndex(start=0, stop=1000, step=1)

    In [15]: s.index.nbytes
    Out[15]: 132

### Changes to str.extract

The .str.extract method takes a regular expression with capture groups, finds the first match in each subject string, and returns the contents of the capture groups (GH 11386).

In v0.18.0, the `expand` argument was added to `extract`.

- `expand=False`: it returns a `Series`, `Index`, or `DataFrame`, depending on the subject and regular expression pattern (same behavior as pre-0.18.0).

- `expand=True`: it always returns a `DataFrame`, which is more consistent and less confusing from the perspective of a user.

Currently the default is `expand=None` which gives a `FutureWarning` and uses `expand=False`. To avoid this warning, please explicitly specify `expand`.

    In [1]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=None)
    FutureWarning: currently extract(expand=None) means expand=False (return Index/Series/DataFrame)
    but in a future version of pandas this will be changed to expand=True (return DataFrame)

    Out[1]:
    0      1
    1      2
    2    NaN
    dtype: object

Extracting a regular expression with one group returns a Series if `expand=False`.

    In [16]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=False)
    Out[16]:
    0      1
    1      2
    2    NaN
    dtype: str

It returns a `DataFrame` with one column if `expand=True`.

    In [17]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=True)
    Out[17]:
         0
    0    1
    1    2
    2  NaN

Calling on an `Index` with a regex with exactly one capture group returns an `Index` if `expand=False`.

    In [18]: s = pd.Series(["a1", "b2", "c3"], ["A11", "B22", "C33"])

    In [19]: s.index
    Out[19]: Index(['A11', 'B22', 'C33'], dtype='str')

    In [20]: s.index.str.extract("(?P<letter>[a-zA-Z])", expand=False)
    Out[20]: Index(['A', 'B', 'C'], dtype='str', name='letter')

It returns a `DataFrame` with one column if `expand=True`.

    In [21]: s.index.str.extract("(?P<letter>[a-zA-Z])", expand=True)
    Out[21]:
      letter
    0      A
    1      B
    2      C

Calling on an `Index` with a regex with more than one capture group raises `ValueError` if `expand=False`.

    >>> s.index.str.extract("(?P<letter>[a-zA-Z])([0-9]+)", expand=False)
    ValueError: only one regex group is supported with Index

It returns a `DataFrame` if `expand=True`.

    In [22]: s.index.str.extract("(?P<letter>[a-zA-Z])([0-9]+)", expand=True)
    Out[22]:
      letter   1
    0      A  11
    1      B  22
    2      C  33

In summary, `extract(expand=True)` always returns a `DataFrame` with a row for every subject string, and a column for every capture group.

### Addition of str.extractall

The .str.extractall method was added (GH 11386). Unlike `extract`, which returns only the first match.

    In [23]: s = pd.Series(["a1a2", "b1", "c1"], ["A", "B", "C"])

    In [24]: s
    Out[24]:
    A    a1a2
    B      b1
    C      c1
    dtype: str

    In [25]: s.str.extract(r"(?P<letter>[ab])(?P<digit>\d)", expand=False)
    Out[25]:
      letter digit
    A      a     1
    B      b     1
    C    NaN   NaN

The `extractall` method returns all matches.

    In [26]: s.str.extractall(r"(?P<letter>[ab])(?P<digit>\d)")
    Out[26]:
            letter digit
      match
    A 0          a     1
      1          a     2
    B 0          b     1

### Changes to str.cat

The method `.str.cat()` concatenates the members of a `Series`. Before, if `NaN` values were present in the Series, calling `.str.cat()` on it would return `NaN`, unlike the rest of the `Series.str.*` API. This behavior has been amended to ignore `NaN` values by default. (GH 11435).

A new, friendlier `ValueError` is added to protect against the mistake of supplying the `sep` as an arg, rather than as a kwarg. (GH 11334).

    In [27]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(sep=' ')
    Out[27]: 'a b c'

    In [28]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(sep=' ', na_rep='?')
    Out[28]: 'a b ? c'

    In [2]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(' ')
    ValueError: Did you mean to supply a ``sep`` keyword?

### Datetimelike rounding

`DatetimeIndex`, `Timestamp`, `TimedeltaIndex`, `Timedelta` have gained the `.round()`, `.floor()` and `.ceil()` method for datetimelike rounding, flooring and ceiling. (GH 4314, GH 11963)

Naive datetimes

    In [29]: dr = pd.date_range('20130101 09:12:56.1234', periods=3)

    In [30]: dr
    Out[30]:
    DatetimeIndex(['2013-01-01 09:12:56.123400', '2013-01-02 09:12:56.123400',
                   '2013-01-03 09:12:56.123400'],
                  dtype='datetime64[us]', freq='D')

    In [31]: dr.round('s')
    Out[31]:
    DatetimeIndex(['2013-01-01 09:12:56', '2013-01-02 09:12:56',
                   '2013-01-03 09:12:56'],
                  dtype='datetime64[us]', freq=None)

    # Timestamp scalar
    In [32]: dr[0]
    Out[32]: Timestamp('2013-01-01 09:12:56.123400')

    In [33]: dr[0].round('10s')
    Out[33]: Timestamp('2013-01-01 09:13:00')

Tz-aware are rounded, floored and ceiled in local times

    In [34]: dr = dr.tz_localize('US/Eastern')

    In [35]: dr
    Out[35]:
    DatetimeIndex(['2013-01-01 09:12:56.123400-05:00',
                   '2013-01-02 09:12:56.123400-05:00',
                   '2013-01-03 09:12:56.123400-05:00'],
                  dtype='datetime64[us, US/Eastern]', freq=None)

    In [36]: dr.round('s')
    Out[36]:
    DatetimeIndex(['2013-01-01 09:12:56-05:00', '2013-01-02 09:12:56-05:00',
                   '2013-01-03 09:12:56-05:00'],
                  dtype='datetime64[us, US/Eastern]', freq=None)

Timedeltas

    In [37]: t = pd.timedelta_range('1 days 2 hr 13 min 45 us', periods=3, freq='d')

    In [38]: t
    Out[38]:
    TimedeltaIndex(['1 days 02:13:00.000045', '2 days 02:13:00.000045',
                    '3 days 02:13:00.000045'],
                   dtype='timedelta64[ns]', freq='D')

    In [39]: t.round('10min')
    Out[39]:
    TimedeltaIndex(['1 days 02:10:00', '2 days 02:10:00',
                    '3 days 02:10:00'],
                   dtype='timedelta64[ns]', freq=None)

    # Timedelta scalar
    In [40]: t[0]
    Out[40]: Timedelta('1 days 02:13:00.000045')

    In [41]: t[0].round('2h')
    Out[41]: Timedelta('1 days 02:00:00')

In addition, `.round()`, `.floor()` and `.ceil()` will be available through the `.dt` accessor of `Series`.

    In [37]: s = pd.Series(dr)

    In [38]: s
    Out[38]:
    0   2013-01-01 09:12:56.123400-05:00
    1   2013-01-02 09:12:56.123400-05:00
    2   2013-01-03 09:12:56.123400-05:00
    dtype: datetime64[us, US/Eastern]

    In [39]: s.dt.round('D')
    Out[39]:
    0   2013-01-01 00:00:00-05:00
    1   2013-01-02 00:00:00-05:00
    2   2013-01-03 00:00:00-05:00
    dtype: datetime64[us, US/Eastern]

### Formatting of integers in FloatIndex

Integers in `FloatIndex`, e.g. 1., are now formatted with a decimal point and a `0` digit, e.g. `1.0` (GH 11713) This change not only affects the display to the console, but also the output of IO methods like `.to_csv` or `.to_html`.

Previous behavior:

    In [2]: s = pd.Series([1, 2, 3], index=np.arange(3.))

    In [3]: s
    Out[3]:
    0    1
    1    2
    2    3
    dtype: int64

    In [4]: s.index
    Out[4]: Float64Index([0.0, 1.0, 2.0], dtype='float64')

    In [5]: print(s.to_csv(path=None))
    0,1
    1,2
    2,3

New behavior:

    In [40]: s = pd.Series([1, 2, 3], index=np.arange(3.))

    In [41]: s
    Out[41]:
    0.0    1
    1.0    2
    2.0    3
    dtype: int64

    In [42]: s.index
    Out[42]: Index([0.0, 1.0, 2.0], dtype='float64')

    In [43]: print(s.to_csv(path_or_buf=None, header=False))
    0.0,1
    1.0,2
    2.0,3

### Changes to dtype assignment behaviors

When a DataFrame’s slice is updated with a new slice of the same dtype, the dtype of the DataFrame will now remain the same. (GH 10503)

Previous behavior:

    In [5]: df = pd.DataFrame({'a': [0, 1, 1],
                               'b': pd.Series([100, 200, 300], dtype='uint32')})

    In [7]: df.dtypes
    Out[7]:
    a     int64
    b    uint32
    dtype: object

    In [8]: ix = df['a'] == 1

    In [9]: df.loc[ix, 'b'] = df.loc[ix, 'b']

    In [11]: df.dtypes
    Out[11]:
    a    int64
    b    int64
    dtype: object

New behavior:

    In [44]: df = pd.DataFrame({'a': [0, 1, 1],
       ....:                    'b': pd.Series([100, 200, 300], dtype='uint32')})
       ....:

    In [45]: df.dtypes
    Out[45]:
    a     int64
    b    uint32
    dtype: object

    In [46]: ix = df['a'] == 1

    In [47]: df.loc[ix, 'b'] = df.loc[ix, 'b']

    In [48]: df.dtypes
    Out[48]:
    a     int64
    b    uint32
    dtype: object

When a DataFrame’s integer slice is partially updated with a new slice of floats that could potentially be down-casted to integer without losing precision, the dtype of the slice will be set to float instead of integer.

Previous behavior:

    In [4]: df = pd.DataFrame(np.array(range(1,10)).reshape(3,3),
                              columns=list('abc'),
                              index=[[4,4,8], [8,10,12]])

    In [5]: df
    Out[5]:
          a  b  c
    4 8   1  2  3
      10  4  5  6
    8 12  7  8  9

    In [7]: df.ix[4, 'c'] = np.array([0., 1.])

    In [8]: df
    Out[8]:
          a  b  c
    4 8   1  2  0
      10  4  5  1
    8 12  7  8  9

New behavior:

    In [49]: df = pd.DataFrame(np.array(range(1,10)).reshape(3,3),
       ....:                   columns=list('abc'),
       ....:                   index=[[4,4,8], [8,10,12]])
       ....:

    In [50]: df
    Out[50]:
          a  b  c
    4 8   1  2  3
      10  4  5  6
    8 12  7  8  9

    In [51]: df.loc[4, 'c'] = np.array([0., 1.])

    In [52]: df
    Out[52]:
          a  b  c
    4 8   1  2  0
      10  4  5  1
    8 12  7  8  9

### Method to_xarray

In a future version of pandas, we will be deprecating `Panel` and other > 2 ndim objects. In order to provide for continuity, all `NDFrame` objects have gained the `.to_xarray()` method in order to convert to `xarray` objects, which has a pandas-like interface for > 2 ndim. (GH 11972)

See the xarray full-documentation here.

    In [1]: p = Panel(np.arange(2*3*4).reshape(2,3,4))

    In [2]: p.to_xarray()
    Out[2]:
    <xarray.DataArray (items: 2, major_axis: 3, minor_axis: 4)>
    array([[[ 0,  1,  2,  3],
            [ 4,  5,  6,  7],
            [ 8,  9, 10, 11]],

           [[12, 13, 14, 15],
            [16, 17, 18, 19],
            [20, 21, 22, 23]]])
    Coordinates:
      * items       (items) int64 0 1
      * major_axis  (major_axis) int64 0 1 2
      * minor_axis  (minor_axis) int64 0 1 2 3

### Latex representation

`DataFrame` has gained a `._repr_latex_()` method in order to allow for conversion to latex in a ipython/jupyter notebook using nbconvert. (GH 11778)

Note that this must be activated by setting the option `pd.display.latex.repr=True` (GH 12182)

For example, if you have a jupyter notebook you plan to convert to latex using nbconvert, place the statement `pd.display.latex.repr=True` in the first cell to have the contained DataFrame output also stored as latex.

The options `display.latex.escape` and `display.latex.longtable` have also been added to the configuration and are used automatically by the `to_latex` method. See the available options docs for more info.

### `pd.read_sas()` changes

`read_sas` has gained the ability to read SAS7BDAT files, including compressed files. The files can be read in entirety, or incrementally. For full details see here. (GH 4052)

### Other enhancements

- Handle truncated floats in SAS xport files (GH 11713)

- Added option to hide index in `Series.to_string` (GH 11729)

- `read_excel` now supports s3 urls of the format `s3://bucketname/filename` (GH 11447)

- add support for `AWS_S3_HOST` env variable when reading from s3 (GH 12198)

- A simple version of `Panel.round()` is now implemented (GH 11763)

- For Python 3.x, `round(DataFrame)`, `round(Series)`, `round(Panel)` will work (GH 11763)

- `sys.getsizeof(obj)` returns the memory usage of a pandas object, including the values it contains (GH 11597)

- `Series` gained an `is_unique` attribute (GH 11946)

- `DataFrame.quantile` and `Series.quantile` now accept `interpolation` keyword (GH 10174).

- Added `DataFrame.style.format` for more flexible formatting of cell values (GH 11692)

- `DataFrame.select_dtypes` now allows the `np.float16` type code (GH 11990)

- `pivot_table()` now accepts most iterables for the `values` parameter (GH 12017)

- Added Google `BigQuery` service account authentication support, which enables authentication on remote servers. (GH 11881, GH 12572). For further details see here

- `HDFStore` is now iterable: `for k in store` is equivalent to `for k in store.keys()` (GH 12221).

- Add missing methods/fields to `.dt` for `Period` (GH 8848)

- The entire code base has been `PEP`-ified (GH 12096)

## Backwards incompatible API changes

- the leading white spaces have been removed from the output of `.to_string(index=False)` method (GH 11833)

- the `out` parameter has been removed from the `Series.round()` method. (GH 11763)

- `DataFrame.round()` leaves non-numeric columns unchanged in its return, rather than raises. (GH 11885)

- `DataFrame.head(0)` and `DataFrame.tail(0)` return empty frames, rather than `self`. (GH 11937)

- `Series.head(0)` and `Series.tail(0)` return empty series, rather than `self`. (GH 11937)

- `to_msgpack` and `read_msgpack` encoding now defaults to `'utf-8'`. (GH 12170)

- the order of keyword arguments to text file parsing functions (`.read_csv()`, `.read_table()`, `.read_fwf()`) changed to group related arguments. (GH 11555)

- `NaTType.isoformat` now returns the string `'NaT` to allow the result to be passed to the constructor of `Timestamp`. (GH 12300)

### NaT and Timedelta operations

`NaT` and `Timedelta` have expanded arithmetic operations, which are extended to `Series` arithmetic where applicable. Operations defined for `datetime64[ns]` or `timedelta64[ns]` are now also defined for `NaT` (GH 11564).

`NaT` now supports arithmetic operations with integers and floats.

    In [53]: pd.NaT * 1
    Out[53]: NaT

    In [54]: pd.NaT * 1.5
    Out[54]: NaT

    In [55]: pd.NaT / 2
    Out[55]: NaT

    In [56]: pd.NaT * np.nan
    Out[56]: NaT

`NaT` defines more arithmetic operations with `datetime64[ns]` and `timedelta64[ns]`.

    In [57]: pd.NaT / pd.NaT
    Out[57]: nan

    In [58]: pd.Timedelta('1s') / pd.NaT
    Out[58]: nan

`NaT` may represent either a `datetime64[ns]` null or a `timedelta64[ns]` null. Given the ambiguity, it is treated as a `timedelta64[ns]`, which allows more operations to succeed.

    In [59]: pd.NaT + pd.NaT
    Out[59]: NaT

    # same as
    In [60]: pd.Timedelta('1s') + pd.Timedelta('1s')
    Out[60]: Timedelta('0 days 00:00:02')

as opposed to

    In [3]: pd.Timestamp('19900315') + pd.Timestamp('19900315')
    TypeError: unsupported operand type(s) for +: 'Timestamp' and 'Timestamp'

However, when wrapped in a `Series` whose `dtype` is `datetime64[ns]` or `timedelta64[ns]`, the `dtype` information is respected.

    In [1]: pd.Series([pd.NaT], dtype='<M8[ns]') + pd.Series([pd.NaT], dtype='<M8[ns]')
    TypeError: can only operate on a datetimes for subtraction,
               but the operator [__add__] was passed

    In [61]: pd.Series([pd.NaT], dtype='<m8[ns]') + pd.Series([pd.NaT], dtype='<m8[ns]')
    Out[61]:
    0   NaT
    dtype: timedelta64[ns]

`Timedelta` division by `floats` now works.

    In [62]: pd.Timedelta('1s') / 2.0
    Out[62]: Timedelta('0 days 00:00:00.500000')

Subtraction by `Timedelta` in a `Series` by a `Timestamp` works (GH 11925)

    In [63]: ser = pd.Series(pd.timedelta_range('1 day', periods=3))

    In [64]: ser
    Out[64]:
    0   1 days
    1   2 days
    2   3 days
    dtype: timedelta64[us]

    In [65]: pd.Timestamp('2012-01-01') - ser
    Out[65]:
    0   2011-12-31
    1   2011-12-30
    2   2011-12-29
    dtype: datetime64[us]

`NaT.isoformat()` now returns `'NaT'`. This change allows `pd.Timestamp` to rehydrate any timestamp like object from its isoformat (GH 12300).

### Changes to msgpack

Forward incompatible changes in `msgpack` writing format were made over 0.17.0 and 0.18.0; older versions of pandas cannot read files packed by newer versions (GH 12129, GH 10527)

Bugs in `to_msgpack` and `read_msgpack` introduced in 0.17.0 and fixed in 0.18.0, caused files packed in Python 2 unreadable by Python 3 (GH 12142). The following table describes the backward and forward compat of msgpacks.

Warning

Packed with | Can be unpacked with
--- | ---
pre-0.17 / Python 2 | any
pre-0.17 / Python 3 | any
0.17 / Python 2 | ==0.17 / Python 2 >=0.18 / any Python
0.17 / Python 3 | >=0.18 / any Python
0.18 | >= 0.18

0.18.0 is backward-compatible for reading files packed by older versions, except for files packed with 0.17 in Python 2, in which case only they can only be unpacked in Python 2.

### Signature change for .rank

`Series.rank` and `DataFrame.rank` now have the same signature (GH 11759)

Previous signature

    In [3]: pd.Series([0,1]).rank(method='average', na_option='keep',
                                  ascending=True, pct=False)
    Out[3]:
    0    1
    1    2
    dtype: float64

    In [4]: pd.DataFrame([0,1]).rank(axis=0, numeric_only=None,
                                     method='average', na_option='keep',
                                     ascending=True, pct=False)
    Out[4]:
       0
    0  1
    1  2

New signature

    In [66]: pd.Series([0,1]).rank(axis=0, method='average', numeric_only=False,
       ....:                       na_option='keep', ascending=True, pct=False)
       ....:
    Out[66]:
    0    1.0
    1    2.0
    dtype: float64

    In [67]: pd.DataFrame([0,1]).rank(axis=0, method='average', numeric_only=False,
       ....:                          na_option='keep', ascending=True, pct=False)
       ....:
    Out[67]:
         0
    0  1.0
    1  2.0

### Bug in QuarterBegin with n=0

In previous versions, the behavior of the QuarterBegin offset was inconsistent depending on the date when the `n` parameter was 0. (GH 11406)

The general semantics of anchored offsets for `n=0` is to not move the date when it is an anchor point (e.g., a quarter start date), and otherwise roll forward to the next anchor point.

    In [68]: d = pd.Timestamp('2014-02-01')

    In [69]: d
    Out[69]: Timestamp('2014-02-01 00:00:00')

    In [70]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2)
    Out[70]: Timestamp('2014-02-01 00:00:00')

    In [71]: d + pd.offsets.QuarterBegin(n=0, startingMonth=1)
    Out[71]: Timestamp('2014-04-01 00:00:00')

For the `QuarterBegin` offset in previous versions, the date would be rolled backwards if date was in the same month as the quarter start date.

    In [3]: d = pd.Timestamp('2014-02-15')

    In [4]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2)
    Out[4]: Timestamp('2014-02-01 00:00:00')

This behavior has been corrected in version 0.18.0, which is consistent with other anchored offsets like `MonthBegin` and `YearBegin`.

    In [72]: d = pd.Timestamp('2014-02-15')

    In [73]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2)
    Out[73]: Timestamp('2014-05-01 00:00:00')

### Resample API

Like the change in the window functions API above, `.resample(...)` is changing to have a more groupby-like API. (GH 11732, GH 12702, GH 12202, GH 12332, GH 12334, GH 12348, GH 12448).

    In [74]: np.random.seed(1234)

    In [75]: df = pd.DataFrame(np.random.rand(10,4),
       ....:                   columns=list('ABCD'),
       ....:                   index=pd.date_range('2010-01-01 09:00:00',
       ....:                                       periods=10, freq='s'))
       ....:

    In [76]: df
    Out[76]:
                                A         B         C         D
    2010-01-01 09:00:00  0.191519  0.622109  0.437728  0.785359
    2010-01-01 09:00:01  0.779976  0.272593  0.276464  0.801872
    2010-01-01 09:00:02  0.958139  0.875933  0.357817  0.500995
    2010-01-01 09:00:03  0.683463  0.712702  0.370251  0.561196
    2010-01-01 09:00:04  0.503083  0.013768  0.772827  0.882641
    2010-01-01 09:00:05  0.364886  0.615396  0.075381  0.368824
    2010-01-01 09:00:06  0.933140  0.651378  0.397203  0.788730
    2010-01-01 09:00:07  0.316836  0.568099  0.869127  0.436173
    2010-01-01 09:00:08  0.802148  0.143767  0.704261  0.704581
    2010-01-01 09:00:09  0.218792  0.924868  0.442141  0.909316

Previous API:

You would write a resampling operation that immediately evaluates. If a `how` parameter was not provided, it would default to `how='mean'`.

    In [6]: df.resample('2s')
    Out[6]:
                             A         B         C         D
    2010-01-01 09:00:00  0.485748  0.447351  0.357096  0.793615
    2010-01-01 09:00:02  0.820801  0.794317  0.364034  0.531096
    2010-01-01 09:00:04  0.433985  0.314582  0.424104  0.625733
    2010-01-01 09:00:06  0.624988  0.609738  0.633165  0.612452
    2010-01-01 09:00:08  0.510470  0.534317  0.573201  0.806949

You could also specify a `how` directly

    In [7]: df.resample('2s', how='sum')
    Out[7]:
                             A         B         C         D
    2010-01-01 09:00:00  0.971495  0.894701  0.714192  1.587231
    2010-01-01 09:00:02  1.641602  1.588635  0.728068  1.062191
    2010-01-01 09:00:04  0.867969  0.629165  0.848208  1.251465
    2010-01-01 09:00:06  1.249976  1.219477  1.266330  1.224904
    2010-01-01 09:00:08  1.020940  1.068634  1.146402  1.613897

New API:

Now, you can write `.resample(..)` as a 2-stage operation like `.groupby(...)`, which yields a `Resampler`.

    In [77]: r = df.resample('2s')

    In [78]: r
    Out[78]: <pandas.core.resample.DatetimeIndexResampler object at 0x7faa32795210>

#### Downsampling

You can then use this object to perform operations. These are downsampling operations (going from a higher frequency to a lower one).

    In [79]: r.mean()
    Out[79]:
                                A         B         C         D
    2010-01-01 09:00:00  0.485748  0.447351  0.357096  0.793615
    2010-01-01 09:00:02  0.820801  0.794317  0.364034  0.531096
    2010-01-01 09:00:04  0.433985  0.314582  0.424104  0.625733
    2010-01-01 09:00:06  0.624988  0.609738  0.633165  0.612452
    2010-01-01 09:00:08  0.510470  0.534317  0.573201  0.806949

    In [80]: r.sum()
    Out[80]:
                                A         B         C         D
    2010-01-01 09:00:00  0.971495  0.894701  0.714192  1.587231
    2010-01-01 09:00:02  1.641602  1.588635  0.728068  1.062191
    2010-01-01 09:00:04  0.867969  0.629165  0.848208  1.251465
    2010-01-01 09:00:06  1.249976  1.219477  1.266330  1.224904
    2010-01-01 09:00:08  1.020940  1.068634  1.146402  1.613897

Furthermore, resample now supports `getitem` operations to perform the resample on specific columns.

    In [81]: r[['A','C']].mean()
    Out[81]:
                                A         C
    2010-01-01 09:00:00  0.485748  0.357096
    2010-01-01 09:00:02  0.820801  0.364034
    2010-01-01 09:00:04  0.433985  0.424104
    2010-01-01 09:00:06  0.624988  0.633165
    2010-01-01 09:00:08  0.510470  0.573201

and `.aggregate` type operations.

    In [82]: r.agg({'A' : 'mean', 'B' : 'sum'})
    Out[82]:
                                A         B
    2010-01-01 09:00:00  0.485748  0.894701
    2010-01-01 09:00:02  0.820801  1.588635
    2010-01-01 09:00:04  0.433985  0.629165
    2010-01-01 09:00:06  0.624988  1.219477
    2010-01-01 09:00:08  0.510470  1.068634

These accessors can of course, be combined

    In [83]: r[['A','B']].agg(['mean','sum'])
    Out[83]:
                                A                   B
                             mean       sum      mean       sum
    2010-01-01 09:00:00  0.485748  0.971495  0.447351  0.894701
    2010-01-01 09:00:02  0.820801  1.641602  0.794317  1.588635
    2010-01-01 09:00:04  0.433985  0.867969  0.314582  0.629165
    2010-01-01 09:00:06  0.624988  1.249976  0.609738  1.219477
    2010-01-01 09:00:08  0.510470  1.020940  0.534317  1.068634

#### Upsampling

Upsampling operations take you from a lower frequency to a higher frequency. These are now performed with the `Resampler` objects with `backfill()`, `ffill()`, `fillna()` and `asfreq()` methods.

    In [89]: s = pd.Series(np.arange(5, dtype='int64'),
                  index=pd.date_range('2010-01-01', periods=5, freq='Q'))

    In [90]: s
    Out[90]:
    2010-03-31    0
    2010-06-30    1
    2010-09-30    2
    2010-12-31    3
    2011-03-31    4
    Freq: Q-DEC, Length: 5, dtype: int64

Previously

    In [6]: s.resample('M', fill_method='ffill')
    Out[6]:
    2010-03-31    0
    2010-04-30    0
    2010-05-31    0
    2010-06-30    1
    2010-07-31    1
    2010-08-31    1
    2010-09-30    2
    2010-10-31    2
    2010-11-30    2
    2010-12-31    3
    2011-01-31    3
    2011-02-28    3
    2011-03-31    4
    Freq: M, dtype: int64

New API

    In [91]: s.resample('M').ffill()
    Out[91]:
    2010-03-31    0
    2010-04-30    0
    2010-05-31    0
    2010-06-30    1
    2010-07-31    1
    2010-08-31    1
    2010-09-30    2
    2010-10-31    2
    2010-11-30    2
    2010-12-31    3
    2011-01-31    3
    2011-02-28    3
    2011-03-31    4
    Freq: M, Length: 13, dtype: int64

Note

In the new API, you can either downsample OR upsample. The prior implementation would allow you to pass an aggregator function (like `mean`) even though you were upsampling, providing a bit of confusion.

#### Previous API will work but with deprecations

Warning

This new API for resample includes some internal changes for the prior-to-0.18.0 API, to work with a deprecation warning in most cases, as the resample operation returns a deferred object. We can intercept operations and just do what the (pre 0.18.0) API did (with a warning). Here is a typical use case:

    In [4]: r = df.resample('2s')

    In [6]: r*10
    pandas/tseries/resample.py:80: FutureWarning: .resample() is now a deferred operation
    use .resample(...).mean() instead of .resample(...)

    Out[6]:
                          A         B         C         D
    2010-01-01 09:00:00  4.857476  4.473507  3.570960  7.936154
    2010-01-01 09:00:02  8.208011  7.943173  3.640340  5.310957
    2010-01-01 09:00:04  4.339846  3.145823  4.241039  6.257326
    2010-01-01 09:00:06  6.249881  6.097384  6.331650  6.124518
    2010-01-01 09:00:08  5.104699  5.343172  5.732009  8.069486

However, getting and assignment operations directly on a `Resampler` will raise a `ValueError`:

    In [7]: r.iloc[0] = 5
    ValueError: .resample() is now a deferred operation
    use .resample(...).mean() instead of .resample(...)

There is a situation where the new API can not perform all the operations when using original code. This code is intending to resample every 2s, take the `mean` AND then take the `min` of those results.

    In [4]: df.resample('2s').min()
    Out[4]:
    A    0.433985
    B    0.314582
    C    0.357096
    D    0.531096
    dtype: float64

The new API will:

    In [84]: df.resample('2s').min()
    Out[84]:
                                A         B         C         D
    2010-01-01 09:00:00  0.191519  0.272593  0.276464  0.785359
    2010-01-01 09:00:02  0.683463  0.712702  0.357817  0.500995
    2010-01-01 09:00:04  0.364886  0.013768  0.075381  0.368824
    2010-01-01 09:00:06  0.316836  0.568099  0.397203  0.436173
    2010-01-01 09:00:08  0.218792  0.143767  0.442141  0.704581

The good news is the return dimensions will differ between the new API and the old API, so this should loudly raise an exception.

To replicate the original operation

    In [85]: df.resample('2s').mean().min()
    Out[85]:
    A    0.433985
    B    0.314582
    C    0.357096
    D    0.531096
    dtype: float64

### Changes to eval

In prior versions, new columns assignments in an `eval` expression resulted in an inplace change to the `DataFrame`. (GH 9297, GH 8664, GH 10486)

    In [86]: df = pd.DataFrame({'a': np.linspace(0, 10, 5), 'b': range(5)})

    In [87]: df
    Out[87]:
          a  b
    0   0.0  0
    1   2.5  1
    2   5.0  2
    3   7.5  3
    4  10.0  4

    In [12]: df.eval('c = a + b')
    FutureWarning: eval expressions containing an assignment currentlydefault to operating inplace.
    This will change in a future version of pandas, use inplace=True to avoid this warning.

    In [13]: df
    Out[13]:
          a  b     c
    0   0.0  0   0.0
    1   2.5  1   3.5
    2   5.0  2   7.0
    3   7.5  3  10.5
    4  10.0  4  14.0

In version 0.18.0, a new `inplace` keyword was added to choose whether the assignment should be done inplace or return a copy.

    In [88]: df
    Out[88]:
          a  b     c
    0   0.0  0   0.0
    1   2.5  1   3.5
    2   5.0  2   7.0
    3   7.5  3  10.5
    4  10.0  4  14.0

    In [89]: df.eval('d = c - b', inplace=False)
    Out[89]:
          a  b     c     d
    0   0.0  0   0.0   0.0
    1   2.5  1   3.5   2.5
    2   5.0  2   7.0   5.0
    3   7.5  3  10.5   7.5
    4  10.0  4  14.0  10.0

    In [90]: df
    Out[90]:
          a  b     c
    0   0.0  0   0.0
    1   2.5  1   3.5
    2   5.0  2   7.0
    3   7.5  3  10.5
    4  10.0  4  14.0

    In [91]: df.eval('d = c - b', inplace=True)

    In [92]: df
    Out[92]:
          a  b     c     d
    0   0.0  0   0.0   0.0
    1   2.5  1   3.5   2.5
    2   5.0  2   7.0   5.0
    3   7.5  3  10.5   7.5
    4  10.0  4  14.0  10.0

Warning

For backwards compatibility, `inplace` defaults to `True` if not specified. This will change in a future version of pandas. If your code depends on an inplace assignment you should update to explicitly set `inplace=True`

The `inplace` keyword parameter was also added the `query` method.

    In [93]: df.query('a > 5')
    Out[93]:
          a  b     c     d
    3   7.5  3  10.5   7.5
    4  10.0  4  14.0  10.0

    In [94]: df.query('a > 5', inplace=True)

    In [95]: df
    Out[95]:
          a  b     c     d
    3   7.5  3  10.5   7.5
    4  10.0  4  14.0  10.0

Warning

Note that the default value for `inplace` in a `query` is `False`, which is consistent with prior versions.

`eval` has also been updated to allow multi-line expressions for multiple assignments. These expressions will be evaluated one at a time in order. Only assignments are valid for multi-line expressions.

    In [96]: df
    Out[96]:
          a  b     c     d
    3   7.5  3  10.5   7.5
    4  10.0  4  14.0  10.0

    In [97]: df.eval("""
       ....: e = d + a
       ....: f = e - 22
       ....: g = f / 2.0""", inplace=True)
       ....:

    In [98]: df
    Out[98]:
          a  b     c     d     e    f    g
    3   7.5  3  10.5   7.5  15.0 -7.0 -3.5
    4  10.0  4  14.0  10.0  20.0 -2.0 -1.0

### Other API changes

- `DataFrame.between_time` and `Series.between_time` now only parse a fixed set of time strings. Parsing of date strings is no longer supported and raises a `ValueError`. (GH 11818)

      In [107]: s = pd.Series(range(10), pd.date_range('2015-01-01', freq='H', periods=10))

      In [108]: s.between_time("7:00am", "9:00am")
      Out[108]:
      2015-01-01 07:00:00    7
      2015-01-01 08:00:00    8
      2015-01-01 09:00:00    9
      Freq: H, Length: 3, dtype: int64

  This will now raise.

      In [2]: s.between_time('20150101 07:00:00','20150101 09:00:00')
      ValueError: Cannot convert arg ['20150101 07:00:00'] to a time.

- `.memory_usage()` now includes values in the index, as does memory_usage in `.info()` (GH 11597)

- `DataFrame.to_latex()` now supports non-ascii encodings (eg `utf-8`) in Python 2 with the parameter `encoding` (GH 7061)

- `pandas.merge()` and `DataFrame.merge()` will show a specific error message when trying to merge with an object that is not of type `DataFrame` or a subclass (GH 12081)

- `DataFrame.unstack` and `Series.unstack` now take `fill_value` keyword to allow direct replacement of missing values when an unstack results in missing values in the resulting `DataFrame`. As an added benefit, specifying `fill_value` will preserve the data type of the original stacked data. (GH 9746)

- As part of the new API for window functions and resampling, aggregation functions have been clarified, raising more informative error messages on invalid aggregations. (GH 9052). A full set of examples are presented in groupby.

- Statistical functions for `NDFrame` objects (like `sum(), mean(), min()`) will now raise if non-numpy-compatible arguments are passed in for `**kwargs` (GH 12301)

- `.to_latex` and `.to_html` gain a `decimal` parameter like `.to_csv`; the default is `'.'` (GH 12031)

- More helpful error message when constructing a `DataFrame` with empty data but with indices (GH 8020)

- `.describe()` will now properly handle bool dtype as a categorical (GH 6625)

- More helpful error message with an invalid `.transform` with user defined input (GH 10165)

- Exponentially weighted functions now allow specifying alpha directly (GH 10789) and raise `ValueError` if parameters violate `0 < alpha <= 1` (GH 12492)

### Deprecations

- The functions `pd.rolling_*`, `pd.expanding_*`, and `pd.ewm*` are deprecated and replaced by the corresponding method call. Note that the new suggested syntax includes all of the arguments (even if default) (GH 11603)

      In [1]: s = pd.Series(range(3))

      In [2]: pd.rolling_mean(s,window=2,min_periods=1)
              FutureWarning: pd.rolling_mean is deprecated for Series and
                   will be removed in a future version, replace with
                   Series.rolling(min_periods=1,window=2,center=False).mean()
      Out[2]:
              0    0.0
              1    0.5
              2    1.5
              dtype: float64

      In [3]: pd.rolling_cov(s, s, window=2)
              FutureWarning: pd.rolling_cov is deprecated for Series and
                   will be removed in a future version, replace with
                   Series.rolling(window=2).cov(other=<Series>)
      Out[3]:
              0    NaN
              1    0.5
              2    0.5
              dtype: float64

- The `freq` and `how` arguments to the `.rolling`, `.expanding`, and `.ewm` (new) functions are deprecated, and will be removed in a future version. You can simply resample the input prior to creating a window function. (GH 11603).

  For example, instead of `s.rolling(window=5,freq='D').max()` to get the max value on a rolling 5 Day window, one could use `s.resample('D').mean().rolling(window=5).max()`, which first resamples the data to daily data, then provides a rolling 5 day window.

- `pd.tseries.frequencies.get_offset_name` function is deprecated. Use offset’s `.freqstr` property as alternative (GH 11192)

- `pandas.stats.fama_macbeth` routines are deprecated and will be removed in a future version (GH 6077)

- `pandas.stats.ols`, `pandas.stats.plm` and `pandas.stats.var` routines are deprecated and will be removed in a future version (GH 6077)

- show a `FutureWarning` rather than a `DeprecationWarning` on using long-time deprecated syntax in `HDFStore.select`, where the `where` clause is not a string-like (GH 12027)

- The `pandas.options.display.mpl_style` configuration has been deprecated and will be removed in a future version of pandas. This functionality is better handled by matplotlib’s style sheets (GH 11783).

### Removal of deprecated float indexers

In GH 4892 indexing with floating point numbers on a non-`Float64Index` was deprecated (in version 0.14.0). In 0.18.0, this deprecation warning is removed and these will now raise a `TypeError`. (GH 12165, GH 12333)

    In [99]: s = pd.Series([1, 2, 3], index=[4, 5, 6])

    In [100]: s
    Out[100]:
    4    1
    5    2
    6    3
    dtype: int64

    In [101]: s2 = pd.Series([1, 2, 3], index=list('abc'))

    In [102]: s2
    Out[102]:
    a    1
    b    2
    c    3
    dtype: int64

Previous behavior:

    # this is label indexing
    In [2]: s[5.0]
    FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
    Out[2]: 2

    # this is positional indexing
    In [3]: s.iloc[1.0]
    FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
    Out[3]: 2

    # this is label indexing
    In [4]: s.loc[5.0]
    FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
    Out[4]: 2

    # .ix would coerce 1.0 to the positional 1, and index
    In [5]: s2.ix[1.0] = 10
    FutureWarning: scalar indexers for index type Index should be integers and not floating point

    In [6]: s2
    Out[6]:
    a     1
    b    10
    c     3
    dtype: int64

New behavior:

For iloc, getting & setting via a float scalar will always raise.

    In [3]: s.iloc[2.0]
    TypeError: cannot do label indexing on <class 'pandas.indexes.numeric.Int64Index'> with these indexers [2.0] of <type 'float'>

Other indexers will coerce to a like integer for both getting and setting. The `FutureWarning` has been dropped for `.loc`, `.ix` and `[]`.

    In [103]: s[5.0]
    Out[103]: np.int64(2)

    In [104]: s.loc[5.0]
    Out[104]: np.int64(2)

and setting

    In [105]: s_copy = s.copy()

    In [106]: s_copy[5.0] = 10

    In [107]: s_copy
    Out[107]:
    4     1
    5    10
    6     3
    dtype: int64

    In [108]: s_copy = s.copy()

    In [109]: s_copy.loc[5.0] = 10

    In [110]: s_copy
    Out[110]:
    4     1
    5    10
    6     3
    dtype: int64

Positional setting with `.ix` and a float indexer will ADD this value to the index, rather than previously setting the value by position.

    In [3]: s2.ix[1.0] = 10
    In [4]: s2
    Out[4]:
    a       1
    b       2
    c       3
    1.0    10
    dtype: int64

Slicing will also coerce integer-like floats to integers for a non-`Float64Index`.

    In [111]: s.loc[5.0:6]
    Out[111]:
    5    2
    6    3
    dtype: int64

Note that for floats that are NOT coercible to ints, the label based bounds will be excluded

    In [112]: s.loc[5.1:6]
    Out[112]:
    6    3
    dtype: int64

Float indexing on a `Float64Index` is unchanged.

    In [113]: s = pd.Series([1, 2, 3], index=np.arange(3.))

    In [114]: s[1.0]
    Out[114]: np.int64(2)

    In [115]: s[1.0:2.5]
    Out[115]:
    1.0    2
    2.0    3
    dtype: int64

### Removal of prior version deprecations/changes

- Removal of `rolling_corr_pairwise` in favor of `.rolling().corr(pairwise=True)` (GH 4950)

- Removal of `expanding_corr_pairwise` in favor of `.expanding().corr(pairwise=True)` (GH 4950)

- Removal of `DataMatrix` module. This was not imported into the pandas namespace in any event (GH 12111)

- Removal of `cols` keyword in favor of `subset` in `DataFrame.duplicated()` and `DataFrame.drop_duplicates()` (GH 6680)

- Removal of the `read_frame` and `frame_query` (both aliases for `pd.read_sql`) and `write_frame` (alias of `to_sql`) functions in the `pd.io.sql` namespace, deprecated since 0.14.0 (GH 6292).

- Removal of the `order` keyword from `.factorize()` (GH 6930)

## Performance improvements

- Improved performance of `andrews_curves` (GH 11534)

- Improved huge `DatetimeIndex`, `PeriodIndex` and `TimedeltaIndex`’s ops performance including `NaT` (GH 10277)

- Improved performance of `pandas.concat` (GH 11958)

- Improved performance of `StataReader` (GH 11591)

- Improved performance in construction of `Categoricals` with `Series` of datetimes containing `NaT` (GH 12077)

- Improved performance of ISO 8601 date parsing for dates without separators (GH 11899), leading zeros (GH 11871) and with white space preceding the time zone (GH 9714)

## Bug fixes

- Bug in `GroupBy.size` when data-frame is empty. (GH 11699)

- Bug in `Period.end_time` when a multiple of time period is requested (GH 11738)

- Regression in `.clip` with tz-aware datetimes (GH 11838)

- Bug in `date_range` when the boundaries fell on the frequency (GH 11804, GH 12409)

- Bug in consistency of passing nested dicts to `.groupby(...).agg(...)` (GH 9052)

- Accept unicode in `Timedelta` constructor (GH 11995)

- Bug in value label reading for `StataReader` when reading incrementally (GH 12014)

- Bug in vectorized `DateOffset` when `n` parameter is `0` (GH 11370)

- Compat for numpy 1.11 w.r.t. `NaT` comparison changes (GH 12049)

- Bug in `read_csv` when reading from a `StringIO` in threads (GH 11790)

- Bug in not treating `NaT` as a missing value in datetimelikes when factorizing & with `Categoricals` (GH 12077)

- Bug in getitem when the values of a `Series` were tz-aware (GH 12089)

- Bug in `Series.str.get_dummies` when one of the variables was ‘name’ (GH 12180)

- Bug in `pd.concat` while concatenating tz-aware NaT series. (GH 11693, GH 11755, GH 12217)

- Bug in `pd.read_stata` with version <= 108 files (GH 12232)

- Bug in `Series.resample` using a frequency of `Nano` when the index is a `DatetimeIndex` and contains non-zero nanosecond parts (GH 12037)

- Bug in resampling with `.nunique` and a sparse index (GH 12352)

- Removed some compiler warnings (GH 12471)

- Work around compat issues with `boto` in python 3.5 (GH 11915)

- Bug in `NaT` subtraction from `Timestamp` or `DatetimeIndex` with timezones (GH 11718)

- Bug in subtraction of `Series` of a single tz-aware `Timestamp` (GH 12290)

- Use compat iterators in PY2 to support `.next()` (GH 12299)

- Bug in `Timedelta.round` with negative values (GH 11690)

- Bug in `.loc` against `CategoricalIndex` may result in normal `Index` (GH 11586)

- Bug in `DataFrame.info` when duplicated column names exist (GH 11761)

- Bug in `.copy` of datetime tz-aware objects (GH 11794)

- Bug in `Series.apply` and `Series.map` where `timedelta64` was not boxed (GH 11349)

- Bug in `DataFrame.set_index()` with tz-aware `Series` (GH 12358)

- Bug in subclasses of `DataFrame` where `AttributeError` did not propagate (GH 11808)

- Bug groupby on tz-aware data where selection not returning `Timestamp` (GH 11616)

- Bug in `pd.read_clipboard` and `pd.to_clipboard` functions not supporting Unicode; upgrade included `pyperclip` to v1.5.15 (GH 9263)

- Bug in `DataFrame.query` containing an assignment (GH 8664)

- Bug in `from_msgpack` where `__contains__()` fails for columns of the unpacked `DataFrame`, if the `DataFrame` has object columns. (GH 11880)

- Bug in `.resample` on categorical data with `TimedeltaIndex` (GH 12169)

- Bug in timezone info lost when broadcasting scalar datetime to `DataFrame` (GH 11682)

- Bug in `Index` creation from `Timestamp` with mixed tz coerces to UTC (GH 11488)

- Bug in `to_numeric` where it does not raise if input is more than one dimension (GH 11776)

- Bug in parsing timezone offset strings with non-zero minutes (GH 11708)

- Bug in `df.plot` using incorrect colors for bar plots under matplotlib 1.5+ (GH 11614)

- Bug in the `groupby` `plot` method when using keyword arguments (GH 11805).

- Bug in `DataFrame.duplicated` and `drop_duplicates` causing spurious matches when setting `keep=False` (GH 11864)

- Bug in `.loc` result with duplicated key may have `Index` with incorrect dtype (GH 11497)

- Bug in `pd.rolling_median` where memory allocation failed even with sufficient memory (GH 11696)

- Bug in `DataFrame.style` with spurious zeros (GH 12134)

- Bug in `DataFrame.style` with integer columns not starting at 0 (GH 12125)

- Bug in `.style.bar` may not rendered properly using specific browser (GH 11678)

- Bug in rich comparison of `Timedelta` with a `numpy.array` of `Timedelta` that caused an infinite recursion (GH 11835)

- Bug in `DataFrame.round` dropping column index name (GH 11986)

- Bug in `df.replace` while replacing value in mixed dtype `Dataframe` (GH 11698)

- Bug in `Index` prevents copying name of passed `Index`, when a new name is not provided (GH 11193)

- Bug in `read_excel` failing to read any non-empty sheets when empty sheets exist and `sheetname=None` (GH 11711)

- Bug in `read_excel` failing to raise `NotImplemented` error when keywords `parse_dates` and `date_parser` are provided (GH 11544)

- Bug in `read_sql` with `pymysql` connections failing to return chunked data (GH 11522)

- Bug in `.to_csv` ignoring formatting parameters `decimal`, `na_rep`, `float_format` for float indexes (GH 11553)

- Bug in `Int64Index` and `Float64Index` preventing the use of the modulo operator (GH 9244)

- Bug in `MultiIndex.drop` for not lexsorted MultiIndexes (GH 12078)

- Bug in `DataFrame` when masking an empty `DataFrame` (GH 11859)

- Bug in `.plot` potentially modifying the `colors` input when the number of columns didn’t match the number of series provided (GH 12039).

- Bug in `Series.plot` failing when index has a `CustomBusinessDay` frequency (GH 7222).

- Bug in `.to_sql` for `datetime.time` values with sqlite fallback (GH 8341)

- Bug in `read_excel` failing to read data with one column when `squeeze=True` (GH 12157)

- Bug in `read_excel` failing to read one empty column (GH 12292, GH 9002)

- Bug in `.groupby` where a `KeyError` was not raised for a wrong column if there was only one row in the dataframe (GH 11741)

- Bug in `.read_csv` with dtype specified on empty data producing an error (GH 12048)

- Bug in `.read_csv` where strings like `'2E'` are treated as valid floats (GH 12237)

- Bug in building pandas with debugging symbols (GH 12123)

- Removed `millisecond` property of `DatetimeIndex`. This would always raise a `ValueError` (GH 12019).

- Bug in `Series` constructor with read-only data (GH 11502)

- Removed `pandas._testing.choice()`. Should use `np.random.choice()`, instead. (GH 12386)

- Bug in `.loc` setitem indexer preventing the use of a TZ-aware DatetimeIndex (GH 12050)

- Bug in `.style` indexes and MultiIndexes not appearing (GH 11655)

- Bug in `to_msgpack` and `from_msgpack` which did not correctly serialize or deserialize `NaT` (GH 12307).

- Bug in `.skew` and `.kurt` due to roundoff error for highly similar values (GH 11974)

- Bug in `Timestamp` constructor where microsecond resolution was lost if HHMMSS were not separated with ‘:’ (GH 10041)

- Bug in `buffer_rd_bytes` src->buffer could be freed more than once if reading failed, causing a segfault (GH 12098)

- Bug in `crosstab` where arguments with non-overlapping indexes would return a `KeyError` (GH 10291)

- Bug in `DataFrame.apply` in which reduction was not being prevented for cases in which `dtype` was not a numpy dtype (GH 12244)

- Bug when initializing categorical series with a scalar value. (GH 12336)

- Bug when specifying a UTC `DatetimeIndex` by setting `utc=True` in `.to_datetime` (GH 11934)

- Bug when increasing the buffer size of CSV reader in `read_csv` (GH 12494)

- Bug when setting columns of a `DataFrame` with duplicate column names (GH 12344)

## Contributors

A total of 101 people contributed patches to this release. People with a “+” by their names contributed a patch for the first time.

- ARF +

- Alex Alekseyev +

- Andrew McPherson +

- Andrew Rosenfeld

- Andy Hayden

- Anthonios Partheniou

- Anton I. Sipos

- Ben +

- Ben North +

- Bran Yang +

- Chris

- Chris Carroux +

- Christopher C. Aycock +

- Christopher Scanlin +

- Cody +

- Da Wang +

- Daniel Grady +

- Dorozhko Anton +

- Dr-Irv +

- Erik M. Bray +

- Evan Wright

- Francis T. O’Donovan +

- Frank Cleary +

- Gianluca Rossi

- Graham Jeffries +

- Guillaume Horel

- Henry Hammond +

- Isaac Schwabacher +

- Jean-Mathieu Deschenes

- Jeff Reback

- Joe Jevnik +

- John Freeman +

- John Fremlin +

- Jonas Hoersch +

- Joris Van den Bossche

- Joris Vankerschaver

- Justin Lecher

- Justin Lin +

- Ka Wo Chen

- Keming Zhang +

- Kerby Shedden

- Kyle +

- Marco Farrugia +

- MasonGallo +

- MattRijk +

- Matthew Lurie +

- Maximilian Roos

- Mayank Asthana +

- Mortada Mehyar

- Moussa Taifi +

- Navreet Gill +

- Nicolas Bonnotte

- Paul Reiners +

- Philip Gura +

- Pietro Battiston

- RahulHP +

- Randy Carnevale

- Rinoc Johnson

- Rishipuri +

- Sangmin Park +

- Scott E Lasley

- Sereger13 +

- Shannon Wang +

- Skipper Seabold

- Thierry Moisan

- Thomas A Caswell

- Toby Dylan Hocking +

- Tom Augspurger

- Travis +

- Trent Hauck

- Tux1

- Varun

- Wes McKinney

- Will Thompson +

- Yoav Ram

- Yoong Kang Lim +

- Yoshiki Vázquez Baeza

- Young Joong Kim +

- Younggun Kim

- Yuval Langer +

- alex argunov +

- behzad nouri

- boombard +

- brian-pantano +

- chromy +

- daniel +

- dgram0 +

- gfyoung +

- hack-c +

- hcontrast +

- jfoo +

- kaustuv deolal +

- llllllllll

- ranarag +

- rockg

- scls19fr

- seales +

- sinhrks

- srib +

- surveymedia.ca +

- tworec +