Version 0.18.0 (March 13, 2016)#

This is a major release from 0.17.1 and includes a small number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. We recommend that all users upgrade to this version.

Warning

pandas >= 0.18.0 no longer supports compatibility with Python version 2.6 and 3.3 (GH 7718, GH 11273)

Warning

numexpr version 2.4.4 will now show a warning and not be used as a computation back-end for pandas because of some buggy behavior. This does not affect other versions (>= 2.1 and >= 2.4.6). (GH 12489)

Highlights include:

Moving and expanding window functions are now methods on Series and DataFrame, similar to .groupby, see here.
Adding support for a RangeIndex as a specialized form of the Int64Index for memory savings, see here.
API breaking change to the .resample method to make it more .groupby like, see here.
Removal of support for positional indexing with floats, which was deprecated since 0.14.0. This will now raise a TypeError, see here.
The .to_xarray() function has been added for compatibility with the xarray package, see here.
The read_sas function has been enhanced to read sas7bdat files, see here.
Addition of the .str.extractall() method, and API changes to the .str.extract() method and .str.cat() method.
pd.test() top-level nose test runner is available (GH 4327).

Check the API Changes and deprecations before updating.

New features#

Window functions are now methods#

Window functions have been refactored to be methods on Series/DataFrame objects, rather than top-level functions, which are now deprecated. This allows these window-type functions, to have a similar API to that of .groupby. See the full documentation here (GH 11603, GH 12373)

In [1]: np.random.seed(1234)

In [2]: df = pd.DataFrame({'A': range(10), 'B': np.random.randn(10)})

In [3]: df
Out[3]: 
   A         B
0  0  0.471435
1  1 -1.190976
2  2  1.432707
3  3 -0.312652
4  4 -0.720589
5  5  0.887163
6  6  0.859588
7  7 -0.636524
8  8  0.015696
9  9 -2.242685

[10 rows x 2 columns]

Previous behavior:

In [8]: pd.rolling_mean(df, window=3)
        FutureWarning: pd.rolling_mean is deprecated for DataFrame and will be removed in a future version, replace with
                       DataFrame.rolling(window=3,center=False).mean()
Out[8]:
    A         B
0 NaN       NaN
1 NaN       NaN
2   1  0.237722
3   2 -0.023640
4   3  0.133155
5   4 -0.048693
6   5  0.342054
7   6  0.370076
8   7  0.079587
9   8 -0.954504

New behavior:

In [4]: r = df.rolling(window=3)

These show a descriptive repr

In [5]: r
Out[5]: Rolling [window=3,center=False,method=single]

with tab-completion of available methods and properties.

In [9]: r.<TAB>  # noqa E225, E999
r.A           r.agg         r.apply       r.count       r.exclusions  r.max         r.median      r.name        r.skew        r.sum
r.B           r.aggregate   r.corr        r.cov         r.kurt        r.mean        r.min         r.quantile    r.std         r.var

The methods operate on the Rolling object itself

In [6]: r.mean()
Out[6]: 
     A         B
NaN       NaN
NaN       NaN
1.0  0.237722
2.0 -0.023640
3.0  0.133155
4.0 -0.048693
5.0  0.342054
6.0  0.370076
7.0  0.079587
8.0 -0.954504

[10 rows x 2 columns]

They provide getitem accessors

In [7]: r['A'].mean()
Out[7]: 
  NaN
  NaN
  1.0
  2.0
  3.0
  4.0
  5.0
  6.0
  7.0
  8.0
Name: A, Length: 10, dtype: float64

And multiple aggregations

In [8]: r.agg({'A': ['mean', 'std'],
   ...:        'B': ['mean', 'std']})
   ...: 
Out[8]: 
     A              B          
  mean  std      mean       std
0  NaN  NaN       NaN       NaN
1  NaN  NaN       NaN       NaN
2  1.0  1.0  0.237722  1.327364
3  2.0  1.0 -0.023640  1.335505
4  3.0  1.0  0.133155  1.143778
5  4.0  1.0 -0.048693  0.835747
6  5.0  1.0  0.342054  0.920379
7  6.0  1.0  0.370076  0.871850
8  7.0  1.0  0.079587  0.750099
9  8.0  1.0 -0.954504  1.162285

[10 rows x 4 columns]

Changes to rename#

Series.rename and NDFrame.rename_axis can now take a scalar or list-like argument for altering the Series or axis name, in addition to their old behaviors of altering labels. (GH 9494, GH 11965)

In [9]: s = pd.Series(np.random.randn(5))

In [10]: s.rename('newname')
Out[10]: 
0    1.150036
1    0.991946
2    0.953324
3   -2.021255
4   -0.334077
Name: newname, Length: 5, dtype: float64

In [11]: df = pd.DataFrame(np.random.randn(5, 2))

In [12]: (df.rename_axis("indexname")
   ....:    .rename_axis("columns_name", axis="columns"))
   ....: 
Out[12]: 
columns_name         0         1
indexname                       
0             0.002118  0.405453
1             0.289092  1.321158
2            -1.546906 -0.202646
3            -0.655969  0.193421
4             0.553439  1.318152

[5 rows x 2 columns]

The new functionality works well in method chains. Previously these methods only accepted functions or dicts mapping a label to a new label. This continues to work as before for function or dict-like values.

Range Index#

A RangeIndex has been added to the Int64Index sub-classes to support a memory saving alternative for common use cases. This has a similar implementation to the python range object (xrange in python 2), in that it only stores the start, stop, and step values for the index. It will transparently interact with the user API, converting to Int64Index if needed.

This will now be the default constructed index for NDFrame objects, rather than previous an Int64Index. (GH 939, GH 12070, GH 12071, GH 12109, GH 12888)

Previous behavior:

In [3]: s = pd.Series(range(1000))

In [4]: s.index
Out[4]:
Int64Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,
            ...
            990, 991, 992, 993, 994, 995, 996, 997, 998, 999], dtype='int64', length=1000)

In [6]: s.index.nbytes
Out[6]: 8000

New behavior:

In [13]: s = pd.Series(range(1000))

In [14]: s.index
Out[14]: RangeIndex(start=0, stop=1000, step=1)

In [15]: s.index.nbytes
Out[15]: 128

Changes to str.extract#

The .str.extract method takes a regular expression with capture groups, finds the first match in each subject string, and returns the contents of the capture groups (GH 11386).

In v0.18.0, the expand argument was added to extract.

expand=False: it returns a Series, Index, or DataFrame, depending on the subject and regular expression pattern (same behavior as pre-0.18.0).
expand=True: it always returns a DataFrame, which is more consistent and less confusing from the perspective of a user.

Currently the default is expand=None which gives a FutureWarning and uses expand=False. To avoid this warning, please explicitly specify expand.

In [1]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=None)
FutureWarning: currently extract(expand=None) means expand=False (return Index/Series/DataFrame)
but in a future version of pandas this will be changed to expand=True (return DataFrame)

Out[1]:
0      1
1      2
2    NaN
dtype: object

Extracting a regular expression with one group returns a Series if expand=False.

In [16]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=False)
Out[16]: 
0      1
1      2
2    NaN
Length: 3, dtype: object

It returns a DataFrame with one column if expand=True.

In [17]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=True)
Out[17]: 
     0
0    1
1    2
2  NaN

[3 rows x 1 columns]

Calling on an Index with a regex with exactly one capture group returns an Index if expand=False.

In [18]: s = pd.Series(["a1", "b2", "c3"], ["A11", "B22", "C33"])

In [19]: s.index
Out[19]: Index(['A11', 'B22', 'C33'], dtype='object')

In [20]: s.index.str.extract("(?P<letter>[a-zA-Z])", expand=False)
Out[20]: Index(['A', 'B', 'C'], dtype='object', name='letter')

It returns a DataFrame with one column if expand=True.

In [21]: s.index.str.extract("(?P<letter>[a-zA-Z])", expand=True)
Out[21]: 
  letter
0      A
1      B
2      C

[3 rows x 1 columns]

Calling on an Index with a regex with more than one capture group raises ValueError if expand=False.

>>> s.index.str.extract("(?P<letter>[a-zA-Z])([0-9]+)", expand=False)
ValueError: only one regex group is supported with Index

It returns a DataFrame if expand=True.

In [22]: s.index.str.extract("(?P<letter>[a-zA-Z])([0-9]+)", expand=True)
Out[22]: 
  letter   1
0      A  11
1      B  22
2      C  33

[3 rows x 2 columns]

In summary, extract(expand=True) always returns a DataFrame with a row for every subject string, and a column for every capture group.

Addition of str.extractall#

The .str.extractall method was added (GH 11386). Unlike extract, which returns only the first match.

In [23]: s = pd.Series(["a1a2", "b1", "c1"], ["A", "B", "C"])

In [24]: s
Out[24]: 
A    a1a2
B      b1
C      c1
Length: 3, dtype: object

In [25]: s.str.extract(r"(?P<letter>[ab])(?P<digit>\d)", expand=False)
Out[25]: 
  letter digit
A      a     1
B      b     1
C    NaN   NaN

[3 rows x 2 columns]

The extractall method returns all matches.

In [26]: s.str.extractall(r"(?P<letter>[ab])(?P<digit>\d)")
Out[26]: 
        letter digit
  match             
A 0          a     1
  1          a     2
B 0          b     1

[3 rows x 2 columns]

Changes to str.cat#

The method .str.cat() concatenates the members of a Series. Before, if NaN values were present in the Series, calling .str.cat() on it would return NaN, unlike the rest of the Series.str.* API. This behavior has been amended to ignore NaN values by default. (GH 11435).

A new, friendlier ValueError is added to protect against the mistake of supplying the sep as an arg, rather than as a kwarg. (GH 11334).

In [27]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(sep=' ')
Out[27]: 'a b c'

In [28]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(sep=' ', na_rep='?')
Out[28]: 'a b ? c'

In [2]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(' ')
ValueError: Did you mean to supply a ``sep`` keyword?

Datetimelike rounding#

DatetimeIndex, Timestamp, TimedeltaIndex, Timedelta have gained the .round(), .floor() and .ceil() method for datetimelike rounding, flooring and ceiling. (GH 4314, GH 11963)

Naive datetimes

In [29]: dr = pd.date_range('20130101 09:12:56.1234', periods=3)

In [30]: dr
Out[30]: 
DatetimeIndex(['2013-01-01 09:12:56.123400', '2013-01-02 09:12:56.123400',
               '2013-01-03 09:12:56.123400'],
              dtype='datetime64[ns]', freq='D')

In [31]: dr.round('s')
Out[31]: 
DatetimeIndex(['2013-01-01 09:12:56', '2013-01-02 09:12:56',
               '2013-01-03 09:12:56'],
              dtype='datetime64[ns]', freq=None)

# Timestamp scalar
In [32]: dr[0]
Out[32]: Timestamp('2013-01-01 09:12:56.123400')

In [33]: dr[0].round('10s')
Out[33]: Timestamp('2013-01-01 09:13:00')

Tz-aware are rounded, floored and ceiled in local times

In [34]: dr = dr.tz_localize('US/Eastern')

In [35]: dr
Out[35]: 
DatetimeIndex(['2013-01-01 09:12:56.123400-05:00',
               '2013-01-02 09:12:56.123400-05:00',
               '2013-01-03 09:12:56.123400-05:00'],
              dtype='datetime64[ns, US/Eastern]', freq=None)

In [36]: dr.round('s')
Out[36]: 
DatetimeIndex(['2013-01-01 09:12:56-05:00', '2013-01-02 09:12:56-05:00',
               '2013-01-03 09:12:56-05:00'],
              dtype='datetime64[ns, US/Eastern]', freq=None)

Timedeltas

In [37]: t = pd.timedelta_range('1 days 2 hr 13 min 45 us', periods=3, freq='d')

In [38]: t
Out[38]:
TimedeltaIndex(['1 days 02:13:00.000045', '2 days 02:13:00.000045',
                '3 days 02:13:00.000045'],
               dtype='timedelta64[ns]', freq='D')

In [39]: t.round('10min')
Out[39]:
TimedeltaIndex(['1 days 02:10:00', '2 days 02:10:00',
                '3 days 02:10:00'],
               dtype='timedelta64[ns]', freq=None)

# Timedelta scalar
In [40]: t[0]
Out[40]: Timedelta('1 days 02:13:00.000045')

In [41]: t[0].round('2h')
Out[41]: Timedelta('1 days 02:00:00')

In addition, .round(), .floor() and .ceil() will be available through the .dt accessor of Series.

In [37]: s = pd.Series(dr)

In [38]: s
Out[38]: 
0   2013-01-01 09:12:56.123400-05:00
1   2013-01-02 09:12:56.123400-05:00
2   2013-01-03 09:12:56.123400-05:00
Length: 3, dtype: datetime64[ns, US/Eastern]

In [39]: s.dt.round('D')
Out[39]: 
0   2013-01-01 00:00:00-05:00
1   2013-01-02 00:00:00-05:00
2   2013-01-03 00:00:00-05:00
Length: 3, dtype: datetime64[ns, US/Eastern]

Formatting of integers in FloatIndex#

Integers in FloatIndex, e.g. 1., are now formatted with a decimal point and a 0 digit, e.g. 1.0 (GH 11713) This change not only affects the display to the console, but also the output of IO methods like .to_csv or .to_html.

Previous behavior:

In [2]: s = pd.Series([1, 2, 3], index=np.arange(3.))

In [3]: s
Out[3]:
0    1
1    2
2    3
dtype: int64

In [4]: s.index
Out[4]: Float64Index([0.0, 1.0, 2.0], dtype='float64')

In [5]: print(s.to_csv(path=None))
0,1
1,2
2,3

New behavior:

In [40]: s = pd.Series([1, 2, 3], index=np.arange(3.))

In [41]: s
Out[41]: 
0.0    1
1.0    2
2.0    3
Length: 3, dtype: int64

In [42]: s.index
Out[42]: Index([0.0, 1.0, 2.0], dtype='float64')

In [43]: print(s.to_csv(path_or_buf=None, header=False))
0.0,1
1.0,2
2.0,3

Changes to dtype assignment behaviors#

When a DataFrame’s slice is updated with a new slice of the same dtype, the dtype of the DataFrame will now remain the same. (GH 10503)

Previous behavior:

In [5]: df = pd.DataFrame({'a': [0, 1, 1],
                           'b': pd.Series([100, 200, 300], dtype='uint32')})

In [7]: df.dtypes
Out[7]:
a     int64
b    uint32
dtype: object

In [8]: ix = df['a'] == 1

In [9]: df.loc[ix, 'b'] = df.loc[ix, 'b']

In [11]: df.dtypes
Out[11]:
a    int64
b    int64
dtype: object

New behavior:

In [44]: df = pd.DataFrame({'a': [0, 1, 1],
   ....:                    'b': pd.Series([100, 200, 300], dtype='uint32')})
   ....: 

In [45]: df.dtypes
Out[45]: 
a     int64
b    uint32
Length: 2, dtype: object

In [46]: ix = df['a'] == 1

In [47]: df.loc[ix, 'b'] = df.loc[ix, 'b']

In [48]: df.dtypes
Out[48]: 
a     int64
b    uint32
Length: 2, dtype: object

When a DataFrame’s integer slice is partially updated with a new slice of floats that could potentially be down-casted to integer without losing precision, the dtype of the slice will be set to float instead of integer.

Previous behavior:

In [4]: df = pd.DataFrame(np.array(range(1,10)).reshape(3,3),
                          columns=list('abc'),
                          index=[[4,4,8], [8,10,12]])

In [5]: df
Out[5]:
      a  b  c
4 8   1  2  3
  10  4  5  6
8 12  7  8  9

In [7]: df.ix[4, 'c'] = np.array([0., 1.])

In [8]: df
Out[8]:
      a  b  c
4 8   1  2  0
  10  4  5  1
8 12  7  8  9

New behavior:

In [49]: df = pd.DataFrame(np.array(range(1,10)).reshape(3,3),
   ....:                   columns=list('abc'),
   ....:                   index=[[4,4,8], [8,10,12]])
   ....: 

In [50]: df
Out[50]: 
      a  b  c
4 8   1  2  3
  10  4  5  6
8 12  7  8  9

[3 rows x 3 columns]

In [51]: df.loc[4, 'c'] = np.array([0., 1.])

In [52]: df
Out[52]: 
      a  b  c
4 8   1  2  0
  10  4  5  1
8 12  7  8  9

[3 rows x 3 columns]

Method to_xarray#

In a future version of pandas, we will be deprecating Panel and other > 2 ndim objects. In order to provide for continuity, all NDFrame objects have gained the .to_xarray() method in order to convert to xarray objects, which has a pandas-like interface for > 2 ndim. (GH 11972)

See the xarray full-documentation here.

In [1]: p = Panel(np.arange(2*3*4).reshape(2,3,4))

In [2]: p.to_xarray()
Out[2]:
<xarray.DataArray (items: 2, major_axis: 3, minor_axis: 4)>
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])
Coordinates:
  * items       (items) int64 0 1
  * major_axis  (major_axis) int64 0 1 2
  * minor_axis  (minor_axis) int64 0 1 2 3

Latex representation#

DataFrame has gained a ._repr_latex_() method in order to allow for conversion to latex in a ipython/jupyter notebook using nbconvert. (GH 11778)

Note that this must be activated by setting the option pd.display.latex.repr=True (GH 12182)

For example, if you have a jupyter notebook you plan to convert to latex using nbconvert, place the statement pd.display.latex.repr=True in the first cell to have the contained DataFrame output also stored as latex.

The options display.latex.escape and display.latex.longtable have also been added to the configuration and are used automatically by the to_latex method. See the available options docs for more info.

`pd.read_sas()` changes#

read_sas has gained the ability to read SAS7BDAT files, including compressed files. The files can be read in entirety, or incrementally. For full details see here. (GH 4052)

Other enhancements#

Handle truncated floats in SAS xport files (GH 11713)
Added option to hide index in Series.to_string (GH 11729)
read_excel now supports s3 urls of the format s3://bucketname/filename (GH 11447)
add support for AWS_S3_HOST env variable when reading from s3 (GH 12198)
A simple version of Panel.round() is now implemented (GH 11763)
For Python 3.x, round(DataFrame), round(Series), round(Panel) will work (GH 11763)
sys.getsizeof(obj) returns the memory usage of a pandas object, including the values it contains (GH 11597)
Series gained an is_unique attribute (GH 11946)
DataFrame.quantile and Series.quantile now accept interpolation keyword (GH 10174).
Added DataFrame.style.format for more flexible formatting of cell values (GH 11692)
DataFrame.select_dtypes now allows the np.float16 type code (GH 11990)
pivot_table() now accepts most iterables for the values parameter (GH 12017)
Added Google BigQuery service account authentication support, which enables authentication on remote servers. (GH 11881, GH 12572). For further details see here
HDFStore is now iterable: for k in store is equivalent to for k in store.keys() (GH 12221).
Add missing methods/fields to .dt for Period (GH 8848)
The entire code base has been PEP-ified (GH 12096)

Backwards incompatible API changes#

the leading white spaces have been removed from the output of .to_string(index=False) method (GH 11833)
the out parameter has been removed from the Series.round() method. (GH 11763)
DataFrame.round() leaves non-numeric columns unchanged in its return, rather than raises. (GH 11885)
DataFrame.head(0) and DataFrame.tail(0) return empty frames, rather than self. (GH 11937)
Series.head(0) and Series.tail(0) return empty series, rather than self. (GH 11937)
to_msgpack and read_msgpack encoding now defaults to 'utf-8'. (GH 12170)
the order of keyword arguments to text file parsing functions (.read_csv(), .read_table(), .read_fwf()) changed to group related arguments. (GH 11555)
NaTType.isoformat now returns the string 'NaT to allow the result to be passed to the constructor of Timestamp. (GH 12300)

NaT and Timedelta operations#

NaT and Timedelta have expanded arithmetic operations, which are extended to Series arithmetic where applicable. Operations defined for datetime64[ns] or timedelta64[ns] are now also defined for NaT (GH 11564).

NaT now supports arithmetic operations with integers and floats.

In [53]: pd.NaT * 1
Out[53]: NaT

In [54]: pd.NaT * 1.5
Out[54]: NaT

In [55]: pd.NaT / 2
Out[55]: NaT

In [56]: pd.NaT * np.nan
Out[56]: NaT

NaT defines more arithmetic operations with datetime64[ns] and timedelta64[ns].

In [57]: pd.NaT / pd.NaT
Out[57]: nan

In [58]: pd.Timedelta('1s') / pd.NaT
Out[58]: nan

NaT may represent either a datetime64[ns] null or a timedelta64[ns] null. Given the ambiguity, it is treated as a timedelta64[ns], which allows more operations to succeed.

In [59]: pd.NaT + pd.NaT
Out[59]: NaT

# same as
In [60]: pd.Timedelta('1s') + pd.Timedelta('1s')
Out[60]: Timedelta('0 days 00:00:02')

as opposed to

In [3]: pd.Timestamp('19900315') + pd.Timestamp('19900315')
TypeError: unsupported operand type(s) for +: 'Timestamp' and 'Timestamp'

However, when wrapped in a Series whose dtype is datetime64[ns] or timedelta64[ns], the dtype information is respected.

In [1]: pd.Series([pd.NaT], dtype='<M8[ns]') + pd.Series([pd.NaT], dtype='<M8[ns]')
TypeError: can only operate on a datetimes for subtraction,
           but the operator [__add__] was passed

In [61]: pd.Series([pd.NaT], dtype='<m8[ns]') + pd.Series([pd.NaT], dtype='<m8[ns]')
Out[61]: 
0   NaT
Length: 1, dtype: timedelta64[ns]

Timedelta division by floats now works.

In [62]: pd.Timedelta('1s') / 2.0
Out[62]: Timedelta('0 days 00:00:00.500000')

Subtraction by Timedelta in a Series by a Timestamp works (GH 11925)

In [63]: ser = pd.Series(pd.timedelta_range('1 day', periods=3))

In [64]: ser
Out[64]: 
0   1 days
1   2 days
2   3 days
Length: 3, dtype: timedelta64[ns]

In [65]: pd.Timestamp('2012-01-01') - ser
Out[65]: 
0   2011-12-31
1   2011-12-30
2   2011-12-29
Length: 3, dtype: datetime64[ns]

NaT.isoformat() now returns 'NaT'. This change allows pd.Timestamp to rehydrate any timestamp like object from its isoformat (GH 12300).

Changes to msgpack#

Forward incompatible changes in msgpack writing format were made over 0.17.0 and 0.18.0; older versions of pandas cannot read files packed by newer versions (GH 12129, GH 10527)

Bugs in to_msgpack and read_msgpack introduced in 0.17.0 and fixed in 0.18.0, caused files packed in Python 2 unreadable by Python 3 (GH 12142). The following table describes the backward and forward compat of msgpacks.

Warning

Packed with	Can be unpacked with
pre-0.17 / Python 2	any
pre-0.17 / Python 3	any
0.17 / Python 2	==0.17 / Python 2 >=0.18 / any Python
0.17 / Python 3	>=0.18 / any Python
0.18	>= 0.18

0.18.0 is backward-compatible for reading files packed by older versions, except for files packed with 0.17 in Python 2, in which case only they can only be unpacked in Python 2.

Signature change for .rank#

Series.rank and DataFrame.rank now have the same signature (GH 11759)

Previous signature

In [3]: pd.Series([0,1]).rank(method='average', na_option='keep',
                              ascending=True, pct=False)
Out[3]:
0    1
1    2
dtype: float64

In [4]: pd.DataFrame([0,1]).rank(axis=0, numeric_only=None,
                                 method='average', na_option='keep',
                                 ascending=True, pct=False)
Out[4]:
   0
0  1
1  2

New signature

In [66]: pd.Series([0,1]).rank(axis=0, method='average', numeric_only=False,
   ....:                       na_option='keep', ascending=True, pct=False)
   ....: 
Out[66]: 
0    1.0
1    2.0
Length: 2, dtype: float64

In [67]: pd.DataFrame([0,1]).rank(axis=0, method='average', numeric_only=False,
   ....:                          na_option='keep', ascending=True, pct=False)
   ....: 
Out[67]: 
     0
0  1.0
1  2.0

[2 rows x 1 columns]

Bug in QuarterBegin with n=0#

In previous versions, the behavior of the QuarterBegin offset was inconsistent depending on the date when the n parameter was 0. (GH 11406)

The general semantics of anchored offsets for n=0 is to not move the date when it is an anchor point (e.g., a quarter start date), and otherwise roll forward to the next anchor point.

In [68]: d = pd.Timestamp('2014-02-01')

In [69]: d
Out[69]: Timestamp('2014-02-01 00:00:00')

In [70]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2)
Out[70]: Timestamp('2014-02-01 00:00:00')

In [71]: d + pd.offsets.QuarterBegin(n=0, startingMonth=1)
Out[71]: Timestamp('2014-04-01 00:00:00')

For the QuarterBegin offset in previous versions, the date would be rolled backwards if date was in the same month as the quarter start date.

In [3]: d = pd.Timestamp('2014-02-15')

In [4]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2)
Out[4]: Timestamp('2014-02-01 00:00:00')

This behavior has been corrected in version 0.18.0, which is consistent with other anchored offsets like MonthBegin and YearBegin.

In [72]: d = pd.Timestamp('2014-02-15')

In [73]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2)
Out[73]: Timestamp('2014-05-01 00:00:00')

Resample API#

Like the change in the window functions API above, .resample(...) is changing to have a more groupby-like API. (GH 11732, GH 12702, GH 12202, GH 12332, GH 12334, GH 12348, GH 12448).

In [74]: np.random.seed(1234)

In [75]: df = pd.DataFrame(np.random.rand(10,4),
   ....:                   columns=list('ABCD'),
   ....:                   index=pd.date_range('2010-01-01 09:00:00',
   ....:                                       periods=10, freq='s'))
   ....: 

In [76]: df
Out[76]: 
                            A         B         C         D
2010-01-01 09:00:00  0.191519  0.622109  0.437728  0.785359
2010-01-01 09:00:01  0.779976  0.272593  0.276464  0.801872
2010-01-01 09:00:02  0.958139  0.875933  0.357817  0.500995
2010-01-01 09:00:03  0.683463  0.712702  0.370251  0.561196
2010-01-01 09:00:04  0.503083  0.013768  0.772827  0.882641
2010-01-01 09:00:05  0.364886  0.615396  0.075381  0.368824
2010-01-01 09:00:06  0.933140  0.651378  0.397203  0.788730
2010-01-01 09:00:07  0.316836  0.568099  0.869127  0.436173
2010-01-01 09:00:08  0.802148  0.143767  0.704261  0.704581
2010-01-01 09:00:09  0.218792  0.924868  0.442141  0.909316

[10 rows x 4 columns]

Previous API:

You would write a resampling operation that immediately evaluates. If a how parameter was not provided, it would default to how='mean'.

In [6]: df.resample('2s')
Out[6]:
                         A         B         C         D
2010-01-01 09:00:00  0.485748  0.447351  0.357096  0.793615
2010-01-01 09:00:02  0.820801  0.794317  0.364034  0.531096
2010-01-01 09:00:04  0.433985  0.314582  0.424104  0.625733
2010-01-01 09:00:06  0.624988  0.609738  0.633165  0.612452
2010-01-01 09:00:08  0.510470  0.534317  0.573201  0.806949

You could also specify a how directly

In [7]: df.resample('2s', how='sum')
Out[7]:
                         A         B         C         D
2010-01-01 09:00:00  0.971495  0.894701  0.714192  1.587231
2010-01-01 09:00:02  1.641602  1.588635  0.728068  1.062191
2010-01-01 09:00:04  0.867969  0.629165  0.848208  1.251465
2010-01-01 09:00:06  1.249976  1.219477  1.266330  1.224904
2010-01-01 09:00:08  1.020940  1.068634  1.146402  1.613897

New API:

Now, you can write .resample(..) as a 2-stage operation like .groupby(...), which yields a Resampler.

In [77]: r = df.resample('2s')

In [78]: r
Out[78]: <pandas.core.resample.DatetimeIndexResampler object at 0x7f9b03139750>

Downsampling#

You can then use this object to perform operations. These are downsampling operations (going from a higher frequency to a lower one).

In [79]: r.mean()
Out[79]: 
                            A         B         C         D
2010-01-01 09:00:00  0.485748  0.447351  0.357096  0.793615
2010-01-01 09:00:02  0.820801  0.794317  0.364034  0.531096
2010-01-01 09:00:04  0.433985  0.314582  0.424104  0.625733
2010-01-01 09:00:06  0.624988  0.609738  0.633165  0.612452
2010-01-01 09:00:08  0.510470  0.534317  0.573201  0.806949

[5 rows x 4 columns]

In [80]: r.sum()
Out[80]: 
                            A         B         C         D
2010-01-01 09:00:00  0.971495  0.894701  0.714192  1.587231
2010-01-01 09:00:02  1.641602  1.588635  0.728068  1.062191
2010-01-01 09:00:04  0.867969  0.629165  0.848208  1.251465
2010-01-01 09:00:06  1.249976  1.219477  1.266330  1.224904
2010-01-01 09:00:08  1.020940  1.068634  1.146402  1.613897

[5 rows x 4 columns]

Furthermore, resample now supports getitem operations to perform the resample on specific columns.

In [81]: r[['A','C']].mean()
Out[81]: 
                            A         C
2010-01-01 09:00:00  0.485748  0.357096
2010-01-01 09:00:02  0.820801  0.364034
2010-01-01 09:00:04  0.433985  0.424104
2010-01-01 09:00:06  0.624988  0.633165
2010-01-01 09:00:08  0.510470  0.573201

[5 rows x 2 columns]

and .aggregate type operations.

In [82]: r.agg({'A' : 'mean', 'B' : 'sum'})
Out[82]: 
                            A         B
2010-01-01 09:00:00  0.485748  0.894701
2010-01-01 09:00:02  0.820801  1.588635
2010-01-01 09:00:04  0.433985  0.629165
2010-01-01 09:00:06  0.624988  1.219477
2010-01-01 09:00:08  0.510470  1.068634

[5 rows x 2 columns]

These accessors can of course, be combined

In [83]: r[['A','B']].agg(['mean','sum'])
Out[83]: 
                            A                   B          
                         mean       sum      mean       sum
2010-01-01 09:00:00  0.485748  0.971495  0.447351  0.894701
2010-01-01 09:00:02  0.820801  1.641602  0.794317  1.588635
2010-01-01 09:00:04  0.433985  0.867969  0.314582  0.629165
2010-01-01 09:00:06  0.624988  1.249976  0.609738  1.219477
2010-01-01 09:00:08  0.510470  1.020940  0.534317  1.068634

[5 rows x 4 columns]

Upsampling#

Upsampling operations take you from a lower frequency to a higher frequency. These are now performed with the Resampler objects with backfill(), ffill(), fillna() and asfreq() methods.

In [89]: s = pd.Series(np.arange(5, dtype='int64'),
              index=pd.date_range('2010-01-01', periods=5, freq='Q'))

In [90]: s
Out[90]:
2010-03-31    0
2010-06-30    1
2010-09-30    2
2010-12-31    3
2011-03-31    4
Freq: Q-DEC, Length: 5, dtype: int64

Previously

In [6]: s.resample('M', fill_method='ffill')
Out[6]:
2010-03-31    0
2010-04-30    0
2010-05-31    0
2010-06-30    1
2010-07-31    1
2010-08-31    1
2010-09-30    2
2010-10-31    2
2010-11-30    2
2010-12-31    3
2011-01-31    3
2011-02-28    3
2011-03-31    4
Freq: M, dtype: int64

New API

In [91]: s.resample('M').ffill()
Out[91]:
2010-03-31    0
2010-04-30    0
2010-05-31    0
2010-06-30    1
2010-07-31    1
2010-08-31    1
2010-09-30    2
2010-10-31    2
2010-11-30    2
2010-12-31    3
2011-01-31    3
2011-02-28    3
2011-03-31    4
Freq: M, Length: 13, dtype: int64

Note

In the new API, you can either downsample OR upsample. The prior implementation would allow you to pass an aggregator function (like mean) even though you were upsampling, providing a bit of confusion.

Previous API will work but with deprecations#

Warning

This new API for resample includes some internal changes for the prior-to-0.18.0 API, to work with a deprecation warning in most cases, as the resample operation returns a deferred object. We can intercept operations and just do what the (pre 0.18.0) API did (with a warning). Here is a typical use case:

In [4]: r = df.resample('2s')

In [6]: r*10
pandas/tseries/resample.py:80: FutureWarning: .resample() is now a deferred operation
use .resample(...).mean() instead of .resample(...)

Out[6]:
                      A         B         C         D
2010-01-01 09:00:00  4.857476  4.473507  3.570960  7.936154
2010-01-01 09:00:02  8.208011  7.943173  3.640340  5.310957
2010-01-01 09:00:04  4.339846  3.145823  4.241039  6.257326
2010-01-01 09:00:06  6.249881  6.097384  6.331650  6.124518
2010-01-01 09:00:08  5.104699  5.343172  5.732009  8.069486

However, getting and assignment operations directly on a Resampler will raise a ValueError:

In [7]: r.iloc[0] = 5
ValueError: .resample() is now a deferred operation
use .resample(...).mean() instead of .resample(...)

There is a situation where the new API can not perform all the operations when using original code. This code is intending to resample every 2s, take the mean AND then take the min of those results.

In [4]: df.resample('2s').min()
Out[4]:
A    0.433985
B    0.314582
C    0.357096
D    0.531096
dtype: float64

The new API will:

In [84]: df.resample('2s').min()
Out[84]: 
                            A         B         C         D
2010-01-01 09:00:00  0.191519  0.272593  0.276464  0.785359
2010-01-01 09:00:02  0.683463  0.712702  0.357817  0.500995
2010-01-01 09:00:04  0.364886  0.013768  0.075381  0.368824
2010-01-01 09:00:06  0.316836  0.568099  0.397203  0.436173
2010-01-01 09:00:08  0.218792  0.143767  0.442141  0.704581

[5 rows x 4 columns]

The good news is the return dimensions will differ between the new API and the old API, so this should loudly raise an exception.

To replicate the original operation

In [85]: df.resample('2s').mean().min()
Out[85]: 
A    0.433985
B    0.314582
C    0.357096
D    0.531096
Length: 4, dtype: float64

Changes to eval#

In prior versions, new columns assignments in an eval expression resulted in an inplace change to the DataFrame. (GH 9297, GH 8664, GH 10486)

In [86]: df = pd.DataFrame({'a': np.linspace(0, 10, 5), 'b': range(5)})

In [87]: df
Out[87]: 
      a  b
0   0.0  0
1   2.5  1
2   5.0  2
3   7.5  3
4  10.0  4

[5 rows x 2 columns]

In [12]: df.eval('c = a + b')
FutureWarning: eval expressions containing an assignment currentlydefault to operating inplace.
This will change in a future version of pandas, use inplace=True to avoid this warning.

In [13]: df
Out[13]:
      a  b     c
0   0.0  0   0.0
1   2.5  1   3.5
2   5.0  2   7.0
3   7.5  3  10.5
4  10.0  4  14.0

In version 0.18.0, a new inplace keyword was added to choose whether the assignment should be done inplace or return a copy.

In [88]: df
Out[88]: 
      a  b     c
0   0.0  0   0.0
1   2.5  1   3.5
2   5.0  2   7.0
3   7.5  3  10.5
4  10.0  4  14.0

[5 rows x 3 columns]

In [89]: df.eval('d = c - b', inplace=False)
Out[89]: 
      a  b     c     d
0   0.0  0   0.0   0.0
1   2.5  1   3.5   2.5
2   5.0  2   7.0   5.0
3   7.5  3  10.5   7.5
4  10.0  4  14.0  10.0

[5 rows x 4 columns]

In [90]: df
Out[90]: 
      a  b     c
0   0.0  0   0.0
1   2.5  1   3.5
2   5.0  2   7.0
3   7.5  3  10.5
4  10.0  4  14.0

[5 rows x 3 columns]

In [91]: df.eval('d = c - b', inplace=True)

In [92]: df
Out[92]: 
      a  b     c     d
0   0.0  0   0.0   0.0
1   2.5  1   3.5   2.5
2   5.0  2   7.0   5.0
3   7.5  3  10.5   7.5
4  10.0  4  14.0  10.0

[5 rows x 4 columns]

Warning

For backwards compatibility, inplace defaults to True if not specified. This will change in a future version of pandas. If your code depends on an inplace assignment you should update to explicitly set inplace=True

The inplace keyword parameter was also added the query method.

In [93]: df.query('a > 5')
Out[93]: 
      a  b     c     d
3   7.5  3  10.5   7.5
4  10.0  4  14.0  10.0

[2 rows x 4 columns]

In [94]: df.query('a > 5', inplace=True)

In [95]: df
Out[95]: 
      a  b     c     d
3   7.5  3  10.5   7.5
4  10.0  4  14.0  10.0

[2 rows x 4 columns]

Warning

Note that the default value for inplace in a query is False, which is consistent with prior versions.

eval has also been updated to allow multi-line expressions for multiple assignments. These expressions will be evaluated one at a time in order. Only assignments are valid for multi-line expressions.

In [96]: df
Out[96]: 
      a  b     c     d
3   7.5  3  10.5   7.5
4  10.0  4  14.0  10.0

[2 rows x 4 columns]

In [97]: df.eval("""
   ....: e = d + a
   ....: f = e - 22
   ....: g = f / 2.0""", inplace=True)
   ....: 

In [98]: df
Out[98]: 
      a  b     c     d     e    f    g
3   7.5  3  10.5   7.5  15.0 -7.0 -3.5
4  10.0  4  14.0  10.0  20.0 -2.0 -1.0

[2 rows x 7 columns]

Other API changes#

DataFrame.between_time and Series.between_time now only parse a fixed set of time strings. Parsing of date strings is no longer supported and raises a ValueError. (GH 11818)

In [107]: s = pd.Series(range(10), pd.date_range('2015-01-01', freq='H', periods=10))

In [108]: s.between_time("7:00am", "9:00am")
Out[108]:
2015-01-01 07:00:00    7
2015-01-01 08:00:00    8
2015-01-01 09:00:00    9
Freq: H, Length: 3, dtype: int64

This will now raise.

In [2]: s.between_time('20150101 07:00:00','20150101 09:00:00')
ValueError: Cannot convert arg ['20150101 07:00:00'] to a time.

.memory_usage() now includes values in the index, as does memory_usage in .info() (GH 11597)
DataFrame.to_latex() now supports non-ascii encodings (eg utf-8) in Python 2 with the parameter encoding (GH 7061)
pandas.merge() and DataFrame.merge() will show a specific error message when trying to merge with an object that is not of type DataFrame or a subclass (GH 12081)
DataFrame.unstack and Series.unstack now take fill_value keyword to allow direct replacement of missing values when an unstack results in missing values in the resulting DataFrame. As an added benefit, specifying fill_value will preserve the data type of the original stacked data. (GH 9746)
As part of the new API for window functions and resampling, aggregation functions have been clarified, raising more informative error messages on invalid aggregations. (GH 9052). A full set of examples are presented in groupby.
Statistical functions for NDFrame objects (like sum(), mean(), min()) will now raise if non-numpy-compatible arguments are passed in for **kwargs (GH 12301)
.to_latex and .to_html gain a decimal parameter like .to_csv; the default is '.' (GH 12031)
More helpful error message when constructing a DataFrame with empty data but with indices (GH 8020)
.describe() will now properly handle bool dtype as a categorical (GH 6625)
More helpful error message with an invalid .transform with user defined input (GH 10165)
Exponentially weighted functions now allow specifying alpha directly (GH 10789) and raise ValueError if parameters violate 0 < alpha <= 1 (GH 12492)

Deprecations#

The functions pd.rolling_*, pd.expanding_*, and pd.ewm* are deprecated and replaced by the corresponding method call. Note that the new suggested syntax includes all of the arguments (even if default) (GH 11603)

In [1]: s = pd.Series(range(3))

In [2]: pd.rolling_mean(s,window=2,min_periods=1)
        FutureWarning: pd.rolling_mean is deprecated for Series and
             will be removed in a future version, replace with
             Series.rolling(min_periods=1,window=2,center=False).mean()
Out[2]:
        0    0.0
        1    0.5
        2    1.5
        dtype: float64

In [3]: pd.rolling_cov(s, s, window=2)
        FutureWarning: pd.rolling_cov is deprecated for Series and
             will be removed in a future version, replace with
             Series.rolling(window=2).cov(other=<Series>)
Out[3]:
        0    NaN
        1    0.5
        2    0.5
        dtype: float64

The freq and how arguments to the .rolling, .expanding, and .ewm (new) functions are deprecated, and will be removed in a future version. You can simply resample the input prior to creating a window function. (GH 11603).

For example, instead of s.rolling(window=5,freq='D').max() to get the max value on a rolling 5 Day window, one could use s.resample('D').mean().rolling(window=5).max(), which first resamples the data to daily data, then provides a rolling 5 day window.
pd.tseries.frequencies.get_offset_name function is deprecated. Use offset’s .freqstr property as alternative (GH 11192)
pandas.stats.fama_macbeth routines are deprecated and will be removed in a future version (GH 6077)
pandas.stats.ols, pandas.stats.plm and pandas.stats.var routines are deprecated and will be removed in a future version (GH 6077)
show a FutureWarning rather than a DeprecationWarning on using long-time deprecated syntax in HDFStore.select, where the where clause is not a string-like (GH 12027)
The pandas.options.display.mpl_style configuration has been deprecated and will be removed in a future version of pandas. This functionality is better handled by matplotlib’s style sheets (GH 11783).

Removal of deprecated float indexers#

In GH 4892 indexing with floating point numbers on a non-Float64Index was deprecated (in version 0.14.0). In 0.18.0, this deprecation warning is removed and these will now raise a TypeError. (GH 12165, GH 12333)

In [99]: s = pd.Series([1, 2, 3], index=[4, 5, 6])

In [100]: s
Out[100]: 
4    1
5    2
6    3
Length: 3, dtype: int64

In [101]: s2 = pd.Series([1, 2, 3], index=list('abc'))

In [102]: s2
Out[102]: 
a    1
b    2
c    3
Length: 3, dtype: int64

Previous behavior:

# this is label indexing
In [2]: s[5.0]
FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
Out[2]: 2

# this is positional indexing
In [3]: s.iloc[1.0]
FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
Out[3]: 2

# this is label indexing
In [4]: s.loc[5.0]
FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
Out[4]: 2

# .ix would coerce 1.0 to the positional 1, and index
In [5]: s2.ix[1.0] = 10
FutureWarning: scalar indexers for index type Index should be integers and not floating point

In [6]: s2
Out[6]:
a     1
b    10
c     3
dtype: int64

New behavior:

For iloc, getting & setting via a float scalar will always raise.

In [3]: s.iloc[2.0]
TypeError: cannot do label indexing on <class 'pandas.indexes.numeric.Int64Index'> with these indexers [2.0] of <type 'float'>

Other indexers will coerce to a like integer for both getting and setting. The FutureWarning has been dropped for .loc, .ix and [].

In [103]: s[5.0]
Out[103]: 2

In [104]: s.loc[5.0]
Out[104]: 2

and setting

In [105]: s_copy = s.copy()

In [106]: s_copy[5.0] = 10

In [107]: s_copy
Out[107]: 
4     1
5    10
6     3
Length: 3, dtype: int64

In [108]: s_copy = s.copy()

In [109]: s_copy.loc[5.0] = 10

In [110]: s_copy
Out[110]: 
4     1
5    10
6     3
Length: 3, dtype: int64

Positional setting with .ix and a float indexer will ADD this value to the index, rather than previously setting the value by position.

In [3]: s2.ix[1.0] = 10
In [4]: s2
Out[4]:
a       1
b       2
c       3
1.0    10
dtype: int64

Slicing will also coerce integer-like floats to integers for a non-Float64Index.

In [111]: s.loc[5.0:6]
Out[111]: 
5    2
6    3
Length: 2, dtype: int64

Note that for floats that are NOT coercible to ints, the label based bounds will be excluded

In [112]: s.loc[5.1:6]
Out[112]: 
6    3
Length: 1, dtype: int64

Float indexing on a Float64Index is unchanged.

In [113]: s = pd.Series([1, 2, 3], index=np.arange(3.))

In [114]: s[1.0]
Out[114]: 2

In [115]: s[1.0:2.5]
Out[115]: 
1.0    2
2.0    3
Length: 2, dtype: int64

Removal of prior version deprecations/changes#

Removal of rolling_corr_pairwise in favor of .rolling().corr(pairwise=True) (GH 4950)
Removal of expanding_corr_pairwise in favor of .expanding().corr(pairwise=True) (GH 4950)
Removal of DataMatrix module. This was not imported into the pandas namespace in any event (GH 12111)
Removal of cols keyword in favor of subset in DataFrame.duplicated() and DataFrame.drop_duplicates() (GH 6680)
Removal of the read_frame and frame_query (both aliases for pd.read_sql) and write_frame (alias of to_sql) functions in the pd.io.sql namespace, deprecated since 0.14.0 (GH 6292).
Removal of the order keyword from .factorize() (GH 6930)

Performance improvements#

Improved performance of andrews_curves (GH 11534)
Improved huge DatetimeIndex, PeriodIndex and TimedeltaIndex’s ops performance including NaT (GH 10277)
Improved performance of pandas.concat (GH 11958)
Improved performance of StataReader (GH 11591)
Improved performance in construction of Categoricals with Series of datetimes containing NaT (GH 12077)
Improved performance of ISO 8601 date parsing for dates without separators (GH 11899), leading zeros (GH 11871) and with white space preceding the time zone (GH 9714)

Bug fixes#

Bug in GroupBy.size when data-frame is empty. (GH 11699)
Bug in Period.end_time when a multiple of time period is requested (GH 11738)
Regression in .clip with tz-aware datetimes (GH 11838)
Bug in date_range when the boundaries fell on the frequency (GH 11804, GH 12409)
Bug in consistency of passing nested dicts to .groupby(...).agg(...) (GH 9052)
Accept unicode in Timedelta constructor (GH 11995)
Bug in value label reading for StataReader when reading incrementally (GH 12014)
Bug in vectorized DateOffset when n parameter is 0 (GH 11370)
Compat for numpy 1.11 w.r.t. NaT comparison changes (GH 12049)
Bug in read_csv when reading from a StringIO in threads (GH 11790)
Bug in not treating NaT as a missing value in datetimelikes when factorizing & with Categoricals (GH 12077)
Bug in getitem when the values of a Series were tz-aware (GH 12089)
Bug in Series.str.get_dummies when one of the variables was ‘name’ (GH 12180)
Bug in pd.concat while concatenating tz-aware NaT series. (GH 11693, GH 11755, GH 12217)
Bug in pd.read_stata with version <= 108 files (GH 12232)
Bug in Series.resample using a frequency of Nano when the index is a DatetimeIndex and contains non-zero nanosecond parts (GH 12037)
Bug in resampling with .nunique and a sparse index (GH 12352)
Removed some compiler warnings (GH 12471)
Work around compat issues with boto in python 3.5 (GH 11915)
Bug in NaT subtraction from Timestamp or DatetimeIndex with timezones (GH 11718)
Bug in subtraction of Series of a single tz-aware Timestamp (GH 12290)
Use compat iterators in PY2 to support .next() (GH 12299)
Bug in Timedelta.round with negative values (GH 11690)
Bug in .loc against CategoricalIndex may result in normal Index (GH 11586)
Bug in DataFrame.info when duplicated column names exist (GH 11761)
Bug in .copy of datetime tz-aware objects (GH 11794)
Bug in Series.apply and Series.map where timedelta64 was not boxed (GH 11349)
Bug in DataFrame.set_index() with tz-aware Series (GH 12358)
Bug in subclasses of DataFrame where AttributeError did not propagate (GH 11808)
Bug groupby on tz-aware data where selection not returning Timestamp (GH 11616)
Bug in pd.read_clipboard and pd.to_clipboard functions not supporting Unicode; upgrade included pyperclip to v1.5.15 (GH 9263)
Bug in DataFrame.query containing an assignment (GH 8664)
Bug in from_msgpack where __contains__() fails for columns of the unpacked DataFrame, if the DataFrame has object columns. (GH 11880)
Bug in .resample on categorical data with TimedeltaIndex (GH 12169)
Bug in timezone info lost when broadcasting scalar datetime to DataFrame (GH 11682)
Bug in Index creation from Timestamp with mixed tz coerces to UTC (GH 11488)
Bug in to_numeric where it does not raise if input is more than one dimension (GH 11776)
Bug in parsing timezone offset strings with non-zero minutes (GH 11708)
Bug in df.plot using incorrect colors for bar plots under matplotlib 1.5+ (GH 11614)
Bug in the groupby plot method when using keyword arguments (GH 11805).
Bug in DataFrame.duplicated and drop_duplicates causing spurious matches when setting keep=False (GH 11864)
Bug in .loc result with duplicated key may have Index with incorrect dtype (GH 11497)
Bug in pd.rolling_median where memory allocation failed even with sufficient memory (GH 11696)
Bug in DataFrame.style with spurious zeros (GH 12134)
Bug in DataFrame.style with integer columns not starting at 0 (GH 12125)
Bug in .style.bar may not rendered properly using specific browser (GH 11678)
Bug in rich comparison of Timedelta with a numpy.array of Timedelta that caused an infinite recursion (GH 11835)
Bug in DataFrame.round dropping column index name (GH 11986)
Bug in df.replace while replacing value in mixed dtype Dataframe (GH 11698)
Bug in Index prevents copying name of passed Index, when a new name is not provided (GH 11193)
Bug in read_excel failing to read any non-empty sheets when empty sheets exist and sheetname=None (GH 11711)
Bug in read_excel failing to raise NotImplemented error when keywords parse_dates and date_parser are provided (GH 11544)
Bug in read_sql with pymysql connections failing to return chunked data (GH 11522)
Bug in .to_csv ignoring formatting parameters decimal, na_rep, float_format for float indexes (GH 11553)
Bug in Int64Index and Float64Index preventing the use of the modulo operator (GH 9244)
Bug in MultiIndex.drop for not lexsorted MultiIndexes (GH 12078)
Bug in DataFrame when masking an empty DataFrame (GH 11859)
Bug in .plot potentially modifying the colors input when the number of columns didn’t match the number of series provided (GH 12039).
Bug in Series.plot failing when index has a CustomBusinessDay frequency (GH 7222).
Bug in .to_sql for datetime.time values with sqlite fallback (GH 8341)
Bug in read_excel failing to read data with one column when squeeze=True (GH 12157)
Bug in read_excel failing to read one empty column (GH 12292, GH 9002)
Bug in .groupby where a KeyError was not raised for a wrong column if there was only one row in the dataframe (GH 11741)
Bug in .read_csv with dtype specified on empty data producing an error (GH 12048)
Bug in .read_csv where strings like '2E' are treated as valid floats (GH 12237)
Bug in building pandas with debugging symbols (GH 12123)
Removed millisecond property of DatetimeIndex. This would always raise a ValueError (GH 12019).
Bug in Series constructor with read-only data (GH 11502)
Removed pandas._testing.choice(). Should use np.random.choice(), instead. (GH 12386)
Bug in .loc setitem indexer preventing the use of a TZ-aware DatetimeIndex (GH 12050)
Bug in .style indexes and MultiIndexes not appearing (GH 11655)
Bug in to_msgpack and from_msgpack which did not correctly serialize or deserialize NaT (GH 12307).
Bug in .skew and .kurt due to roundoff error for highly similar values (GH 11974)
Bug in Timestamp constructor where microsecond resolution was lost if HHMMSS were not separated with ‘:’ (GH 10041)
Bug in buffer_rd_bytes src->buffer could be freed more than once if reading failed, causing a segfault (GH 12098)
Bug in crosstab where arguments with non-overlapping indexes would return a KeyError (GH 10291)
Bug in DataFrame.apply in which reduction was not being prevented for cases in which dtype was not a numpy dtype (GH 12244)
Bug when initializing categorical series with a scalar value. (GH 12336)
Bug when specifying a UTC DatetimeIndex by setting utc=True in .to_datetime (GH 11934)
Bug when increasing the buffer size of CSV reader in read_csv (GH 12494)
Bug when setting columns of a DataFrame with duplicate column names (GH 12344)

Contributors#

A total of 101 people contributed patches to this release. People with a “+” by their names contributed a patch for the first time.

ARF +
Alex Alekseyev +
Andrew McPherson +
Andrew Rosenfeld
Andy Hayden
Anthonios Partheniou
Anton I. Sipos
Ben +
Ben North +
Bran Yang +
Chris
Chris Carroux +
Christopher C. Aycock +
Christopher Scanlin +
Cody +
Da Wang +
Daniel Grady +
Dorozhko Anton +
Dr-Irv +
Erik M. Bray +
Evan Wright
Francis T. O’Donovan +
Frank Cleary +
Gianluca Rossi
Graham Jeffries +
Guillaume Horel
Henry Hammond +
Isaac Schwabacher +
Jean-Mathieu Deschenes
Jeff Reback
Joe Jevnik +
John Freeman +
John Fremlin +
Jonas Hoersch +
Joris Van den Bossche
Joris Vankerschaver
Justin Lecher
Justin Lin +
Ka Wo Chen
Keming Zhang +
Kerby Shedden
Kyle +
Marco Farrugia +
MasonGallo +
MattRijk +
Matthew Lurie +
Maximilian Roos
Mayank Asthana +
Mortada Mehyar
Moussa Taifi +
Navreet Gill +
Nicolas Bonnotte
Paul Reiners +
Philip Gura +
Pietro Battiston
RahulHP +
Randy Carnevale
Rinoc Johnson
Rishipuri +
Sangmin Park +
Scott E Lasley
Sereger13 +
Shannon Wang +
Skipper Seabold
Thierry Moisan
Thomas A Caswell
Toby Dylan Hocking +
Tom Augspurger
Travis +
Trent Hauck
Tux1
Varun
Wes McKinney
Will Thompson +
Yoav Ram
Yoong Kang Lim +
Yoshiki Vázquez Baeza
Young Joong Kim +
Younggun Kim
Yuval Langer +
alex argunov +
behzad nouri
boombard +
brian-pantano +
chromy +
daniel +
dgram0 +
gfyoung +
hack-c +
hcontrast +
jfoo +
kaustuv deolal +
llllllllll
ranarag +
rockg
scls19fr
seales +
sinhrks
srib +
surveymedia.ca +
tworec +

Version 0.18.0 (March 13, 2016)#

New features#

Window functions are now methods#

Changes to rename#

Range Index#

Changes to str.extract#

Addition of str.extractall#

Changes to str.cat#

Datetimelike rounding#

Formatting of integers in FloatIndex#

Changes to dtype assignment behaviors#

Method to_xarray#

Latex representation#

pd.read_sas() changes#

Other enhancements#

Backwards incompatible API changes#

NaT and Timedelta operations#

Changes to msgpack#

Signature change for .rank#

Bug in QuarterBegin with n=0#

Resample API#

Downsampling#

Upsampling#

Previous API will work but with deprecations#

Changes to eval#

Other API changes#

Deprecations#

Removal of deprecated float indexers#

Removal of prior version deprecations/changes#

Performance improvements#

Bug fixes#

Contributors#

`pd.read_sas()` changes#