These are the changes in pandas 1.2.0. See Release notes for a full changelog including other versions of pandas.
Warning
The xlwt package for writing old-style .xls excel files is no longer maintained. The xlrd package is now only for reading old-style .xls files.
.xls
Previously, the default argument engine=None to read_excel() would result in using the xlrd engine in many cases, including new Excel 2007+ (.xlsx) files. If openpyxl is installed, many of these cases will now default to using the openpyxl engine. See the read_excel() documentation for more details.
engine=None
read_excel()
xlrd
.xlsx
openpyxl
Thus, it is strongly encouraged to install openpyxl to read Excel 2007+ (.xlsx) files. Please do not report issues when using ``xlrd`` to read ``.xlsx`` files. This is no longer supported, switch to using openpyxl instead.
Attempting to use the xlwt engine will raise a FutureWarning unless the option io.excel.xls.writer is set to "xlwt". While this option is now deprecated and will also raise a FutureWarning, it can be globally set and the warning suppressed. Users are recommended to write .xlsx files using the openpyxl engine instead.
xlwt
FutureWarning
io.excel.xls.writer
"xlwt"
Series and DataFrame can now be created with allows_duplicate_labels=False flag to control whether the index or columns can contain duplicate labels (GH28394). This can be used to prevent accidental introduction of duplicate labels, which can affect downstream operations.
Series
DataFrame
allows_duplicate_labels=False
By default, duplicates continue to be allowed.
In [1]: pd.Series([1, 2], index=['a', 'a']) Out[1]: a 1 a 2 Length: 2, dtype: int64 In [2]: pd.Series([1, 2], index=['a', 'a']).set_flags(allows_duplicate_labels=False) ... DuplicateLabelError: Index has duplicates. positions label a [0, 1]
pandas will propagate the allows_duplicate_labels property through many operations.
allows_duplicate_labels
In [3]: a = ( ...: pd.Series([1, 2], index=['a', 'b']) ...: .set_flags(allows_duplicate_labels=False) ...: ) In [4]: a Out[4]: a 1 b 2 Length: 2, dtype: int64 # An operation introducing duplicates In [5]: a.reindex(['a', 'b', 'a']) ... DuplicateLabelError: Index has duplicates. positions label a [0, 2] [1 rows x 1 columns]
This is an experimental feature. Currently, many methods fail to propagate the allows_duplicate_labels value. In future versions it is expected that every method taking or returning one or more DataFrame or Series objects will propagate allows_duplicate_labels.
See Duplicate Labels for more.
The allows_duplicate_labels flag is stored in the new DataFrame.flags attribute. This stores global attributes that apply to the pandas object. This differs from DataFrame.attrs, which stores information that applies to the dataset.
DataFrame.flags
DataFrame.attrs
Many read/write functions have acquired the storage_options optional argument, to pass a dictionary of parameters to the storage backend. This allows, for example, for passing credentials to S3 and GCS storage. The details of what parameters can be passed to which backends can be found in the documentation of the individual storage backends (detailed from the fsspec docs for builtin implementations and linked to external ones). See Section Reading/writing remote files.
storage_options
GH35655 added fsspec support (including storage_options) for reading excel files.
to_csv
to_csv() supports file handles in binary mode (GH19827 and GH35058) with encoding (GH13068 and GH23854) and compression (GH22555). If pandas does not automatically detect whether the file handle is opened in binary or text mode, it is necessary to provide mode="wb".
to_csv()
encoding
compression
mode="wb"
For example:
In [1]: import io In [2]: data = pd.DataFrame([0, 1, 2]) In [3]: buffer = io.BytesIO() In [4]: data.to_csv(buffer, encoding="utf-8", compression="gzip")
to_latex
DataFrame.to_latex() now allows one to specify a floating table position (GH35281) and a short caption (GH36267).
DataFrame.to_latex()
The keyword position has been added to set the position.
position
In [5]: data = pd.DataFrame({'a': [1, 2], 'b': [3, 4]}) In [6]: table = data.to_latex(position='ht') In [7]: print(table) \begin{table}[ht] \centering \begin{tabular}{lrr} \toprule {} & a & b \\ \midrule 0 & 1 & 3 \\ 1 & 2 & 4 \\ \bottomrule \end{tabular} \end{table}
Usage of the keyword caption has been extended. Besides taking a single string as an argument, one can optionally provide a tuple (full_caption, short_caption) to add a short caption macro.
caption
(full_caption, short_caption)
In [8]: data = pd.DataFrame({'a': [1, 2], 'b': [3, 4]}) In [9]: table = data.to_latex(caption=('the full long caption', 'short caption')) In [10]: print(table) \begin{table} \centering \caption[short caption]{the full long caption} \begin{tabular}{lrr} \toprule {} & a & b \\ \midrule 0 & 1 & 3 \\ 1 & 2 & 4 \\ \bottomrule \end{tabular} \end{table}
read_csv
read_table
For the C parsing engine, the methods read_csv() and read_table() previously defaulted to a parser that could read floating point numbers slightly incorrectly with respect to the last bit in precision. The option floating_precision="high" has always been available to avoid this issue. Beginning with this version, the default is now to use the more accurate parser by making floating_precision=None correspond to the high precision parser, and the new option floating_precision="legacy" to use the legacy parser. The change to using the higher precision parser by default should have no impact on performance. (GH17154)
read_csv()
read_table()
floating_precision="high"
floating_precision=None
floating_precision="legacy"
We’ve added Float32Dtype / Float64Dtype and FloatingArray. These are extension data types dedicated to floating point data that can hold the pd.NA missing value indicator (GH32265, GH34307).
Float32Dtype
Float64Dtype
FloatingArray
pd.NA
While the default float data type already supports missing values using np.nan, these new data types use pd.NA (and its corresponding behavior) as the missing value indicator, in line with the already existing nullable integer and boolean data types.
np.nan
One example where the behavior of np.nan and pd.NA is different is comparison operations:
# the default NumPy float64 dtype In [11]: s1 = pd.Series([1.5, None]) In [12]: s1 Out[12]: 0 1.5 1 NaN Length: 2, dtype: float64 In [13]: s1 > 1 Out[13]: 0 True 1 False Length: 2, dtype: bool
# the new nullable float64 dtype In [14]: s2 = pd.Series([1.5, None], dtype="Float64") In [15]: s2 Out[15]: 0 1.5 1 <NA> Length: 2, dtype: Float64 In [16]: s2 > 1 Out[16]: 0 True 1 <NA> Length: 2, dtype: boolean
See the Experimental NA scalar to denote missing values doc section for more details on the behavior when using the pd.NA missing value indicator.
As shown above, the dtype can be specified using the “Float64” or “Float32” string (capitalized to distinguish it from the default “float64” data type). Alternatively, you can also use the dtype object:
In [17]: pd.Series([1.5, None], dtype=pd.Float32Dtype()) Out[17]: 0 1.5 1 <NA> Length: 2, dtype: Float32
Operations with the existing integer or boolean nullable data types that give float results will now also use the nullable floating data types (GH38178).
Experimental: the new floating data types are currently experimental, and their behavior or API may still change without warning. Especially the behavior regarding NaN (distinct from NA missing values) is subject to change.
When aggregating using concat() or the DataFrame constructor, pandas will now attempt to preserve index and column names whenever possible (GH35847). In the case where all inputs share a common name, this name will be assigned to the result. When the input names do not all agree, the result will be unnamed. Here is an example where the index name is preserved:
concat()
In [18]: idx = pd.Index(range(5), name='abc') In [19]: ser = pd.Series(range(5, 10), index=idx) In [20]: pd.concat({'x': ser[1:], 'y': ser[:-1]}, axis=1) Out[20]: x y abc 0 NaN 5.0 1 6.0 6.0 2 7.0 7.0 3 8.0 8.0 4 9.0 NaN [5 rows x 2 columns]
The same is true for MultiIndex, but the logic is applied separately on a level-by-level basis.
MultiIndex
DataFrameGroupBy now supports exponentially weighted window operations directly (GH16037).
DataFrameGroupBy
In [21]: df = pd.DataFrame({'A': ['a', 'b', 'a', 'b'], 'B': range(4)}) In [22]: df Out[22]: A B 0 a 0 1 b 1 2 a 2 3 b 3 [4 rows x 2 columns] In [23]: df.groupby('A').ewm(com=1.0).mean() Out[23]: B A a 0 0.000000 2 1.333333 b 1 1.000000 3 2.333333 [4 rows x 1 columns]
Additionally mean supports execution via Numba with the engine and engine_kwargs arguments. Numba must be installed as an optional dependency to use this feature.
mean
engine
engine_kwargs
Added day_of_week (compatibility alias dayofweek) property to Timestamp, DatetimeIndex, Period, PeriodIndex (GH9605)
day_of_week
dayofweek
Timestamp
DatetimeIndex
Period
PeriodIndex
Added day_of_year (compatibility alias dayofyear) property to Timestamp, DatetimeIndex, Period, PeriodIndex (GH9605)
day_of_year
dayofyear
Added set_flags() for setting table-wide flags on a Series or DataFrame (GH28394)
set_flags()
DataFrame.applymap() now supports na_action (GH23803)
DataFrame.applymap()
na_action
Index with object dtype supports division and multiplication (GH34160)
Index
io.sql.get_schema() now supports a schema keyword argument that will add a schema into the create table statement (GH28486)
io.sql.get_schema()
schema
DataFrame.explode() and Series.explode() now support exploding of sets (GH35614)
DataFrame.explode()
Series.explode()
DataFrame.hist() now supports time series (datetime) data (GH32590)
DataFrame.hist()
Styler.set_table_styles() now allows the direct styling of rows and columns and can be chained (GH35607)
Styler.set_table_styles()
Styler now allows direct CSS class name addition to individual data cells (GH36159)
Styler
Rolling.mean() and Rolling.sum() use Kahan summation to calculate the mean to avoid numerical problems (GH10319, GH11645, GH13254, GH32761, GH36031)
Rolling.mean()
Rolling.sum()
DatetimeIndex.searchsorted(), TimedeltaIndex.searchsorted(), PeriodIndex.searchsorted(), and Series.searchsorted() with datetime-like dtypes will now try to cast string arguments (list-like and scalar) to the matching datetime-like type (GH36346)
DatetimeIndex.searchsorted()
TimedeltaIndex.searchsorted()
PeriodIndex.searchsorted()
Series.searchsorted()
Added methods IntegerArray.prod(), IntegerArray.min(), and IntegerArray.max() (GH33790)
IntegerArray.prod()
IntegerArray.min()
IntegerArray.max()
Calling a NumPy ufunc on a DataFrame with extension types now preserves the extension types when possible (GH23743)
Calling a binary-input NumPy ufunc on multiple DataFrame objects now aligns, matching the behavior of binary operations and ufuncs on Series (GH23743). This change has been reverted in pandas 1.2.1, and the behaviour to not align DataFrames is deprecated instead, see the the 1.2.1 release notes.
Where possible RangeIndex.difference() and RangeIndex.symmetric_difference() will return RangeIndex instead of Int64Index (GH36564)
RangeIndex.difference()
RangeIndex.symmetric_difference()
RangeIndex
Int64Index
DataFrame.to_parquet() now supports MultiIndex for columns in parquet format (GH34777)
DataFrame.to_parquet()
read_parquet() gained a use_nullable_dtypes=True option to use nullable dtypes that use pd.NA as missing value indicator where possible for the resulting DataFrame (default is False, and only applicable for engine="pyarrow") (GH31242)
read_parquet()
use_nullable_dtypes=True
False
engine="pyarrow"
Added Rolling.sem() and Expanding.sem() to compute the standard error of the mean (GH26476)
Rolling.sem()
Expanding.sem()
Rolling.var() and Rolling.std() use Kahan summation and Welford’s Method to avoid numerical issues (GH37051)
Rolling.var()
Rolling.std()
DataFrame.corr() and DataFrame.cov() use Welford’s Method to avoid numerical issues (GH37448)
DataFrame.corr()
DataFrame.cov()
DataFrame.plot() now recognizes xlabel and ylabel arguments for plots of type scatter and hexbin (GH37001)
DataFrame.plot()
xlabel
ylabel
scatter
hexbin
DataFrame now supports the divmod operation (GH37165)
divmod
DataFrame.to_parquet() now returns a bytes object when no path argument is passed (GH37105)
bytes
path
Rolling now supports the closed argument for fixed windows (GH34315)
Rolling
closed
DatetimeIndex and Series with datetime64 or datetime64tz dtypes now support std (GH37436)
datetime64
datetime64tz
std
Window now supports all Scipy window types in win_type with flexible keyword argument support (GH34556)
Window
win_type
testing.assert_index_equal() now has a check_order parameter that allows indexes to be checked in an order-insensitive manner (GH37478)
testing.assert_index_equal()
check_order
read_csv() supports memory-mapping for compressed files (GH37621)
Add support for min_count keyword for DataFrame.groupby() and DataFrame.resample() for functions min, max, first and last (GH37821, GH37768)
min_count
DataFrame.groupby()
DataFrame.resample()
min
max
first
last
Improve error reporting for DataFrame.merge() when invalid merge column definitions were given (GH16228)
DataFrame.merge()
Improve numerical stability for Rolling.skew(), Rolling.kurt(), Expanding.skew() and Expanding.kurt() through implementation of Kahan summation (GH6929)
Rolling.skew()
Rolling.kurt()
Expanding.skew()
Expanding.kurt()
Improved error reporting for subsetting columns of a DataFrameGroupBy with axis=1 (GH37725)
axis=1
Implement method cross for DataFrame.merge() and DataFrame.join() (GH5401)
cross
DataFrame.join()
When read_csv(), read_sas() and read_json() are called with chunksize/iterator they can be used in a with statement as they return context-managers (GH38225)
read_sas()
read_json()
chunksize
iterator
with
Augmented the list of named colors available for styling Excel exports, enabling all of CSS4 colors (GH38247)
These are bug fixes that might have notable behavior changes.
DataFrame.any() and DataFrame.all() with bool_only=True now determines whether to exclude object-dtype columns on a column-by-column basis, instead of checking if all object-dtype columns can be considered boolean.
DataFrame.any()
DataFrame.all()
bool_only=True
This prevents pathological behavior where applying the reduction on a subset of columns could result in a larger Series result. See (GH37799).
In [24]: df = pd.DataFrame({"A": ["foo", "bar"], "B": [True, False]}, dtype=object) In [25]: df["C"] = pd.Series([True, True])
Previous behavior:
In [5]: df.all(bool_only=True) Out[5]: C True dtype: bool In [6]: df[["B", "C"]].all(bool_only=True) Out[6]: B False C True dtype: bool
New behavior:
In [26]: In [5]: df.all(bool_only=True) Out[26]: B False C True Length: 2, dtype: bool In [27]: In [6]: df[["B", "C"]].all(bool_only=True) Out[27]: B False C True Length: 2, dtype: bool
Other DataFrame reductions with numeric_only=None will also avoid this pathological behavior (GH37827):
numeric_only=None
In [28]: df = pd.DataFrame({"A": [0, 1, 2], "B": ["a", "b", "c"]}, dtype=object)
In [3]: df.mean() Out[3]: Series([], dtype: float64) In [4]: df[["A"]].mean() Out[4]: A 1.0 dtype: float64
In [29]: df.mean() Out[29]: A 1.0 Length: 1, dtype: float64 In [30]: df[["A"]].mean() Out[30]: A 1.0 Length: 1, dtype: float64
Moreover, DataFrame reductions with numeric_only=None will now be consistent with their Series counterparts. In particular, for reductions where the Series method raises TypeError, the DataFrame reduction will now consider that column non-numeric instead of casting to a NumPy array which may have different semantics (GH36076, GH28949, GH21020).
TypeError
In [31]: ser = pd.Series([0, 1], dtype="category", name="A") In [32]: df = ser.to_frame()
In [5]: df.any() Out[5]: A True dtype: bool
In [33]: df.any() Out[33]: Series([], Length: 0, dtype: bool)
pandas 1.2.0 supports Python 3.7.1 and higher (GH35214).
Some minimum supported versions of dependencies were updated (GH35214). If installed, we now require:
Package
Minimum Version
Required
Changed
numpy
1.16.5
X
pytz
2017.3
python-dateutil
2.7.3
bottleneck
1.2.1
numexpr
2.6.8
pytest (dev)
5.0.1
mypy (dev)
0.782
For optional libraries the general recommendation is to use the latest version. The following table lists the lowest version per library that is currently being tested throughout the development of pandas. Optional libraries below the lowest tested version may still work, but are not considered supported.
beautifulsoup4
4.6.0
fastparquet
0.3.2
fsspec
0.7.4
gcsfs
0.6.0
lxml
4.3.0
matplotlib
2.2.3
numba
0.46.0
2.6.0
pyarrow
0.15.0
pymysql
0.7.11
pytables
3.5.1
s3fs
0.4.0
scipy
1.2.0
sqlalchemy
1.2.8
xarray
0.12.3
xlsxwriter
1.0.2
1.3.0
pandas-gbq
0.12.0
See Dependencies and Optional dependencies for more.
Sorting in descending order is now stable for Series.sort_values() and Index.sort_values() for Datetime-like Index subclasses. This will affect sort order when sorting a DataFrame on multiple columns, sorting with a key function that produces duplicates, or requesting the sorting index when using Index.sort_values(). When using Series.value_counts(), the count of missing values is no longer necessarily last in the list of duplicate counts. Instead, its position corresponds to the position in the original Series. When using Index.sort_values() for Datetime-like Index subclasses, NaTs ignored the na_position argument and were sorted to the beginning. Now they respect na_position, the default being last, same as other Index subclasses (GH35992)
Series.sort_values()
Index.sort_values()
Series.value_counts()
na_position
Passing an invalid fill_value to Categorical.take(), DatetimeArray.take(), TimedeltaArray.take(), or PeriodArray.take() now raises a TypeError instead of a ValueError (GH37733)
fill_value
Categorical.take()
DatetimeArray.take()
TimedeltaArray.take()
PeriodArray.take()
ValueError
Passing an invalid fill_value to Series.shift() with a CategoricalDtype now raises a TypeError instead of a ValueError (GH37733)
Series.shift()
CategoricalDtype
Passing an invalid value to IntervalIndex.insert() or CategoricalIndex.insert() now raises a TypeError instead of a ValueError (GH37733)
IntervalIndex.insert()
CategoricalIndex.insert()
Attempting to reindex a Series with a CategoricalIndex with an invalid fill_value now raises a TypeError instead of a ValueError (GH37733)
CategoricalIndex
CategoricalIndex.append() with an index that contains non-category values will now cast instead of raising TypeError (GH38098)
CategoricalIndex.append()
Deprecated parameter inplace in MultiIndex.set_codes() and MultiIndex.set_levels() (GH35626)
inplace
MultiIndex.set_codes()
MultiIndex.set_levels()
Deprecated parameter dtype of method copy() for all Index subclasses. Use the astype() method instead for changing dtype (GH35853)
dtype
copy()
astype()
Deprecated parameters levels and codes in MultiIndex.copy(). Use the set_levels() and set_codes() methods instead (GH36685)
levels
codes
MultiIndex.copy()
set_levels()
set_codes()
Date parser functions parse_date_time(), parse_date_fields(), parse_all_fields() and generic_parser() from pandas.io.date_converters are deprecated and will be removed in a future version; use to_datetime() instead (GH35741)
parse_date_time()
parse_date_fields()
parse_all_fields()
generic_parser()
pandas.io.date_converters
to_datetime()
DataFrame.lookup() is deprecated and will be removed in a future version, use DataFrame.melt() and DataFrame.loc() instead (GH35224)
DataFrame.lookup()
DataFrame.melt()
DataFrame.loc()
The method Index.to_native_types() is deprecated. Use .astype(str) instead (GH28867)
Index.to_native_types()
.astype(str)
Deprecated indexing DataFrame rows with a single datetime-like string as df[string] (given the ambiguity whether it is indexing the rows or selecting a column), use df.loc[string] instead (GH36179)
df[string]
df.loc[string]
Deprecated Index.is_all_dates() (GH27744)
Index.is_all_dates()
The default value of regex for Series.str.replace() will change from True to False in a future release. In addition, single character regular expressions will not be treated as literal strings when regex=True is set (GH24804)
regex
Series.str.replace()
True
regex=True
Deprecated automatic alignment on comparison operations between DataFrame and Series, do frame, ser = frame.align(ser, axis=1, copy=False) before e.g. frame == ser (GH28759)
frame, ser = frame.align(ser, axis=1, copy=False)
frame == ser
Rolling.count() with min_periods=None will default to the size of the window in a future version (GH31302)
Rolling.count()
min_periods=None
Using “outer” ufuncs on DataFrames to return 4d ndarray is now deprecated. Convert to an ndarray first (GH23743)
Deprecated slice-indexing on tz-aware DatetimeIndex with naive datetime objects, to match scalar indexing behavior (GH36148)
datetime
Index.ravel() returning a np.ndarray is deprecated, in the future this will return a view on the same index (GH19956)
Index.ravel()
np.ndarray
Deprecate use of strings denoting units with ‘M’, ‘Y’ or ‘y’ in to_timedelta() (GH36666)
to_timedelta()
Index methods &, |, and ^ behaving as the set operations Index.intersection(), Index.union(), and Index.symmetric_difference(), respectively, are deprecated and in the future will behave as pointwise boolean operations matching Series behavior. Use the named set methods instead (GH36758)
&
|
^
Index.intersection()
Index.union()
Index.symmetric_difference()
Categorical.is_dtype_equal() and CategoricalIndex.is_dtype_equal() are deprecated, will be removed in a future version (GH37545)
Categorical.is_dtype_equal()
CategoricalIndex.is_dtype_equal()
Series.slice_shift() and DataFrame.slice_shift() are deprecated, use Series.shift() or DataFrame.shift() instead (GH37601)
Series.slice_shift()
DataFrame.slice_shift()
DataFrame.shift()
Partial slicing on unordered DatetimeIndex objects with keys that are not in the index is deprecated and will be removed in a future version (GH18531)
The how keyword in PeriodIndex.astype() is deprecated and will be removed in a future version, use index.to_timestamp(how=how) instead (GH37982)
how
PeriodIndex.astype()
index.to_timestamp(how=how)
Deprecated Index.asi8() for Index subclasses other than DatetimeIndex, TimedeltaIndex, and PeriodIndex (GH37877)
Index.asi8()
TimedeltaIndex
The inplace parameter of Categorical.remove_unused_categories() is deprecated and will be removed in a future version (GH37643)
Categorical.remove_unused_categories()
The null_counts parameter of DataFrame.info() is deprecated and replaced by show_counts. It will be removed in a future version (GH37999)
null_counts
DataFrame.info()
show_counts
Calling NumPy ufuncs on non-aligned DataFrames
Calling NumPy ufuncs on non-aligned DataFrames changed behaviour in pandas 1.2.0 (to align the inputs before calling the ufunc), but this change is reverted in pandas 1.2.1. The behaviour to not align is now deprecated instead, see the the 1.2.1 release notes for more details.
Performance improvements when creating DataFrame or Series with dtype str or StringDtype from array with many string elements (GH36304, GH36317, GH36325, GH36432, GH37371)
str
StringDtype
Performance improvement in GroupBy.agg() with the numba engine (GH35759)
GroupBy.agg()
Performance improvements when creating Series.map() from a huge dictionary (GH34717)
Series.map()
Performance improvement in GroupBy.transform() with the numba engine (GH36240)
GroupBy.transform()
Styler uuid method altered to compress data transmission over web whilst maintaining reasonably low table collision probability (GH36345)
Performance improvement in to_datetime() with non-ns time unit for float dtype columns (GH20445)
float
Performance improvement in setting values on an IntervalArray (GH36310)
IntervalArray
The internal index method _shallow_copy() now makes the new index and original index share cached attributes, avoiding creating these again, if created on either. This can speed up operations that depend on creating copies of existing indexes (GH36840)
_shallow_copy()
Performance improvement in RollingGroupby.count() (GH35625)
RollingGroupby.count()
Small performance decrease to Rolling.min() and Rolling.max() for fixed windows (GH36567)
Rolling.min()
Rolling.max()
Reduced peak memory usage in DataFrame.to_pickle() when using protocol=5 in python 3.8+ (GH34244)
DataFrame.to_pickle()
protocol=5
Faster dir calls when the object has many index labels, e.g. dir(ser) (GH37450)
dir
dir(ser)
Performance improvement in ExpandingGroupby (GH37064)
ExpandingGroupby
Performance improvement in Series.astype() and DataFrame.astype() for Categorical (GH8628)
Series.astype()
DataFrame.astype()
Categorical
Performance improvement in DataFrame.groupby() for float dtype (GH28303), changes of the underlying hash-function can lead to changes in float based indexes sort ordering for ties (e.g. Index.value_counts())
Index.value_counts()
Performance improvement in pd.isin() for inputs with more than 1e6 elements (GH36611)
pd.isin()
Performance improvement for DataFrame.__setitem__() with list-like indexers (GH37954)
DataFrame.__setitem__()
read_json() now avoids reading entire file into memory when chunksize is specified (GH34548)
Categorical.fillna() will always return a copy, validate a passed fill value regardless of whether there are any NAs to fill, and disallow an NaT as a fill value for numeric categories (GH36530)
Categorical.fillna()
NaT
Bug in Categorical.__setitem__() that incorrectly raised when trying to set a tuple value (GH20439)
Categorical.__setitem__()
Bug in CategoricalIndex.equals() incorrectly casting non-category entries to np.nan (GH37667)
CategoricalIndex.equals()
Bug in CategoricalIndex.where() incorrectly setting non-category entries to np.nan instead of raising TypeError (GH37977)
CategoricalIndex.where()
Bug in Categorical.to_numpy() and np.array(categorical) with tz-aware datetime64 categories incorrectly dropping the time zone information instead of casting to object dtype (GH38136)
Categorical.to_numpy()
np.array(categorical)
Bug in DataFrame.combine_first() that would convert datetime-like column on other DataFrame to integer when the column is not present in original DataFrame (GH28481)
DataFrame.combine_first()
Bug in DatetimeArray.date where a ValueError would be raised with a read-only backing array (GH33530)
DatetimeArray.date
Bug in NaT comparisons failing to raise TypeError on invalid inequality comparisons (GH35046)
Bug in DateOffset where attributes reconstructed from pickle files differ from original objects when input values exceed normal ranges (e.g. months=12) (GH34511)
DateOffset
Bug in DatetimeIndex.get_slice_bound() where datetime.date objects were not accepted or naive Timestamp with a tz-aware DatetimeIndex (GH35690)
DatetimeIndex.get_slice_bound()
datetime.date
Bug in DatetimeIndex.slice_locs() where datetime.date objects were not accepted (GH34077)
DatetimeIndex.slice_locs()
Bug in DatetimeIndex.searchsorted(), TimedeltaIndex.searchsorted(), PeriodIndex.searchsorted(), and Series.searchsorted() with datetime64, timedelta64 or Period dtype placement of NaT values being inconsistent with NumPy (GH36176, GH36254)
timedelta64
Inconsistency in DatetimeArray, TimedeltaArray, and PeriodArray method __setitem__ casting arrays of strings to datetime-like scalars but not scalar strings (GH36261)
DatetimeArray
TimedeltaArray
PeriodArray
__setitem__
Bug in DatetimeArray.take() incorrectly allowing fill_value with a mismatched time zone (GH37356)
Bug in DatetimeIndex.shift incorrectly raising when shifting empty indexes (GH14811)
DatetimeIndex.shift
Timestamp and DatetimeIndex comparisons between tz-aware and tz-naive objects now follow the standard library datetime behavior, returning True/False for !=/== and raising for inequality comparisons (GH28507)
!=
==
Bug in DatetimeIndex.equals() and TimedeltaIndex.equals() incorrectly considering int64 indexes as equal (GH36744)
DatetimeIndex.equals()
TimedeltaIndex.equals()
int64
Series.to_json(), DataFrame.to_json(), and read_json() now implement time zone parsing when orient structure is table (GH35973)
Series.to_json()
DataFrame.to_json()
table
astype() now attempts to convert to datetime64[ns, tz] directly from object with inferred time zone from string (GH35973)
datetime64[ns, tz]
object
Bug in TimedeltaIndex.sum() and Series.sum() with timedelta64 dtype on an empty index or series returning NaT instead of Timedelta(0) (GH31751)
TimedeltaIndex.sum()
Series.sum()
Timedelta(0)
Bug in DatetimeArray.shift() incorrectly allowing fill_value with a mismatched time zone (GH37299)
DatetimeArray.shift()
Bug in adding a BusinessDay with nonzero offset to a non-scalar other (GH37457)
BusinessDay
offset
Bug in to_datetime() with a read-only array incorrectly raising (GH34857)
Bug in Series.isin() with datetime64[ns] dtype and DatetimeIndex.isin() incorrectly casting integers to datetimes (GH36621)
Series.isin()
datetime64[ns]
DatetimeIndex.isin()
Bug in Series.isin() with datetime64[ns] dtype and DatetimeIndex.isin() failing to consider tz-aware and tz-naive datetimes as always different (GH35728)
Bug in Series.isin() with PeriodDtype dtype and PeriodIndex.isin() failing to consider arguments with different PeriodDtype as always different (GH37528)
PeriodDtype
PeriodIndex.isin()
Bug in Period constructor now correctly handles nanoseconds in the value argument (GH34621 and GH17053)
value
Bug in TimedeltaIndex, Series, and DataFrame floor-division with timedelta64 dtypes and NaT in the denominator (GH35529)
Bug in parsing of ISO 8601 durations in Timedelta and to_datetime() (GH29773, GH36204)
Timedelta
Bug in to_timedelta() with a read-only array incorrectly raising (GH34857)
Bug in Timedelta incorrectly truncating to sub-second portion of a string input when it has precision higher than nanoseconds (GH36738)
Bug in date_range() was raising AmbiguousTimeError for valid input with ambiguous=False (GH35297)
date_range()
AmbiguousTimeError
ambiguous=False
Bug in Timestamp.replace() was losing fold information (GH37610)
Timestamp.replace()
Bug in to_numeric() where float precision was incorrect (GH31364)
to_numeric()
Bug in DataFrame.any() with axis=1 and bool_only=True ignoring the bool_only keyword (GH32432)
bool_only
Bug in Series.equals() where a ValueError was raised when NumPy arrays were compared to scalars (GH35267)
Series.equals()
Bug in Series where two Series each have a DatetimeIndex with different time zones having those indexes incorrectly changed when performing arithmetic operations (GH33671)
Bug in pandas.testing module functions when used with check_exact=False on complex numeric types (GH28235)
pandas.testing
check_exact=False
Bug in DataFrame.__rmatmul__() error handling reporting transposed shapes (GH21581)
DataFrame.__rmatmul__()
Bug in Series flex arithmetic methods where the result when operating with a list, tuple or np.ndarray would have an incorrect name (GH36760)
list
tuple
Bug in IntegerArray multiplication with timedelta and np.timedelta64 objects (GH36870)
IntegerArray
timedelta
np.timedelta64
Bug in MultiIndex comparison with tuple incorrectly treating tuple as array-like (GH21517)
Bug in DataFrame.diff() with datetime64 dtypes including NaT values failing to fill NaT results correctly (GH32441)
DataFrame.diff()
Bug in DataFrame arithmetic ops incorrectly accepting keyword arguments (GH36843)
Bug in IntervalArray comparisons with Series not returning Series (GH36908)
Bug in DataFrame allowing arithmetic operations with list of array-likes with undefined results. Behavior changed to raising ValueError (GH36702)
Bug in DataFrame.std() with timedelta64 dtype and skipna=False (GH37392)
DataFrame.std()
skipna=False
Bug in DataFrame.min() and DataFrame.max() with datetime64 dtype and skipna=False (GH36907)
DataFrame.min()
DataFrame.max()
Bug in DataFrame.idxmax() and DataFrame.idxmin() with mixed dtypes incorrectly raising TypeError (GH38195)
DataFrame.idxmax()
DataFrame.idxmin()
Bug in DataFrame.to_dict() with orient='records' now returns python native datetime objects for datetime-like columns (GH21256)
DataFrame.to_dict()
orient='records'
Bug in Series.astype() conversion from string to float raised in presence of pd.NA values (GH37626)
string
Bug in Series.to_string(), DataFrame.to_string(), and DataFrame.to_latex() adding a leading space when index=False (GH24980)
Series.to_string()
DataFrame.to_string()
index=False
Bug in to_numeric() raising a TypeError when attempting to convert a string dtype Series containing only numeric strings and NA (GH37262)
NA
Bug in DataFrame.replace() and Series.replace() where Interval dtypes would be converted to object dtypes (GH34871)
DataFrame.replace()
Series.replace()
Interval
Bug in IntervalIndex.take() with negative indices and fill_value=None (GH37330)
IntervalIndex.take()
fill_value=None
Bug in IntervalIndex.putmask() with datetime-like dtype incorrectly casting to object dtype (GH37968)
IntervalIndex.putmask()
Bug in IntervalArray.astype() incorrectly dropping dtype information with a CategoricalDtype object (GH37984)
IntervalArray.astype()
Bug in PeriodIndex.get_loc() incorrectly raising ValueError on non-datelike strings instead of KeyError, causing similar errors in Series.__getitem__(), Series.__contains__(), and Series.loc.__getitem__() (GH34240)
PeriodIndex.get_loc()
KeyError
Series.__getitem__()
Series.__contains__()
Series.loc.__getitem__()
Bug in Index.sort_values() where, when empty values were passed, the method would break by trying to compare missing values instead of pushing them to the end of the sort order (GH35584)
Bug in Index.get_indexer() and Index.get_indexer_non_unique() where int64 arrays are returned instead of intp (GH36359)
Index.get_indexer()
Index.get_indexer_non_unique()
intp
Bug in DataFrame.sort_index() where parameter ascending passed as a list on a single level index gives wrong result (GH32334)
DataFrame.sort_index()
Bug in DataFrame.reset_index() was incorrectly raising a ValueError for input with a MultiIndex with missing values in a level with Categorical dtype (GH24206)
DataFrame.reset_index()
Bug in indexing with boolean masks on datetime-like values sometimes returning a view instead of a copy (GH36210)
Bug in DataFrame.__getitem__() and DataFrame.loc.__getitem__() with IntervalIndex columns and a numeric indexer (GH26490)
DataFrame.__getitem__()
DataFrame.loc.__getitem__()
IntervalIndex
Bug in Series.loc.__getitem__() with a non-unique MultiIndex and an empty-list indexer (GH13691)
Bug in indexing on a Series or DataFrame with a MultiIndex and a level named "0" (GH37194)
"0"
Bug in Series.__getitem__() when using an unsigned integer array as an indexer giving incorrect results or segfaulting instead of raising KeyError (GH37218)
Bug in Index.where() incorrectly casting numeric values to strings (GH37591)
Index.where()
Bug in DataFrame.loc() returning empty result when indexer is a slice with negative step size (GH38071)
Bug in Series.loc() and DataFrame.loc() raises when the index was of object dtype and the given numeric label was in the index (GH26491)
Series.loc()
Bug in DataFrame.loc() returned requested key plus missing values when loc was applied to single level from a MultiIndex (GH27104)
loc
Bug in indexing on a Series or DataFrame with a CategoricalIndex using a list-like indexer containing NA values (GH37722)
Bug in DataFrame.loc.__setitem__() expanding an empty DataFrame with mixed dtypes (GH37932)
DataFrame.loc.__setitem__()
Bug in DataFrame.xs() ignored droplevel=False for columns (GH19056)
DataFrame.xs()
droplevel=False
Bug in DataFrame.reindex() raising IndexingError wrongly for empty DataFrame with tolerance not None or method="nearest" (GH27315)
DataFrame.reindex()
IndexingError
tolerance
None
method="nearest"
Bug in indexing on a Series or DataFrame with a CategoricalIndex using list-like indexer that contains elements that are in the index’s categories but not in the index itself failing to raise KeyError (GH37901)
categories
Bug on inserting a boolean label into a DataFrame with a numeric Index columns incorrectly casting to integer (GH36319)
Bug in DataFrame.iloc() and Series.iloc() aligning objects in __setitem__ (GH22046)
DataFrame.iloc()
Series.iloc()
Bug in MultiIndex.drop() does not raise if labels are partially found (GH37820)
MultiIndex.drop()
Bug in DataFrame.loc() did not raise KeyError when missing combination was given with slice(None) for remaining levels (GH19556)
slice(None)
Bug in DataFrame.loc() raising TypeError when non-integer slice was given to select values from MultiIndex (GH25165, GH24263)
Bug in Series.at() returning Series with one element instead of scalar when index is a MultiIndex with one level (GH38053)
Series.at()
Bug in DataFrame.loc() returning and assigning elements in wrong order when indexer is differently ordered than the MultiIndex to filter (GH31330, GH34603)
Bug in DataFrame.loc() and DataFrame.__getitem__() raising KeyError when columns were MultiIndex with only one level (GH29749)
Bug in Series.__getitem__() and DataFrame.__getitem__() raising blank KeyError without missing keys for IntervalIndex (GH27365)
Bug in setting a new label on a DataFrame or Series with a CategoricalIndex incorrectly raising TypeError when the new label is not among the index’s categories (GH38098)
Bug in Series.loc() and Series.iloc() raising ValueError when inserting a list-like np.array, list or tuple in an object Series of equal length (GH37748, GH37486)
np.array
Bug in Series.loc() and Series.iloc() setting all the values of an object Series with those of a list-like ExtensionArray instead of inserting it (GH38271)
ExtensionArray
Bug in SeriesGroupBy.transform() now correctly handles missing values for dropna=False (GH35014)
SeriesGroupBy.transform()
dropna=False
Bug in Series.nunique() with dropna=True was returning incorrect results when both NA and None missing values were present (GH37566)
Series.nunique()
dropna=True
Bug in Series.interpolate() where kwarg limit_area and limit_direction had no effect when using methods pad and backfill (GH31048)
Series.interpolate()
limit_area
limit_direction
pad
backfill
Bug in DataFrame.xs() when used with IndexSlice raises TypeError with message "Expected label or tuple of labels" (GH35301)
IndexSlice
"Expected label or tuple of labels"
Bug in DataFrame.reset_index() with NaT values in index raises ValueError with message "cannot convert float NaN to integer" (GH36541)
"cannot convert float NaN to integer"
Bug in DataFrame.combine_first() when used with MultiIndex containing string and NaN values raises TypeError (GH36562)
NaN
Bug in MultiIndex.drop() dropped NaN values when non existing key was given as input (GH18853)
Bug in MultiIndex.drop() dropping more values than expected when index has duplicates and is not sorted (GH33494)
read_sas() no longer leaks resources on failure (GH35566)
Bug in DataFrame.to_csv() and Series.to_csv() caused a ValueError when it was called with a filename in combination with mode containing a b (GH35058)
DataFrame.to_csv()
Series.to_csv()
mode
b
Bug in read_csv() with float_precision='round_trip' did not handle decimal and thousands parameters (GH35365)
float_precision='round_trip'
decimal
thousands
to_pickle() and read_pickle() were closing user-provided file objects (GH35679)
to_pickle()
read_pickle()
to_csv() passes compression arguments for 'gzip' always to gzip.GzipFile (GH28103)
'gzip'
gzip.GzipFile
to_csv() did not support zip compression for binary file object not having a filename (GH35058)
to_csv() and read_csv() did not honor compression and encoding for path-like objects that are internally converted to file-like objects (GH35677, GH26124, GH32392)
DataFrame.to_pickle(), Series.to_pickle(), and read_pickle() did not support compression for file-objects (GH26237, GH29054, GH29570)
Series.to_pickle()
Bug in LongTableBuilder.middle_separator() was duplicating LaTeX longtable entries in the List of Tables of a LaTeX document (GH34360)
LongTableBuilder.middle_separator()
Bug in read_csv() with engine='python' truncating data if multiple items present in first row and first element started with BOM (GH36343)
engine='python'
Removed private_key and verbose from read_gbq() as they are no longer supported in pandas-gbq (GH34654, GH30200)
private_key
verbose
read_gbq()
Bumped minimum pytables version to 3.5.1 to avoid a ValueError in read_hdf() (GH24839)
read_hdf()
Bug in read_table() and read_csv() when delim_whitespace=True and sep=default (GH36583)
delim_whitespace=True
sep=default
Bug in DataFrame.to_json() and Series.to_json() when used with lines=True and orient='records' the last line of the record is not appended with ‘new line character’ (GH36888)
lines=True
Bug in read_parquet() with fixed offset time zones. String representation of time zones was not recognized (GH35997, GH36004)
Bug in DataFrame.to_html(), DataFrame.to_string(), and DataFrame.to_latex() ignoring the na_rep argument when float_format was also specified (GH9046, GH13828)
DataFrame.to_html()
na_rep
float_format
Bug in output rendering of complex numbers showing too many trailing zeros (GH36799)
Bug in HDFStore threw a TypeError when exporting an empty DataFrame with datetime64[ns, tz] dtypes with a fixed HDF5 store (GH20594)
HDFStore
Bug in HDFStore was dropping time zone information when exporting a Series with datetime64[ns, tz] dtypes with a fixed HDF5 store (GH20594)
read_csv() was closing user-provided binary file handles when engine="c" and an encoding was requested (GH36980)
engine="c"
Bug in DataFrame.to_hdf() was not dropping missing rows with dropna=True (GH35719)
DataFrame.to_hdf()
Bug in read_html() was raising a TypeError when supplying a pathlib.Path argument to the io parameter (GH37705)
read_html()
pathlib.Path
io
DataFrame.to_excel(), Series.to_excel(), DataFrame.to_markdown(), and Series.to_markdown() now support writing to fsspec URLs such as S3 and Google Cloud Storage (GH33987)
DataFrame.to_excel()
Series.to_excel()
DataFrame.to_markdown()
Series.to_markdown()
Bug in read_fwf() with skip_blank_lines=True was not skipping blank lines (GH37758)
read_fwf()
skip_blank_lines=True
Parse missing values using read_json() with dtype=False to NaN instead of None (GH28501)
dtype=False
read_fwf() was inferring compression with compression=None which was not consistent with the other read_* functions (GH37909)
compression=None
read_*
DataFrame.to_html() was ignoring formatters argument for ExtensionDtype columns (GH36525)
formatters
ExtensionDtype
Bumped minimum xarray version to 0.12.3 to avoid reference to the removed Panel class (GH27101, GH37983)
Panel
DataFrame.to_csv() was re-opening file-like handles that also implement os.PathLike (GH38125)
os.PathLike
Bug in the conversion of a sliced pyarrow.Table with missing values to a DataFrame (GH38525)
pyarrow.Table
Bug in read_sql_table() raising a sqlalchemy.exc.OperationalError when column names contained a percentage sign (GH37517)
read_sql_table()
sqlalchemy.exc.OperationalError
Bug in DataFrame.replace() and Series.replace() where Period dtypes would be converted to object dtypes (GH34871)
Bug in DataFrame.plot() was rotating xticklabels when subplots=True, even if the x-axis wasn’t an irregular time series (GH29460)
subplots=True
Bug in DataFrame.plot() where a marker letter in the style keyword sometimes caused a ValueError (GH21003)
style
Bug in DataFrame.plot.bar() and Series.plot.bar() where ticks positions were assigned by value order instead of using the actual value for numeric or a smart ordering for string (GH26186, GH11465). This fix has been reverted in pandas 1.2.1, see What’s new in 1.2.1 (January 20, 2021)
DataFrame.plot.bar()
Series.plot.bar()
Twinned axes were losing their tick labels which should only happen to all but the last row or column of ‘externally’ shared axes (GH33819)
Bug in Series.plot() and DataFrame.plot() was throwing a ValueError when the Series or DataFrame was indexed by a TimedeltaIndex with a fixed frequency and the x-axis lower limit was greater than the upper limit (GH37454)
Series.plot()
Bug in DataFrameGroupBy.boxplot() when subplots=False would raise a KeyError (GH16748)
DataFrameGroupBy.boxplot()
subplots=False
Bug in DataFrame.plot() and Series.plot() was overwriting matplotlib’s shared y axes behavior when no sharey parameter was passed (GH37942)
sharey
Bug in DataFrame.plot() was raising a TypeError with ExtensionDtype columns (GH32073)
Bug in Styler.render() HTML was generated incorrectly because of formatting error in rowspan attribute, it now matches with w3 syntax (GH38234)
Styler.render()
rowspan
Bug in DataFrameGroupBy.count() and SeriesGroupBy.sum() returning NaN for missing categories when grouped on multiple Categoricals. Now returning 0 (GH35028)
DataFrameGroupBy.count()
SeriesGroupBy.sum()
Categoricals
0
Bug in DataFrameGroupBy.apply() that would sometimes throw an erroneous ValueError if the grouping axis had duplicate entries (GH16646)
DataFrameGroupBy.apply()
Bug in DataFrame.resample() that would throw a ValueError when resampling from "D" to "24H" over a transition into daylight savings time (DST) (GH35219)
"D"
"24H"
Bug when combining methods DataFrame.groupby() with DataFrame.resample() and DataFrame.interpolate() raising a TypeError (GH35325)
DataFrame.interpolate()
Bug in DataFrameGroupBy.apply() where a non-nuisance grouping column would be dropped from the output columns if another groupby method was called before .apply (GH34656)
.apply
Bug when subsetting columns on a DataFrameGroupBy (e.g. df.groupby('a')[['b']])) would reset the attributes axis, dropna, group_keys, level, mutated, sort, and squeeze to their default values (GH9959)
df.groupby('a')[['b']])
axis
dropna
group_keys
level
mutated
sort
squeeze
Bug in DataFrameGroupBy.tshift() failing to raise ValueError when a frequency cannot be inferred for the index of a group (GH35937)
DataFrameGroupBy.tshift()
Bug in DataFrame.groupby() does not always maintain column index name for any, all, bfill, ffill, shift (GH29764)
any
all
bfill
ffill
shift
Bug in DataFrameGroupBy.apply() raising error with np.nan group(s) when dropna=False (GH35889)
Bug in Rolling.sum() returned wrong values when dtypes where mixed between float and integer and axis=1 (GH20649, GH35596)
Bug in Rolling.count() returned np.nan with FixedForwardWindowIndexer as window, min_periods=0 and only missing values in the window (GH35579)
FixedForwardWindowIndexer
min_periods=0
Bug where pandas.core.window.Rolling produces incorrect window sizes when using a PeriodIndex (GH34225)
pandas.core.window.Rolling
Bug in DataFrameGroupBy.ffill() and DataFrameGroupBy.bfill() where a NaN group would return filled values instead of NaN when dropna=True (GH34725)
DataFrameGroupBy.ffill()
DataFrameGroupBy.bfill()
Bug in RollingGroupby.count() where a ValueError was raised when specifying the closed parameter (GH35869)
Bug in DataFrameGroupBy.rolling() returning wrong values with partial centered window (GH36040)
DataFrameGroupBy.rolling()
Bug in DataFrameGroupBy.rolling() returned wrong values with time aware window containing NaN. Raises ValueError because windows are not monotonic now (GH34617)
Bug in Rolling.__iter__() where a ValueError was not raised when min_periods was larger than window (GH37156)
Rolling.__iter__()
min_periods
window
Using Rolling.var() instead of Rolling.std() avoids numerical issues for Rolling.corr() when Rolling.var() is still within floating point precision while Rolling.std() is not (GH31286)
Rolling.corr()
Bug in DataFrameGroupBy.quantile() and Resampler.quantile() raised TypeError when values were of type Timedelta (GH29485)
DataFrameGroupBy.quantile()
Resampler.quantile()
Bug in Rolling.median() and Rolling.quantile() returned wrong values for BaseIndexer subclasses with non-monotonic starting or ending points for windows (GH37153)
Rolling.median()
Rolling.quantile()
BaseIndexer
Bug in DataFrame.groupby() dropped nan groups from result with dropna=False when grouping over a single column (GH35646, GH35542)
nan
Bug in DataFrameGroupBy.head(), DataFrameGroupBy.tail(), SeriesGroupBy.head(), and SeriesGroupBy.tail() would raise when used with axis=1 (GH9772)
DataFrameGroupBy.head()
DataFrameGroupBy.tail()
SeriesGroupBy.head()
SeriesGroupBy.tail()
Bug in DataFrameGroupBy.transform() would raise when used with axis=1 and a transformation kernel (e.g. “shift”) (GH36308)
DataFrameGroupBy.transform()
Bug in DataFrameGroupBy.resample() using .agg with sum produced different result than just calling .sum (GH33548)
DataFrameGroupBy.resample()
.agg
.sum
Bug in DataFrameGroupBy.apply() dropped values on nan group when returning the same axes with the original frame (GH38227)
Bug in DataFrameGroupBy.quantile() couldn’t handle with arraylike q when grouping by columns (GH33795)
q
Bug in DataFrameGroupBy.rank() with datetime64tz or period dtype incorrectly casting results to those dtypes instead of returning float64 dtype (GH38187)
DataFrameGroupBy.rank()
float64
Bug in DataFrame.crosstab() was returning incorrect results on inputs with duplicate row names, duplicate column names or duplicate names between row and column labels (GH22529)
DataFrame.crosstab()
Bug in DataFrame.pivot_table() with aggfunc='count' or aggfunc='sum' returning NaN for missing categories when pivoted on a Categorical. Now returning 0 (GH31422)
DataFrame.pivot_table()
aggfunc='count'
aggfunc='sum'
Bug in concat() and DataFrame constructor where input index names are not preserved in some cases (GH13475)
Bug in func crosstab() when using multiple columns with margins=True and normalize=True (GH35144)
crosstab()
margins=True
normalize=True
Bug in DataFrame.stack() where an empty DataFrame.stack would raise an error (GH36113). Now returning an empty Series with empty MultiIndex.
DataFrame.stack()
Bug in Series.unstack(). Now a Series with single level of Index trying to unstack would raise a ValueError (GH36113)
Series.unstack()
Bug in DataFrame.agg() with func={'name':<FUNC>} incorrectly raising TypeError when DataFrame.columns==['Name'] (GH36212)
DataFrame.agg()
func={'name':<FUNC>}
DataFrame.columns==['Name']
Bug in Series.transform() would give incorrect results or raise when the argument func was a dictionary (GH35811)
Series.transform()
func
Bug in DataFrame.pivot() did not preserve MultiIndex level names for columns when rows and columns are both multiindexed (GH36360)
DataFrame.pivot()
Bug in DataFrame.pivot() modified index argument when columns was passed but values was not (GH37635)
index
columns
values
Bug in DataFrame.join() returned a non deterministic level-order for the resulting MultiIndex (GH36910)
Bug in DataFrame.combine_first() caused wrong alignment with dtype string and one level of MultiIndex containing only NA (GH37591)
Fixed regression in merge() on merging DatetimeIndex with empty DataFrame (GH36895)
merge()
Bug in DataFrame.apply() not setting index of return value when func return type is dict (GH37544)
DataFrame.apply()
dict
Bug in DataFrame.merge() and pandas.merge() returning inconsistent ordering in result for how=right and how=left (GH35382)
pandas.merge()
how=right
how=left
Bug in merge_ordered() couldn’t handle list-like left_by or right_by (GH35269)
merge_ordered()
left_by
right_by
Bug in merge_ordered() returned wrong join result when length of left_by or right_by equals to the rows of left or right (GH38166)
left
right
Bug in merge_ordered() didn’t raise when elements in left_by or right_by not exist in left columns or right columns (GH38167)
Bug in DataFrame.drop_duplicates() not validating bool dtype for ignore_index keyword (GH38274)
DataFrame.drop_duplicates()
ignore_index
Fixed bug where DataFrame column set to scalar extension type via a dict instantiation was considered an object type rather than the extension type (GH35965)
Fixed bug where astype() with equal dtype and copy=False would return a new object (GH28488)
copy=False
Fixed bug when applying a NumPy ufunc with multiple outputs to an IntegerArray returning None (GH36913)
Fixed an inconsistency in PeriodArray’s __init__ signature to those of DatetimeArray and TimedeltaArray (GH37289)
__init__
Reductions for BooleanArray, Categorical, DatetimeArray, FloatingArray, IntegerArray, PeriodArray, TimedeltaArray, and PandasArray are now keyword-only methods (GH37541)
BooleanArray
PandasArray
Fixed a bug where a TypeError was wrongly raised if a membership check was made on an ExtensionArray containing nan-like values (GH37867)
Bug in DataFrame.replace() and Series.replace() incorrectly raising an AssertionError instead of a ValueError when invalid parameter combinations are passed (GH36045)
AssertionError
Bug in DataFrame.replace() and Series.replace() with numeric values and string to_replace (GH34789)
to_replace
Fixed metadata propagation in Series.abs() and ufuncs called on Series and DataFrames (GH28283)
Series.abs()
Bug in DataFrame.replace() and Series.replace() incorrectly casting from PeriodDtype to object dtype (GH34871)
Fixed bug in metadata propagation incorrectly copying DataFrame columns as metadata when the column name overlaps with the metadata name (GH37037)
Fixed metadata propagation in the Series.dt, Series.str accessors, DataFrame.duplicated, DataFrame.stack, DataFrame.unstack, DataFrame.pivot, DataFrame.append, DataFrame.diff, DataFrame.applymap and DataFrame.update methods (GH28283, GH37381)
Series.dt
Series.str
DataFrame.duplicated
DataFrame.stack
DataFrame.unstack
DataFrame.pivot
DataFrame.append
DataFrame.diff
DataFrame.applymap
DataFrame.update
Fixed metadata propagation when selecting columns with DataFrame.__getitem__ (GH28283)
DataFrame.__getitem__
Bug in Index.intersection() with non-Index failing to set the correct name on the returned Index (GH38111)
Bug in RangeIndex.intersection() failing to set the correct name on the returned Index in some corner cases (GH38197)
RangeIndex.intersection()
Bug in Index.difference() failing to set the correct name on the returned Index in some corner cases (GH38268)
Index.difference()
Bug in Index.union() behaving differently depending on whether operand is an Index or other list-like (GH36384)
Bug in Index.intersection() with non-matching numeric dtypes casting to object dtype instead of minimal common dtype (GH38122)
Bug in IntervalIndex.union() returning an incorrectly-typed Index when empty (GH38282)
IntervalIndex.union()
Passing an array with 2 or more dimensions to the Series constructor now raises the more specific ValueError rather than a bare Exception (GH35744)
Exception
Bug in dir where dir(obj) wouldn’t show attributes defined on the instance for pandas objects (GH37173)
dir(obj)
Bug in Index.drop() raising InvalidIndexError when index has duplicates (GH38051)
Index.drop()
InvalidIndexError
Bug in RangeIndex.difference() returning Int64Index in some cases where it should return RangeIndex (GH38028)
Fixed bug in assert_series_equal() when comparing a datetime-like array with an equivalent non extension dtype array (GH37609)
assert_series_equal()
Bug in is_bool_dtype() would raise when passed a valid string such as "boolean" (GH38386)
is_bool_dtype()
"boolean"
Fixed regression in logical operators raising ValueError when columns of DataFrame are a CategoricalIndex with unused categories (GH38367)
A total of 257 people contributed patches to this release. People with a “+” by their names contributed a patch for the first time.
21CSM +
AbdulMAbdi +
Abhiraj Hinge +
Abhishek Mangla +
Abo7atm +
Adam Spannbauer +
Albert Villanova del Moral
Alex Kirko
Alex Lim +
Alex Thorne +
Aleš Erjavec +
Ali McMaster
Amanda Dsouza +
Amim Knabben +
Andrew Wieteska
Anshoo Rajput +
Anthony Milbourne
Arun12121 +
Asish Mahapatra
Avinash Pancham +
BeanNan +
Ben Forbes +
Brendan Wilby +
Bruno Almeida +
Byron Boulton +
Chankey Pathak
Chris Barnes +
Chris Lynch +
Chris Withers
Christoph Deil +
Christopher Hadley +
Chuanzhu Xu
Coelhudo +
Dan Moore
Daniel Saxton
David Kwong +
David Li +
David Mrva +
Deepak Pandey +
Deepyaman Datta
Devin Petersohn
Dmitriy Perepelkin +
Douglas Hanley +
Dāgs Grīnbergs +
Eli Treuherz +
Elliot Rampono +
Erfan Nariman
Eric Goddard
Eric Leung +
Eric Wieser
Ethan Chen +
Eve +
Eyal Trabelsi +
Fabian Gebhart +
Fangchen Li
Felix Claessen +
Finlay Maguire +
Florian Roscheck +
Gabriel Monteiro
Gautham +
Gerard Jorgensen +
Gregory Livschitz
Hans
Harsh Sharma
Honfung Wong +
Igor Gotlibovych +
Iqrar Agalosi Nureyza
Irv Lustig
Isaac Virshup
Jacob Peacock
Jacob Stevens-Haas +
Jan Müller +
Janus
Jeet Parekh
Jeff Hernandez +
Jeff Reback
Jiaxiang
Joao Pedro Berno Zanutto +
Joel Nothman
Joel Whittier +
John Karasinski +
John McGuigan +
Johnny Pribyl +
Jonas Laursen +
Jonathan Shreckengost +
Joris Van den Bossche
Jose +
JoseNavy +
Josh Temple +
Jun Kudo +
Justin Essert
Justin Sexton +
Kaiqi Dong
Kamil Trocewicz +
Karthik Mathur
Kashif +
Kenny Huynh
Kevin Sheppard
Kumar Shivam +
Leonardus Chen +
Levi Matus +
Lucas Rodés-Guirao +
Luis Pinto +
Lynch +
Marc Garcia
Marco Gorelli
Maria-Alexandra Ilie +
Marian Denes
Mark Graham +
Martin Durant
Matt Roeschke
Matthew Roeschke
Matthias Bussonnier
Maxim Ivanov +
Mayank Chaudhary +
MeeseeksMachine
Meghana Varanasi +
Metehan Kutlu +
Micael Jarniac +
Micah Smith +
Michael Marino
Miroslav Šedivý
Mohammad Jafar Mashhadi
Mohammed Kashif +
Nagesh Kumar C +
Nidhi Zare +
Nikhil Choudhary +
Number42
Oleh Kozynets +
OlivierLuG
Pandas Development Team
Paolo Lammens +
Paul Ganssle
Pax +
Peter Liu +
Philip Cerles +
Pranjal Bhardwaj +
Prayag Savsani +
Purushothaman Srikanth +
Qbiwan +
Rahul Chauhan +
Rahul Sathanapalli +
Rajat Bishnoi +
Ray Bell
Reshama Shaikh +
Richard Shadrach
Robert Bradshaw
Robert de Vries
Rohith295
S Mono +
S.TAKENO +
Sahid Velji +
Sam Cohen +
Sam Ezebunandu +
Sander +
Sarthak +
Sarthak Vineet Kumar +
Satrio H Wicaksono +
Scott Lasley
Shao Yang Hong +
Sharon Woo +
Shubham Mehra +
Simon Hawkins
Sixuan (Cherie) Wu +
Souris Ash +
Steffen Rehberg
Suvayu Ali
Sven
SylvainLan +
T. JEGHAM +
Terji Petersen
Thomas Dickson +
Thomas Heavey +
Thomas Smith
Tobias Pitters
Tom Augspurger
Tomasz Sakrejda +
Torsten Wörtwein +
Ty Mick +
UrielMaD +
Uwe L. Korn
Vikramaditya Gaonkar +
VirosaLi +
W.R +
Warren White +
Wesley Boelrijk +
William Ayd
Yanxian Lin +
Yassir Karroum +
Yong Kai Yi +
Yuanhao Geng +
Yury Mikhaylov +
Yutaro Ikeda
Yuya Takashina +
Zach Brookler +
Zak Kohler +
ZhihuiChen0903 +
abmyii
alexhtn +
asharma13524 +
attack68
beanan +
chinhwee
cleconte987
danchev +
ebardie +
edwardkong
elliot rampono +
estasney +
gabicca
geetha-rangaswamaiah +
gfyoung
guru kiran
hardikpnsp +
icanhazcodeplz +
ivanovmg +
jbrockmendel
jeschwar
jnecus
joooeey +
junk +
krajatcl +
lacrosse91 +
leo +
lpkirwin +
lrjball
lucasrodes +
ma3da +
mavismonica +
mlondschien +
mzeitlin11 +
nguevara +
nrebena
parkdj1 +
partev
patrick
realead
rxxg +
samilAyoub +
sanderland
shawnbrown
sm1899 +
smartvinnetou
ssortman +
steveya +
taytzehao +
tiagohonorato +
timhunderwood
tkmz-n +
tnwei +
tpanza +
vineethraj510 +
vmdhhh +
xinrong-databricks +
yonas kassa +
yonashub +
Ádám Lippai +