Skip to content

fix 'null' in lower case as a missing value in the new Yahoo API, make compatible with the older pandas versions #357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 6, 2017
1 change: 1 addition & 0 deletions docs/source/whatsnew/v0.5.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ Bug Fixes
- Test suite fixes for test_get_options_data (:issue:`352`)
- Test suite fixes for test_wdi_download (:issue:`350`)
- avoid monkey patching requests.Session (:issue:`301`)
- :func:`get_data_yahoo` now treats ``'null'`` strings as missing values (:issue:`342`)
3 changes: 2 additions & 1 deletion pandas_datareader/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,8 @@ def _get_crumb(self, *args):
raise NotImplementedError("Subclass has not implemented method.")

def _read_lines(self, out):
rs = read_csv(out, index_col=0, parse_dates=True, na_values='-')[::-1]
rs = read_csv(out, index_col=0, parse_dates=True,
na_values=('-', 'null'))[::-1]
# Yahoo! Finance sometimes does this awesome thing where they
# return 2 rows for the most recent business day
if len(rs) > 2 and rs.index[-1] == rs.index[-2]: # pragma: no cover
Expand Down
13 changes: 13 additions & 0 deletions pandas_datareader/tests/yahoo/test_yahoo.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,19 @@ def test_get_data_multiple_symbols(self):
sl = ['AAPL', 'AMZN', 'GOOG']
web.get_data_yahoo(sl, '2012')

@pytest.mark.parametrize('adj_pr', [True, False])
def test_get_data_null_as_missing_data(self, adj_pr):
# TODO: We can't decorate the test function along with
# parametrization because it errors. Investigate this later.
@skip_on_exception(RemoteDataError)
def null_as_missing_data(adj_price):
result = web.get_data_yahoo('SRCE', '20160626', '20160705',
adjust_price=adj_pr)
# sanity checking
assert result.dtypes.all() == np.floating

null_as_missing_data(adj_pr)

@skip_on_exception(RemoteDataError)
def test_get_data_multiple_symbols_two_dates(self):
pan = web.get_data_yahoo(['GE', 'MSFT', 'INTC'], 'JAN-01-12',
Expand Down