CLN: clean up data.py #4002
CLN: clean up data.py #4002
Conversation
ghost
assigned 
| @@ -107,12 +107,13 @@ def get_quote_yahoo(symbols): | |||
| request = str.join('', codes.values()) # code request string | |||
| header = codes.keys() | |||
|
|
|||
| data = dict(zip(codes.keys(), [[] for i in range(len(codes))])) | |||
There was a problem hiding this comment.
would it make more sense to change this to a defaultdict?
There was a problem hiding this comment.
yep will do, does py26 have defaultdict?
There was a problem hiding this comment.
@cpcloud came in with 2.5, so yes :)
There was a problem hiding this comment.
sweet i like defaultdict.
|
everything still work? (not sure how many tests we actually have on io.data) can u squash the num of commits down a bit? |
|
@jreback yes everything still works, but not quite finished yet...will squash when i check off the list items |
|
the fact that everything still works after all these changes makes me think that there is potentially a lot of untested code in |
|
@cpcloud I agree with you on that. Do you know why the main entry point is named like a class ( |
|
Would it make sense to set a default timeout on these requests? (you can pass a |
|
possibly...somewhat related is that there's a |
|
|
||
| lines = urllib2.urlopen(urlStr).readlines() | ||
| with closing(urlopen(url_str)) as url: |
There was a problem hiding this comment.
@cpcloud you could use timeout here...just so you can explicitly set when you want to give up on a server responding...
There was a problem hiding this comment.
basically every line like this.
There was a problem hiding this comment.
i'm not sure that timing out is an issue in these cases...but i suppose that a timeout parameter might be useful. i'm not sure abou it though
There was a problem hiding this comment.
Well, weren't you saying that the test_google method was hanging? Was it
because of iterations? Might help it...but yeah prob not a big deal.
On Mon, Jun 24, 2013 at 12:20 AM, Phillip Cloud [email protected]:
In pandas/io/data.py:
- lines = urllib2.urlopen(urlStr).readlines()
- with closing(urlopen(url_str)) as url:
i'm not sure that timing out is an issue in these cases...but i suppose
that a timeout parameter might be useful. i'm not sure abou it though—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4002/files#r4835821
.
There was a problem hiding this comment.
i'm not sure what the reason is. i'll check and let u know
|
@jreback on this clean up i have those tests that fail skip if they raise an |
|
I would have an exception fail them. I guess you can't really 'validate' these consistently. I think these reutrn a list, so a len(0) list is valid then; only complete the rest of the test if you have len > 0. Maybe I would print a warning (in the test) if this is the case, so that when running in the command line, we know this has 'failed'. AssertionWarning? a little bit cheesy...but what can you do |
|
hold on maybe there's something different between using expiry vs passing month/year explicitly. only the warning tests seem to be failing |
|
nope that looks ok |
|
after this passes i'm going to do one more rehash and push then im gonna merge |
|
going to also run tox a few times in a row to see if i can get the warnings to show up |
|
@jreback any objections here? |
|
kind of a big overhaul, but badly needed |
|
looks good |
|
round 42 on this...here we go. |
|
I picked a hell of a time to decide to make a minor change to data.py! Things are moving quickly 👍 I'll have to check back on #4044 once the dust has settled |
|
sorry man there's just a bunch of stuff that needed to essentially be rewritten and network stuff cleared up (still some issues there) |
closes #4001
closes #3982
closes #4028