DOC: Implement javascript to redirect old whatsnew pages to new ones #23708

ismailuddin · 2018-11-14T21:50:43Z

closes DOC: Redirect old whatsnew links #23695
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Javascript file to reformat / redirect 'whatsnew' href links in documentation.

gfyoung · 2018-11-15T06:51:16Z

doc/source/_static/js/whatsnew_links.js

+	function reformatVersion(version) {
+		return "v" + version.slice(0,1) + "." + version.slice(1,-1) + "." + version.slice(-1) + ".html#"
+	}
+})();


Newline at end of file.

Also, are these whatsnew functions parameterized on the version of the whatsnew that we are parsing?

So the function assumes the format according to all the other links (#23695) and assumes the version number is vX.X(X).X; so 1 digit, 1 or 2 digits and then 1 digit. Do you think it should be more general?

Ah, sorry, so this wasn't 100% clear. When I said "parameterized on version," I was referring to this comment in the original issue:

We should be able to detect this in Javascript. When we're on the whatsnew for 0.24.0 or greater, we want to look for anchors pointing to /whatsnew.html#whatsnew- for 0.23.4 or older. Then we'd rewrite /whatsnew.html#whatsnew--<title> to /whatsnew/.html#<title>.

So this is referring to your script in general. I didn't see any checks for version?

Ahh OK I missed this part, I shall amend this!

I have a fix to check for the version number, but since I use a regex to check if the link format is '#whatsnew-XXX-...' before attempting to change the URL, this should prevent this issue, since all links with v0.23.5 or greater will be of the new format of 'v.0.28.9-enhancements...' and so would fail the regex match anyways?

If that's not the case, I'll push this latest fix to check for version number and only amend links that are v0.23.4 or older.

doc/source/_static/js/whatsnew_links.js

gfyoung · 2018-11-15T06:58:10Z

@ismailuddin : Can you rebase your PR on master ? You have a lot of old Circle CI builds in the test modal that we do not use anymore.

pep8speaks · 2018-11-15T09:36:02Z

Hello @ismailuddin! Thanks for updating the PR.

There are no PEP8 issues in the file doc/source/conf.py !

codecov · 2018-11-15T09:36:09Z

Codecov Report

Merging #23708 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #23708      +/-   ##
==========================================
+ Coverage   92.24%   92.25%   +<.01%     
==========================================
  Files         161      161              
  Lines       51336    51383      +47     
==========================================
+ Hits        47357    47404      +47     
  Misses       3979     3979

Flag	Coverage Δ
#multiple	`90.64% <ø> (ø)`	⬆️
#single	`42.31% <ø> (-0.03%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/formats/format.py	`97.87% <0%> (-0.01%)`	⬇️
pandas/core/strings.py	`98.58% <0%> (ø)`	⬆️
pandas/plotting/_core.py	`83.63% <0%> (ø)`	⬆️
pandas/core/indexing.py	`93.87% <0%> (ø)`	⬆️
pandas/core/generic.py	`96.82% <0%> (ø)`	⬆️
pandas/core/arrays/categorical.py	`95.34% <0%> (ø)`	⬆️
pandas/core/window.py	`96.4% <0%> (ø)`	⬆️
pandas/io/parsers.py	`95.55% <0%> (ø)`	⬆️
pandas/core/arrays/datetimes.py	`98.47% <0%> (ø)`	⬆️
pandas/core/frame.py	`97.02% <0%> (ø)`	⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c55057f...7b7315d. Read the comment docs.

doc/source/_static/js/whatsnew_links.js

doc/source/conf.py

Fix function name typo Co-Authored-By: ismailuddin <[email protected]>

ismailuddin · 2018-11-16T15:33:17Z

Just a bump regarding @gfyoung 's comment about checking version; do I need to check the version number since all new links will be of the new format and hence not match the regex pattern? Or will they not be? See above for discussion..

TomAugspurger · 2018-11-16T15:39:44Z

IIUC, you don't need to worry about @gfyoung's concern. We aren't rebuilding the old docs with this JS applied.

TomAugspurger · 2018-11-16T15:41:13Z

Just tried this out locally, didn't quite work. For a URL like

<root>/whatsnew.html#whatsnew-0190-enhancements-other

I basically get a 404, since the page whatsnew.html no longer exists. So, do we need to drop in a blank page there?

datapythonista · 2018-12-02T00:47:16Z

@ismailuddin do you have time to review the mentioned problem, and see if you can fix it.

ismailuddin · 2018-12-02T09:24:23Z

@datapythonista Ah yes sorry, I thought Tom was asking someone also about this. I'll look into it!

OK, so if I'm understanding this correctly, the broken links are arising due to old links let's say on Google pointing to this whatsnew.html page. So even if a blank page is added in, the user still won't really find the page they want? In which case, one solution could be to use nginx's rewrite flag (http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) as follows:

rewrite (http://pandas.pydata.org/pandas-docs/stable/)whatsnew.html#whatsnew-(\d{1})(\d{1,2})(\d{1})-((\w+-*)+) $1whatsnew/v$2.$3.$4.html#$5

This approach would only change the /stable/ links which point to the old whatsnew.html pages, which should be the only problem?

datapythonista · 2018-12-19T12:13:18Z

didn't check in detail, but what I understand is that even for old version of pandas, we changed the urls in this way:

before: http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0190-enhancements-other
after:  http://pandas.pydata.org/pandas-docs/stable/whatsnew/v0.19.0.html#other-enhancements

So, we need to add javascript code to http://pandas.pydata.org/pandas-docs/stable/whatsnew.html, that checks what's after the #, and generates a redirect to the new URL

TomAugspurger · 2018-12-20T22:30:28Z

@datapythonista I believe that's correct. Though note that there isn't a /whatsnew.html anymore that's generated by sphinx. http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html. So if want to add JS to that page we'll need to generate an empty HTML page.

datapythonista · 2018-12-20T22:58:11Z

Implementing it shouldn't be difficult, we can just add the whatsnew.html to html_additional_pages in doc/source/conf.py, and have the javascript in its content.

We already have the system for the API redirects, but that won't work, even if we make it generic to handle non-api pages, as in this case we have a 1 to N redirect.

But personally, I'd prefer to have a nice 404 page, with a link to the home page, and not add more complexity with redirects. I think we should reorganize several pages (have a directory for the user guide, another for the contributor docs...), and having redirects for those changes will be quite complex.

TomAugspurger · 2018-12-21T15:16:25Z

Do you expect this to be complex? This one is quite limited in scope I think.

…

On Thu, Dec 20, 2018 at 4:58 PM Marc Garcia ***@***.***> wrote: Implementing it shouldn't be difficult, we can just add the whatsnew.html to html_additional_pages in doc/source/conf.py, and have the javascript in its content. We already have the system for the API redirects, but that won't work, even if we make it generic to handle non-api pages, as in this case we have a 1 to N redirect. But personally, I'd prefer to have a nice 404 page, with a link to the home page, and not add more complexity with redirects. I think we should reorganize several pages (have a directory for the user guide, another for the contributor docs...), and having redirects for those changes will be quite complex. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#23708 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIh5P5tVt_B5sxi0nZZcBl0W2cnqNks5u7BYIgaJpZM4Yepgt> .

datapythonista · 2018-12-21T15:35:37Z

No, not this particular case, I think redirecting the whatsnew it's quite simple, what I described in my last comment. What I think will be complex enough to not be worth, is redirecting if we reorganize the documentation. So, I was wondering if it was worth adding redirects every time we move pages, or just improve the 404, and let the users find the new content.

TomAugspurger · 2018-12-21T15:38:36Z

Ahh yes I see what you mean, and I think I agree.

…

On Fri, Dec 21, 2018 at 9:35 AM Marc Garcia ***@***.***> wrote: No, not this particular case, I think redirecting the whatsnew it's quite simple, what I described in my last comment. What I think will be complex enough to not be worth, is redirecting if we reorganize the documentation. So, I was wondering if it was worth adding redirects every time we move pages, or just improve the 404, and let the users find the new content. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#23708 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIn0YY9TwniDkYQ8qPqci4w1TRmKBks5u7P_PgaJpZM4Yepgt> .

datapythonista · 2018-12-25T00:13:54Z

Closing this, as it's not solving the problem.

@ismailuddin , as discussed, not sure if it's worth to add the redirects. But feel free to open a new PR (or reopen this), if you want to implement the approach described in #23708 (comment)

Thanks!

gfyoung reviewed Nov 15, 2018

View reviewed changes

doc/source/_static/js/whatsnew_links.js Outdated Show resolved Hide resolved

gfyoung reviewed Nov 15, 2018

View reviewed changes

doc/source/_static/js/whatsnew_links.js Outdated Show resolved Hide resolved

gfyoung added the Docs label Nov 15, 2018

Add JS function to reformat 'Whats New' links for new format

01f5905

ismailuddin force-pushed the whatsnewlinks/23695 branch from 58157c7 to 01f5905 Compare November 15, 2018 09:42

TomAugspurger added this to the 0.24.0 milestone Nov 16, 2018

TomAugspurger reviewed Nov 16, 2018

View reviewed changes

doc/source/_static/js/whatsnew_links.js Outdated Show resolved Hide resolved

doc/source/conf.py Outdated Show resolved Hide resolved

TomAugspurger and others added 2 commits November 16, 2018 15:27

Update doc/source/conf.py

8cc0f5f

Fix function name typo Co-Authored-By: ismailuddin <[email protected]>

Remove trailing whitespace

7b7315d

jreback removed this from the 0.24.0 milestone Nov 25, 2018

datapythonista changed the title ~~Whatsnewlinks/23695~~ DOC: Implement javascript to redirect old whatsnew pages to new ones Dec 2, 2018

datapythonista self-assigned this Dec 19, 2018

datapythonista closed this Dec 25, 2018

datapythonista mentioned this pull request Jan 4, 2019

DOC: Split api.rst in sections #24451

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: Implement javascript to redirect old whatsnew pages to new ones #23708

DOC: Implement javascript to redirect old whatsnew pages to new ones #23708

ismailuddin commented Nov 14, 2018

gfyoung Nov 15, 2018

gfyoung Nov 15, 2018

ismailuddin Nov 15, 2018

gfyoung Nov 15, 2018

ismailuddin Nov 15, 2018

ismailuddin Nov 15, 2018

gfyoung commented Nov 15, 2018

pep8speaks commented Nov 15, 2018

codecov bot commented Nov 15, 2018 •

edited

Loading

ismailuddin commented Nov 16, 2018

TomAugspurger commented Nov 16, 2018

TomAugspurger commented Nov 16, 2018

datapythonista commented Dec 2, 2018

ismailuddin commented Dec 2, 2018 •

edited

Loading

datapythonista commented Dec 19, 2018

TomAugspurger commented Dec 20, 2018 •

edited

Loading

datapythonista commented Dec 20, 2018

TomAugspurger commented Dec 21, 2018 via email

datapythonista commented Dec 21, 2018

TomAugspurger commented Dec 21, 2018 via email

datapythonista commented Dec 25, 2018

DOC: Implement javascript to redirect old whatsnew pages to new ones #23708

DOC: Implement javascript to redirect old whatsnew pages to new ones #23708

Conversation

ismailuddin commented Nov 14, 2018

gfyoung Nov 15, 2018

Choose a reason for hiding this comment

gfyoung Nov 15, 2018

Choose a reason for hiding this comment

ismailuddin Nov 15, 2018

Choose a reason for hiding this comment

gfyoung Nov 15, 2018

Choose a reason for hiding this comment

ismailuddin Nov 15, 2018

Choose a reason for hiding this comment

ismailuddin Nov 15, 2018

Choose a reason for hiding this comment

gfyoung commented Nov 15, 2018

pep8speaks commented Nov 15, 2018

codecov bot commented Nov 15, 2018 • edited Loading

Codecov Report

ismailuddin commented Nov 16, 2018

TomAugspurger commented Nov 16, 2018

TomAugspurger commented Nov 16, 2018

datapythonista commented Dec 2, 2018

ismailuddin commented Dec 2, 2018 • edited Loading

datapythonista commented Dec 19, 2018

TomAugspurger commented Dec 20, 2018 • edited Loading

datapythonista commented Dec 20, 2018

TomAugspurger commented Dec 21, 2018 via email

datapythonista commented Dec 21, 2018

TomAugspurger commented Dec 21, 2018 via email

datapythonista commented Dec 25, 2018

codecov bot commented Nov 15, 2018 •

edited

Loading

ismailuddin commented Dec 2, 2018 •

edited

Loading

TomAugspurger commented Dec 20, 2018 •

edited

Loading