Accept text/fragment+html as a content-type

[koddsson]

Turns out that text/html; fragment isn't a valid mimetype which causes some issues with gzip compression in nginx. Changing it to use the extension fragment in the text/html mimetype should correct this oversight.

This PR adds the text/fragment+html mimetype to the accept header rather than straight replacing it so we can migrate github.com without downtime.

[keithamus]

👍

I'd like to put it here for posterity:

The Accept header, as defined in RFC2616 Sec 14.1 does allow for text/html; fragment. An Accept header consists of a media-range (which is a media-type where the mime can be or type/* or */*) followed by any number of accept-params (which are ; token or ; token = "string" ). The Content-Type header (defined by RFC2616 Sec 14.17) allows for a media-type (defined in Sec 3.7) which looks much the same but cannot use the media-range wildcards.

text/html; fragment is a perfectly valid media-type, therefore a perfectly valid Content-Type header and also a valid Accept header.

The reason this needs to be done is that for certain parts of Nginx - namely the gzip module, and how it determines it can gzip content - only accepts mime-types not media-types. Mime Types are different to Media Types and only allow for a grammar of type/subtype or type/extension+subtype (as defined in RFC2045).

In other words, when nginx sees a Content-Type header of text/html; fragment - it discards it as an invalid Mime Type and will not gzip it. Nginx is both wrong and right here - text/html; fragment is not a valid mime type! But Content-Type is not a header that contains only mime types! So nginx is not following the spec.

Getting nginx to follow the spec and have gzip allow lists use Media Types over Mime Types is an uphill battle compared to simply changing our Media Type to be one that is acceptable as a Mime Type, hence this change.

[muan]

when nginx sees a Content-Type header of text/html; fragment - it discards it as an invalid Mime Type and will not gzip it.

How does this new string, which still contains media-type, work? Do we know that they gets parsed successfully regardless and that the "invalid" part is ignored?

[keithamus]

This new string doesn't have any real effects, because it's an Accept header which is a media-range - and which nginx just passes on. The problem is that nginx switches on Content-Type to determine what it can gzip, and what it cannot. The Content-Type (for nginx to work) must be a valid mime-type.

[muan]

Thanks for the context. Looks reasonable to me.