Backwardly-extensible history in bridged portal rooms (SPEC-440)

[matrixbot]

It would be lovely if bridged portal rooms that come from some bridged universe that supports history fetching (e.g. Gitter or Slack) could themselves be represented by lazy history backfill inside Matrix, matching the way that individual Matrix HSes can lazily backfill history from other HSes.

There's an annoying asymmetry in that native Matrix rooms can be lazily backfilled over server-to-server federation, but there isn't a similar ability for portal rooms bridged from a 3PN.

(Imported from https://matrix.org/jira/browse/SPEC-440)

(Reported by @leonerd)

[matrixbot]

Jira watchers: @leonerd

[ara4n]

Just had a big discussion with @leonerd and @erikjohnston on this. Quick notes were:

1. You could insert the ancient history as a big 'netsplit' in the DAG, forking from the m.room.create event, possibly putting some kind of metadata on m.room.create at room creation time in order to bypass anti-DoS heuristics for the HS sending the ancient history.

Erik doesn't like this though because it doesn't handle state nicely, and to display the netsplitted history in correct chronological order we'd need to be much smarter than the current implementation.

2. You could insert the ancient history as a 'u-turn' in the DAG at the point of the m.room.create event. I'm a bit unclear as to how the causality of the events work here and how it differs from Also send room members' presence in per-room initialSync matrix-spec-proposals#1 or Proposal for human ID rules. matrix-spec-proposals#3.

3. You could just create synthetic events which look like normal DAG events but exist before the m.room.create event. Their signatures and prev-events would all be missing, but they would be blindly trusted based on the HS who is allowed to serve them (based on metadata in the m.room.create event). Thus you'd have a perimeter in the DAG beyond which events are no longer decentralised or signed, but are blindly trusted to let HSes insert ancient history provided by ASes.

This seemed to be the preferred route, as it leaves the existing federation DAG code entirely as it is, and just has one whopping big special-case to support these "pre-history" events - in contrast with option matrix-org/matrix-spec-proposals#1 which meant improving and special-casing net-split semantics significantly to be aware of pre-history.

The convo came up based on whether ASes are better handling pre-history via a 'side-car' architecture where they just proxy Matrix straight through to the backing protocol - the conclusion is that Matrix itself needs to handle pre-history nicely, as per this bug.

[kegsay]

Uhhh, fairly sure we specced this (but not implemented) in AS API v1. See http://matrix.org/docs/spec/application_service/unstable.html#timestamp-massaging

[AndrewJDR]

I'm curious how this may fit in with message editing. Specifically @Half-Shot 's matrix-org/matrix-spec-proposals#1695 refers to m.relates_to ... could this same bit of metadata be used for inserting pre-history messages, but rather than redacting the message it relates to, it is chaining onto it?

Curious if there are opportunities for unification here on these problems.