Background federated joins are currently broken since they timeout after
30s. This timeout didn't exist before the refactor. It should still exist but it needs to be extended to allow for the additional time it can take a server to generate the /send_join response when joining a complex room.
- CSAPI used wrong function to create pagination cache (which caused nil map errs)
- Copied an array wrong in room hierarchy impl, which caused a nil room list to return
- A few commits ago we moved `Queryer` to be init when fed API is set
- This causes Queryer to be nil before it is set
- Causes nil error in `userapi.NewInternalAPI`
- Rearrange to avoid this
- Opt for referring to it as 'room hierarchies', reflecting the route
- Split msc2945.go across clientapi and roomserverapi
- Still TODO fed api
- Some other TODOs still to do
The previous version was getting **ALL** membership events (as
`ClientEvents`, so going through `NewEventFromTrustedJSONWithID`) for a
given room.
Now we are querying only locally joined users as `ClientEvents`, which
should **significantly** reduce allocations.
Take for example a large room with 2k membership events, but only 1
local user - avoiding 1999 `NewEventFromTrustedJSONWithID` calls just to
calculate the `roomSize` which we can also query by other means.
This is also getting called for every `OutputRoomEvent` in the userAPI.
Benchmark with 1 local user and 100 remote users.
```
pkg: github.com/matrix-org/dendrite/userapi/consumers
cpu: 12th Gen Intel(R) Core(TM) i5-12500H
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
LocalRoomMembers-16 375.9µ ± 7% 327.6µ ± 6% -12.85% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
LocalRoomMembers-16 79.426Ki ± 0% 8.507Ki ± 0% -89.29% (p=0.000 n=10)
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
LocalRoomMembers-16 1015.0 ± 0% 277.0 ± 0% -72.71% (p=0.000 n=10)
```
Since the removal of `build.sh`, we don't include any information about
the revision Dendrite was build from. Since go1.18, the revision a
binary was build from is automatically included, so we can try to get
that instead.
This also adds a `dendrite_up` metric showing the current version
(`dendrite_up{version="0.13.1+c796f20"} 1`)
Closes#2993
If old messages build up in the input stream and do not get processed
successfully, this can create a significant drift between the stream
first sequence and the consumer ack floors, which results in a slow and
expensive start-up when interest-based retention is in use.
If a message is sat in the stream for 24 hours, it's probably not going
to get processed successfully, so let NATS drop them instead. Dendrite
can reconcile by fetching missing events later if it needs to.
---------
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
The syncapi operates using userID's so when querying for the previous
state event we need to lookup the userID from the given senderID before
the state query.
When we're adding state to the database, we check which eventNIDs are
already in a block, if we already have that eventNID, we remove it from
the list. In its current form we would skip over eventNIDs in the case
we already found a match (we're decrementing `i` twice)
My theory is, that when we later get the state blocks, we are receiving
"too many" eventNIDs (well, yea, we stored too many), which may or may
not can result in state resets when comparing different state snapshots.
(e.g. when adding state we stored a eventNID by accident because we
skipped it, later we add more state and are not adding it because we
don't skip it)
This set of changes introduces a few (compatible) changes to the Helm
chart:
- Allow PVC class to be set on each PVC, not only one-for-all.
- Allow Prometheus servicemonitor and rules labels to be empty.
- Have the option to generate the ingress (incl. TLS config) based on
dendrite_config.
* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
Signed-off-by: Omar Pakker <Omar007@users.noreply.github.com>
---------
Signed-off-by: Omar Pakker <Omar007@users.noreply.github.com>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
