mirror of
https://github.com/matrix-org/dendrite.git
synced 2025-12-06 22:43:10 -06:00
* Correctly redact events over federation (#2526) * Ensure we check powerlevel/origin before redacting an event * Add passing test * Use pl.UserLevel * Make check more readable, also check for the sender * Add new next steps page to the documentation * Highlighting in docs * Rename the page to "Optimise your installation" * Attempt to raise the file descriptor limit at startup (#2527) * Add `--difference` to `resolve-state` tool * Make the linter happy again * generic CaddyFile in front of Dendrite (monolith) (#2531) for Caddy 2.5.x Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu> * Handle state before, send history visibility in output (#2532) * Check state before event * Tweaks * Refactor a bit, include in output events * Don't waste time if soft failed either * Tweak control flow, comments, use GMSL history visibility type * Fix rare panic when returning user devices over federation (#2534) * Add `InputDeviceListUpdate` to the keyserver, remove old input API (#2536) * Add `InputDeviceListUpdate` to the keyserver, remove old input API * Fix copyright * Log more information when a device list update fails * Fix nats.go commit (#2540) Signed-off-by: Jean Lucas <jean@4ray.co> * Don't return `end` if there are not more messages (#2542) * Be more spec compliant * Move lazyLoadMembers to own method * Return an error if trying to invite a malformed user ID (#2543) * Add `evacuateUser` endpoint, use it when deactivating accounts (#2545) * Add `evacuateUser` endpoint, use it when deactivating accounts * Populate the API * Clean up user devices when deactivating * Include invites, delete pushers * Silence presence logs (#2547) * Blacklist `Guest users can join guest_access rooms` test until it can be investigated * Disable WebAssembly builds for now * Try to fix backfilling (#2548) * Try to fix backfilling * Return start/end to not confuse clients * Update GMSL * Update GMSL * Roomserver producers package (#2546) * Give the roomserver a producers package * Change init point * Populate ACLs API * Fix build issues * `RoomEventProducer` naming * Version 0.8.9 (#2549) * Version 0.8.9 * Update changelog * feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550) * fix: ignore unknown keys in appservice configs fixes matrix-org/dendrite#1567 * feat: verify required fields in appservice configs * Use new testrig for key changes tests (#2552) * Use new testrig for tests * Log the error message * Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554) * Make more use of base.BaseDendrite * Fix QuerySharedUsers if no UserIDs are supplied * Return clearer error when no state NID exists for an event (#2555) * Wrap error from `SnapshotNIDFromEventID` * Hopefully fix read receipts timestamps (#2557) This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`. * Fix nil pointer access when redacting events (#2560) * Fix issue `uint64 values with high bit are not supported` in presence (#2562) * Fix issue #2528 * Use gomatrixserverlib.Timestamp * Use ParseUint instead of ParseInt * Update Pinecone to matrix-org/pinecone@1ce778f * Ristretto cache (#2563) * Try Ristretto cache * Tweak * It's beautiful * Update GMSL * More strict keyable interface * Fix that some more * Make less panicky * Don't enforce mutability checks for now * Determine mutability using deep equality * Tweaks * Namespace keys * Make federation caches mutable * Update cost estimation, add metric * Update GMSL * Estimate cost for metrics better * Reduce counters a bit * Try caching events * Some guards * Try again * Try this * Use separate caches for hopefully better hash distribution * Fix bug with admitting events into cache * Try to fix bugs * Check nil * Try that again * Preserve order jeezo this is messy * thanks VS Code for doing exactly the wrong thing * Try this again * Be more specific * aaaaargh * One more time * That might be better * Stronger sorting * Cache expiries, async publishing of EDUs * Put it back * Use a shared cache again * Cost estimation fixes * Update ristretto * Reduce counters a bit * Clean up a bit * Update GMSL * 1GB * Configurable cache sizees * Tweaks * Add `config.DataUnit` for specifying friendly cache sizes * Various tweaks * Update GMSL * Add back some lazy loading caching * Include key in cost * Include key in cost * Tweak max age handling, config key name * Only register prometheus metrics if requested * Review comments @S7evinK * Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere) * Review comments * Update sample configs * Update GHA Workflow * Update Complement images to Go 1.18 * Remove the cache test from the federation API as we no longer guarantee immediate cache admission * Don't check the caches in the renewal test * Possibly fix the upgrade tests * Update to matrix-org/gomatrixserverlib#322 * Update documentation to refer to Go 1.18 * Minor SendToDevice fix (#2565) * Avoid unnecessary marshalling if sending to the local server * Fix ordering of ToDevice messages * Revive SendToDevice test * Use `/v3` to request media from remote servers (update to matrix-org/gomatrixserverlib#324) * Pointerise `types.RoomInfo` in the cache so we can update it in-place in the latest events updater * Add a Troubleshooting page * Update `sytest-whitelist` * Use sync API database in `filterSharedUsers` (#2572) * Add function to the sync API storage package for filtering shared users * Use the database instead of asking the RS API * Fix unit tests * Fix map handling in `filterSharedUsers` * Update 1_createusers.md (#2571) * Update 1_createusers.md Added description on how to create user accounts when running in docker. * Update 1_createusers.md Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Fix connection_string format in dendrite-sample.polylith.yaml (#2574) * History visibility database changes (#2533) * Add new history_visibility column * Update SQL queries to include history_visibility * Store the history visibilty calculated by the roomserver * Update GMSL * Update migrations * Fix migration * Update GMSL * Fix `go.sum` * Update GMSL to use sql.Scanner & sql.Valuer * Re-order migration/table creation * Update gomatrixserverlib * Add history_visibility column to current_room_state * Fix migrations * Return error instead of Fatal log Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Tweak cache counters (#2575) * Tweak cache counters This makes the number of counters relative to the maximum cache size. Since the counters effectively manage the size of the bloom filter, larger caches need more counters and smaller caches need less. 10 counters per 1KB data means that the default cache size of 1GB should result in a bloom filter and TinyLRU admission set of about 16MB estimated. * Remove line left by accident * Set historyVisibility in rowsToStreamEvents * Update FAQ * Add event state key cache (#2576) * Explain how SRV works in Matrix and discourage using it (#2577) * Explain how SRV works in Matrix and discourage using it * Minor tweaks to formatting Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Fix issue with membership event_nid being 0 (#2580) * docs: Add build page; correct proxy info; fix Caddy example (#2579) * Add build page; correct proxy info; fix Caddy example * Improve Caddyfile example * Apply review comments; add polylith Caddyfile * Bump tzinfo from 1.2.9 to 1.2.10 in /docs (#2584) Bumps [tzinfo](https://github.com/tzinfo/tzinfo) from 1.2.9 to 1.2.10. - [Release notes](https://github.com/tzinfo/tzinfo/releases) - [Changelog](https://github.com/tzinfo/tzinfo/blob/master/CHANGES.md) - [Commits](https://github.com/tzinfo/tzinfo/compare/v1.2.9...v1.2.10) --- updated-dependencies: - dependency-name: tzinfo dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Membership updater refactoring (#2541) * Membership updater refactoring * Pass in membership state * Use membership check rather than referring to state directly * Delete irrelevant membership states * We don't need the leave event after all * Tweaks * Put a log entry in that I might stand a chance of finding * Be less panicky * Tweak invite handling * Don't freak if we can't find the event NID * Use event NID from `types.Event` * Clean up * Better invite handling * Placate the almighty linter * Blacklist a Sytest which is otherwise fine under Complement for reasons I don't understand * Fix the sytest after all (thanks @S7evinK for the spot) * Try to fix HTTP 500s on `/members` (#2581) * Update database migrations, remove goose (#2264) * Add new db migration * Update migrations Remove goose * Add possibility to test direct upgrades * Try to fix WASM test * Add checks for specific migrations * Remove AddMigration Use WithTransaction Add Dendrite version to table * Fix linter issues * Update tests * Update comments, outdent if * Namespace migrations * Add direct upgrade tests, skipping over one version * Split migrations * Update go version in CI * Fix copy&paste mistake * Use contexts in migrations Co-authored-by: kegsay <kegan@matrix.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Add .well-known/matrix/client to clientapi (#2551) Signed-off-by: Jonathan Bartlett <jonathan@jonnobrow.co.uk> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Remove `room_id` field from MSC2946 stripped events (closes #2588) * Remove `goose` from Dockerfiles * Make the User API responsible for sending account data output events (#2592) * Make the User API responsible for sending account data output events * Clean up producer * Review comments * Update NATS Server and nats.go to use upstream * Set CORS headers for HTTP 404 and 405 errors (#2599) * Set CORS headers for the 404s * Use custom handlers, plus one for HTTP 405 too * Tweak setup * Add to muxes too * Tidy up some more * Use built-in HTTP 404 handler * Don't bother setting it for federation-facing * Optimise checking other servers allowed to see events (#2596) * Try optimising checking if server is allowed to see event * Fix error * Handle case where snapshot NID is 0 * Fix query * Update SQL * Clean up `CheckServerAllowedToSeeEvent` * Not supported on SQLite * Maybe placate the unit tests * Review comments * De-race `types.RoomInfo` (#2600) * De-race `CompleteSync` (#2601) The `err` was coming from outside of the goroutine and being written to by concurrent goroutines. * Version 0.9.0 (#2602) Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: Till Faelligen <davidf@element.io> Co-authored-by: Emanuele Aliberti <dev@mtka.eu> Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu> Co-authored-by: Jean Lucas <jean@4ray.co> Co-authored-by: Kabir Kwatra <kabir@kwatra.me> Co-authored-by: andreever <52261463+andreever@users.noreply.github.com> Co-authored-by: Maximilian Gaedig <38767445+MaximilianGaedig@users.noreply.github.com> Co-authored-by: Tulir Asokan <tulir@maunium.net> Co-authored-by: Matt Holt <mholt@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kegsay <kegan@matrix.org> Co-authored-by: Jonathan Bartlett <34320158+Jonnobrow@users.noreply.github.com>
172 lines
11 KiB
Go
172 lines
11 KiB
Go
// Copyright 2020 The Matrix.org Foundation C.I.C.
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package storage
|
|
|
|
import (
|
|
"context"
|
|
|
|
"github.com/matrix-org/dendrite/roomserver/storage/shared"
|
|
"github.com/matrix-org/dendrite/roomserver/storage/tables"
|
|
"github.com/matrix-org/dendrite/roomserver/types"
|
|
"github.com/matrix-org/gomatrixserverlib"
|
|
)
|
|
|
|
type Database interface {
|
|
// Do we support processing input events for more than one room at a time?
|
|
SupportsConcurrentRoomInputs() bool
|
|
// RoomInfo returns room information for the given room ID, or nil if there is no room.
|
|
RoomInfo(ctx context.Context, roomID string) (*types.RoomInfo, error)
|
|
// Store the room state at an event in the database
|
|
AddState(
|
|
ctx context.Context,
|
|
roomNID types.RoomNID,
|
|
stateBlockNIDs []types.StateBlockNID,
|
|
state []types.StateEntry,
|
|
) (types.StateSnapshotNID, error)
|
|
|
|
MissingAuthPrevEvents(
|
|
ctx context.Context, e *gomatrixserverlib.Event,
|
|
) (missingAuth, missingPrev []string, err error)
|
|
|
|
// Look up the state of a room at each event for a list of string event IDs.
|
|
// Returns an error if there is an error talking to the database.
|
|
// The length of []types.StateAtEvent is guaranteed to equal the length of eventIDs if no error is returned.
|
|
// Returns a types.MissingEventError if the room state for the event IDs aren't in the database
|
|
StateAtEventIDs(ctx context.Context, eventIDs []string) ([]types.StateAtEvent, error)
|
|
// Look up the numeric IDs for a list of string event types.
|
|
// Returns a map from string event type to numeric ID for the event type.
|
|
EventTypeNIDs(ctx context.Context, eventTypes []string) (map[string]types.EventTypeNID, error)
|
|
// Look up the numeric IDs for a list of string event state keys.
|
|
// Returns a map from string state key to numeric ID for the state key.
|
|
EventStateKeyNIDs(ctx context.Context, eventStateKeys []string) (map[string]types.EventStateKeyNID, error)
|
|
// Look up the numeric state data IDs for each numeric state snapshot ID
|
|
// The returned slice is sorted by numeric state snapshot ID.
|
|
StateBlockNIDs(ctx context.Context, stateNIDs []types.StateSnapshotNID) ([]types.StateBlockNIDList, error)
|
|
// Look up the state data for each numeric state data ID
|
|
// The returned slice is sorted by numeric state data ID.
|
|
StateEntries(ctx context.Context, stateBlockNIDs []types.StateBlockNID) ([]types.StateEntryList, error)
|
|
// Look up the state data for the state key tuples for each numeric state block ID
|
|
// This is used to fetch a subset of the room state at a snapshot.
|
|
// If a block doesn't contain any of the requested tuples then it can be discarded from the result.
|
|
// The returned slice is sorted by numeric state block ID.
|
|
StateEntriesForTuples(
|
|
ctx context.Context,
|
|
stateBlockNIDs []types.StateBlockNID,
|
|
stateKeyTuples []types.StateKeyTuple,
|
|
) ([]types.StateEntryList, error)
|
|
// Look up the Events for a list of numeric event IDs.
|
|
// Returns a sorted list of events.
|
|
Events(ctx context.Context, eventNIDs []types.EventNID) ([]types.Event, error)
|
|
// Look up snapshot NID for an event ID string
|
|
SnapshotNIDFromEventID(ctx context.Context, eventID string) (types.StateSnapshotNID, error)
|
|
// Stores a matrix room event in the database. Returns the room NID, the state snapshot and the redacted event ID if any, or an error.
|
|
StoreEvent(
|
|
ctx context.Context, event *gomatrixserverlib.Event, authEventNIDs []types.EventNID,
|
|
isRejected bool,
|
|
) (types.EventNID, types.RoomNID, types.StateAtEvent, *gomatrixserverlib.Event, string, error)
|
|
// Look up the state entries for a list of string event IDs
|
|
// Returns an error if the there is an error talking to the database
|
|
// Returns a types.MissingEventError if the event IDs aren't in the database.
|
|
StateEntriesForEventIDs(ctx context.Context, eventIDs []string) ([]types.StateEntry, error)
|
|
// Look up the string event state keys for a list of numeric event state keys
|
|
// Returns an error if there was a problem talking to the database.
|
|
EventStateKeys(ctx context.Context, eventStateKeyNIDs []types.EventStateKeyNID) (map[types.EventStateKeyNID]string, error)
|
|
// Look up the numeric IDs for a list of events.
|
|
// Returns an error if there was a problem talking to the database.
|
|
EventNIDs(ctx context.Context, eventIDs []string) (map[string]types.EventNID, error)
|
|
// Set the state at an event. FIXME TODO: "at"
|
|
SetState(ctx context.Context, eventNID types.EventNID, stateNID types.StateSnapshotNID) error
|
|
// Lookup the event IDs for a batch of event numeric IDs.
|
|
// Returns an error if the retrieval went wrong.
|
|
EventIDs(ctx context.Context, eventNIDs []types.EventNID) (map[types.EventNID]string, error)
|
|
// Opens and returns a room updater, which locks the room and opens a transaction.
|
|
// The GetRoomUpdater must have Commit or Rollback called on it if this doesn't return an error.
|
|
// If this returns an error then no further action is required.
|
|
GetRoomUpdater(ctx context.Context, roomInfo *types.RoomInfo) (*shared.RoomUpdater, error)
|
|
// Look up event references for the latest events in the room and the current state snapshot.
|
|
// Returns the latest events, the current state and the maximum depth of the latest events plus 1.
|
|
// Returns an error if there was a problem talking to the database.
|
|
LatestEventIDs(ctx context.Context, roomNID types.RoomNID) ([]gomatrixserverlib.EventReference, types.StateSnapshotNID, int64, error)
|
|
// Look up the active invites targeting a user in a room and return the
|
|
// numeric state key IDs for the user IDs who sent them along with the event IDs for the invites.
|
|
// Returns an error if there was a problem talking to the database.
|
|
GetInvitesForUser(ctx context.Context, roomNID types.RoomNID, targetUserNID types.EventStateKeyNID) (senderUserIDs []types.EventStateKeyNID, eventIDs []string, err error)
|
|
// Save a given room alias with the room ID it refers to.
|
|
// Returns an error if there was a problem talking to the database.
|
|
SetRoomAlias(ctx context.Context, alias string, roomID string, creatorUserID string) error
|
|
// Look up the room ID a given alias refers to.
|
|
// Returns an error if there was a problem talking to the database.
|
|
GetRoomIDForAlias(ctx context.Context, alias string) (string, error)
|
|
// Look up all aliases referring to a given room ID.
|
|
// Returns an error if there was a problem talking to the database.
|
|
GetAliasesForRoomID(ctx context.Context, roomID string) ([]string, error)
|
|
// Get the user ID of the creator of an alias.
|
|
// Returns an error if there was a problem talking to the database.
|
|
GetCreatorIDForAlias(ctx context.Context, alias string) (string, error)
|
|
// Remove a given room alias.
|
|
// Returns an error if there was a problem talking to the database.
|
|
RemoveRoomAlias(ctx context.Context, alias string) error
|
|
// Build a membership updater for the target user in a room.
|
|
MembershipUpdater(ctx context.Context, roomID, targetUserID string, targetLocal bool, roomVersion gomatrixserverlib.RoomVersion) (*shared.MembershipUpdater, error)
|
|
// Lookup the membership of a given user in a given room.
|
|
// Returns the numeric ID of the latest membership event sent from this user
|
|
// in this room, along a boolean set to true if the user is still in this room,
|
|
// false if not.
|
|
// Returns an error if there was a problem talking to the database.
|
|
GetMembership(ctx context.Context, roomNID types.RoomNID, requestSenderUserID string) (membershipEventNID types.EventNID, membershipNID tables.MembershipState, stillInRoom, isRoomForgotten bool, err error)
|
|
// Lookup the membership event numeric IDs for all user that are or have
|
|
// been members of a given room. Only lookup events of "join" membership if
|
|
// joinOnly is set to true.
|
|
// Returns an error if there was a problem talking to the database.
|
|
GetMembershipEventNIDsForRoom(ctx context.Context, roomNID types.RoomNID, joinOnly bool, localOnly bool) ([]types.EventNID, error)
|
|
// EventsFromIDs looks up the Events for a list of event IDs. Does not error if event was
|
|
// not found.
|
|
// Returns an error if the retrieval went wrong.
|
|
EventsFromIDs(ctx context.Context, eventIDs []string) ([]types.Event, error)
|
|
// Publish or unpublish a room from the room directory.
|
|
PublishRoom(ctx context.Context, roomID string, publish bool) error
|
|
// Returns a list of room IDs for rooms which are published.
|
|
GetPublishedRooms(ctx context.Context) ([]string, error)
|
|
// Returns whether a given room is published or not.
|
|
GetPublishedRoom(ctx context.Context, roomID string) (bool, error)
|
|
|
|
// TODO: factor out - from currentstateserver
|
|
|
|
// GetStateEvent returns the state event of a given type for a given room with a given state key
|
|
// If no event could be found, returns nil
|
|
// If there was an issue during the retrieval, returns an error
|
|
GetStateEvent(ctx context.Context, roomID, evType, stateKey string) (*gomatrixserverlib.HeaderedEvent, error)
|
|
GetStateEventsWithEventType(ctx context.Context, roomID, evType string) ([]*gomatrixserverlib.HeaderedEvent, error)
|
|
// GetRoomsByMembership returns a list of room IDs matching the provided membership and user ID (as state_key).
|
|
GetRoomsByMembership(ctx context.Context, userID, membership string) ([]string, error)
|
|
// GetBulkStateContent returns all state events which match a given room ID and a given state key tuple. Both must be satisfied for a match.
|
|
// If a tuple has the StateKey of '*' and allowWildcards=true then all state events with the EventType should be returned.
|
|
GetBulkStateContent(ctx context.Context, roomIDs []string, tuples []gomatrixserverlib.StateKeyTuple, allowWildcards bool) ([]tables.StrippedEvent, error)
|
|
// JoinedUsersSetInRooms returns how many times each of the given users appears across the given rooms.
|
|
JoinedUsersSetInRooms(ctx context.Context, roomIDs, userIDs []string) (map[string]int, error)
|
|
// GetLocalServerInRoom returns true if we think we're in a given room or false otherwise.
|
|
GetLocalServerInRoom(ctx context.Context, roomNID types.RoomNID) (bool, error)
|
|
// GetServerInRoom returns true if we think a server is in a given room or false otherwise.
|
|
GetServerInRoom(ctx context.Context, roomNID types.RoomNID, serverName gomatrixserverlib.ServerName) (bool, error)
|
|
// GetKnownUsers searches all users that userID knows about.
|
|
GetKnownUsers(ctx context.Context, userID, searchString string, limit int) ([]string, error)
|
|
// GetKnownRooms returns a list of all rooms we know about.
|
|
GetKnownRooms(ctx context.Context) ([]string, error)
|
|
// ForgetRoom sets a flag in the membership table, that the user wishes to forget a specific room
|
|
ForgetRoom(ctx context.Context, userID, roomID string, forget bool) error
|
|
|
|
GetHistoryVisibilityState(ctx context.Context, roomInfo *types.RoomInfo, eventID string, domain string) ([]*gomatrixserverlib.Event, error)
|
|
}
|