matrix.org/content/blog/2020/10/2020-10-16-this-week-in-mat...

423 lines
23 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

+++
title = "This Week in Matrix 2020-10-16"
path = "/blog/2020/10/16/this-week-in-matrix-2020-10-16"
[taxonomies]
author = ["Ben Parsons"]
category = ["This Week in Matrix"]
[extra]
image = "https://matrix.org/blog/img/2020-10-16-cpu.png"
+++
## Matrix Live 🎙
{{ youtube_player(video_id="xwsmdscTjJM") }}
In which I chat with Kitsune about the work done to get a [Matrix URI schema](https://github.com/matrix-org/matrix-doc/pull/2312/) agreed, and the state of the work.
See also, [Open Tech Will Save Us #7] took place this week! Go watch.
## Dept of *Status of Matrix* 🌡️
### Meta-counting
As a crude measure of growth, [Matthew](https://matrix.to/#/@matthew:matrix.org) commented about [#twim:matrix.org]:
> I love that this room has ~700 people in it, spread over ~350 servers :D
That is something to love. Come join us in the room to share your news and see what's new from others.
### Elokapina (Extinction Rebellion Finland) migrates to Matrix
[kapina-jaywink](https://matrix.to/#/@jaywink:elokapina.fi) told us:
> In recent weeks the XR Finland community has been moving over from Wire to our own Matrix homeserver for encrypted secure chat. This was something that had been planned for a while but kicked off in recent weeks due to Wire suffering from serious encryption key delivery issues, causing messages for many to be unreadable in large groups. Currently we've migrated almost 300 rebels with more to come. Feedback has mostly been very positive, people generally like the Element clients 🤩 One of the interesting changes has been a huge uptick in the amount of discussion, which can be taken as a good sign. The plan is to next start bridging to some of the international XR chapters, for example those on Mattermost, Telegram and Slack. And maybe get them over to Matrix too eventually ;)
>
> To aid in community management, we've started creating [a bot called Bubo](https://github.com/elokapina/bubo). Right now it mostly helps with maintaining rooms and allowing mass invites, but more features to help the community cooperate are coming. We were planning to utilize (actual) communities so it has some functionality for those, but decided then to wait for the communities rewrite. It doesn't yet have any releases, will update in coming weeks as features are added and releases made.
[kapina-jaywink](https://matrix.to/#/@jaywink:elokapina.fi) also
> wants to clarify that XR is a decentralized movement and this does not mean other chapters will adapt Matrix - but we can hope and for sure here in Finland we'll be spreading the good experiences to other chapters ;)
## Dept of Spec 📜
### New Spec Website!
[wbamberg](https://matrix.to/#/@wbamberg:matrix.org) told us:
> We're working on a new platform for the specification docs, aimed initially at improving navigation and general usability.
>
> The initial demo site is at <https://adoring-einstein-5ea514.netlify.app/spec/> and the main tracking bug is at <https://github.com/matrix-org/matrix-doc/issues/2822>
### Spec
[anoa](https://matrix.to/#/@andrewm:amorgan.xyz) reported:
> Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at <https://matrix.org/docs/spec/proposals>.
>
>
> #### MSC Status
>
> **Merged MSCs:**
>
> * *No MSCs were merged this week.*
>
> **MSCs in Final Comment Period:**
>
> * *No MSCs are in FCP.*
>
> **New MSCs:**
>
> * [MSC2821: Testing Pushers](https://github.com/matrix-org/matrix-doc/pull/2821)
> * [MSC2815: Proposal to allow room moderators to view redacted event content](https://github.com/matrix-org/matrix-doc/pull/2815)
>
> * [MSC2813: Widget API error handling and validation](https://github.com/matrix-org/matrix-doc/pull/2813)
>
> #### Spec Core Team
>
> In terms of Spec Core Team MSC focus for this week, widgets are still the main focus: [MSC2774](https://github.com/matrix-org/matrix-doc/issues/2774) (widget URL template param), [MSC2765](https://github.com/matrix-org/matrix-doc/issues/2765) (widget avatars), and [MSC2790](https://github.com/matrix-org/matrix-doc/issues/2790) (modal widgets).
![2020-10-16-KA5Yo-stacked_area_chart.png](/blog/img/2020-10-16-KA5Yo-stacked_area_chart.png)
## Dept of Servers 🏢
### Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
[Neil Alexander](https://matrix.to/#/@neilalexander:matrix.org) reported:
> We [released our first beta last week](https://matrix.org/blog/2020/10/08/dendrite-is-entering-beta) and we've been busy taking feedback from the community and fixing problems as they have been reported to us. We will be cutting a v0.2.0 release candidate later today, which contains a significant number of important fixes and performance improvements.
>
> We didn't publish a full changelog last week due to the beta announcement, therefore the following list includes changes from the last two weeks:
>
> * Database migrations are now ran automatically at startup
>
> * State resolution v2 performance has been improved dramatically on large state sets, seen to be running up to 20x faster in some rooms
> * Dendrite no longer runs state resolution over single trusted state snapshots
>
> * Monolith deployments with Kafka now work again (each component sets up connections independently to avoid duplicate consumers)
> * Dendrite now correctly rejects invalid UTF-8 - thanks to [Lesterpig](https://github.com/Lesterpig)
>
> * Fully read markers are now handled - thanks to [Lesterpig](https://github.com/Lesterpig)
> * The `completed` field is now returned properly for user-interactive auth - thanks to [Lesterpig](https://github.com/Lesterpig)
>
> * The devices table now tracks last seen timestamps, IPs and user agents - thanks to [S7evinK](https://github.com/S7evinK)
> * A bug has been fixed in the reverse topological ordering algorithm which resulted in us giving up on inbound references after the first prev/auth event
>
> * A bug with concurrent map writes in the rate limiting in the client API has been fixed
> * Forward extremities and their previous events are now checked fully against the database
>
> * Typing events are now ignored if the sender domain doesn't match the origin
> * Duplicate redaction entries no longer result in database errors
>
> * Some bugs in `/send` have been fixed where the full room state wasn't requested properly before sending a new snapshot to the roomserver
> * The membership updaters now use database writers properly, which fixes some SQLite locking issues
>
> * The sync API no longer burns CPU processing unnecessary device key change notifications
> * `QueryStateAfterEvents` now resolves state from multiple snapshots properly
>
> * Cumulative room state is now sent to the roomserver when creating rooms locally
> * Missing auth events will now be retrieved from multiple servers in the room if necessary
>
> * Federation requests now have variable timeouts, allowing us to wait much longer for a remote server to process certain tasks
> * The `/send` endpoint now returns 500 errors far less often, reducing the frequency that other servers back off from sending to Dendrite
>
> * Backfill no longer uses the request context for persisting events, which was resulting in us failing to store those events sometimes
> * Invite stripped state from the sync API now includes the stripped invite itself, so that Element Web can display who sent the invite properly
>
> * The signing key fetch mechanism no longer gives up if it is unable to fetch specific keys
> * Handling of invalid display name or avatar URLs in membership events has been improved
>
> Spec compliance has improved a little:
>
> * Client-server APIs: 56%, same as last week
>
> * Server-server APIs: 80%, up from 79% last week
>
> As always, please feel free to join us in [#dendrite:matrix.org](https://matrix.to/#/#dendrite:matrix.org) for general Dendrite chat, and [#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org) if you are interested in contributing!
Good grief that's a big update! For a video discussion of the status and future of Dendrite, check out [Open Tech Will Save Us #7].
[Matthew](https://matrix.to/#/@matthew:matrix.org) added:
> my personal dendrite is now in roughly the same set of rooms as my personal synapse. Dendrite is idling at 180MB of RAM, Synapse is idling at 1.8GB of RAM :)
### Conduit
[Timo](https://matrix.to/#/@timo:conduit.rs):
> Hello everyone, this week we merged the federation branch into master. It's not ready to be used properly yet, but we're merging it as it seems stable enough for now. We also improved performance of the federation branch a lot by turning off debug logs.
>
> Other news:
> - I opened two issues on element-ios which currently break register and login support on Conduit, making it completely unusable. Hopefully they can be resolved soon (https://github.com/vector-im/element-ios/issues/3736, https://github.com/vector-im/element-ios/issues/3737)
> - I'm working on an MSC for threading. It's still WIP, but you can take a look here: https://demo.codimd.org/s/SykbuAUwP
>
> Thanks to everyone who supports me on [Liberapay](https://liberapay.com/timokoesters) or Bitcoin!
### Synapse
[Neil](https://matrix.to/#/@neilj:matrix.org) offered:
> [Stonking](https://dictionary.cambridge.org/dictionary/english/stonking) week for Synapse as we landed sharded event persisters and deployed to matrix.org. This is the last significant component other than the main process to go through the sharding process and a major hurdle in horizontal scalability of Synapse.
>
> Initial results look good with event persistence [apdex](https://en.wikipedia.org/wiki/Apdex) improving, however we think there are still some significant performance improvements available through configuration and will continue to experiment.
![2020-10-16-I-Plk-Screenshot2020-10-16at17.23.31.png](/blog/img/2020-10-16-I-Plk-Screenshot2020-10-16at17.23.31.png)
> We also moved off background processes from the main process. This is significant because it means that while the main process is not shardable it really doesnt do anything anymore other than orchestration.
>
> Again the initial impact looks very promising and we will continue to tune. Having moved the background processes away it also makes profiling the main process that much easier.
![2020-10-16-fuKDh-main-cpu2.png](/blog/img/2020-10-16-fuKDh-main-cpu2.png)
> Aside from all of that we continue to progress room knocking put out a [1.21.2](https://matrix.org/blog/2020/10/15/synapse-1-21-2-released-and-security-advisory) - a bug fix release **though please please ensure you are running at least Synapse >= 1.21.0 since 1.21.0 contains a XSS security fix**.
>
> Next week we will carrying on tuning matrix.org and start to look at improving state resolution performance.
## Synapse Deployment 📥️
### Kubernetes
[Ananace](https://matrix.to/#/@ace:kittenface.studio) told us:
> Just pushed the updated image tags and [chart version](https://gitlab.com/ananace/charts) for my [K8s-optimized Synapse image](https://github.com/ananace/matrix-synapse) for version 1.21.0, as well as a [chart](https://gitlab.com/ananace/charts) update for element-web 1.7.9
then later
> And to expand on my previous update, got Synapse 1.21.2 up on the chart and K8s-optimized image.
### YunoHost
[Pierre](https://matrix.to/#/@pierre:mamieserv.fr) said:
> [YunoHost](https://yunohost.org) is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.
>
> [Synapse](https://github.com/YunoHost-Apps/synapse_ynh/tree/master) integration had been updated to 1.20.1 (1.21.0 available in branch `testing`)
>
> [Element Web](https://github.com/YunoHost-Apps/element_ynh/tree/master) integration had been updated to 1.7.8 (1.7.9 available in branch `testing`)
## Dept of Bridges 🌉
### Bridges increase their presence
[Half-Shot](https://matrix.to/#/@Half-Shot:half-shot.uk) told us:
> Small bit of news I wanted to talk about from Bridge Island. My [implementation](https://github.com/matrix-org/synapse/pull/8437) for [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409) has been merged which means appservices / bridges can now listen in for incoming presence, typing and read receipt events! This means that the Slack bridge can now reliably send your typing status to Slack, and Bifrost can reliably bridge your everything to XMPP. The MSC is still in flux and could change, but for now this could really improve the native feeling of bridges :)
>
> (Oh and I should mention anyone using matrix-appservice-bridge v2.2.0+ can use this behaviour for free)
### mautrix-python implements Half-Shot's new features
Hot on his heels, [Tulir](https://matrix.to/#/@tulir:maunium.net) announced:
> I've been adding support for the MSCs Half-Shot implemented in Synapse to my bridges:
>
> * Enabling end-to-bridge encryption now uses appservice login ([MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778)), which means setting up the shared secret login module is no longer required for e2be.
>
> * mautrix-python has support for receiving ephemeral events via [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409) in a branch, which will be merged once Synapse v1.22 is released. After it's merged, `/sync`ing with double puppets will no longer be necessary to bridge ephemeral events.
>
> Both of these will also be implemented in mautrix-go/whatsapp soon.
>
> Now I just need Half-Shot to make synapse send to-device events to appservices, after which bridges won't need any hacky `/sync`ing at all.
## Dept of Clients 📱
### Hydrogen
[Bruno](https://matrix.to/#/@bwindels:matrix.org) offered:
> Several releases this week (0.1.11 to 0.1.15) with lots of changes:
>
> * url-based navigation has landed! All navigation in the app is now done through urls, meaning you can also bookmark any UI state (e.g. grid configurations).
>
> * fixed 2 memory leaks (exposed now because you can unload your session without refreshing the page)
> * fixed an issue with libolm running out of memory if you send a message to more than 44 devices (see issue #150).
>
> * some logical additions now we have url navigation: restoring the last url when opening the app with the default route, and a button to close your session and go back to the picker.
> * the app now blocks concurrent access to the same session from different tabs (it just closes the session in the non-active tab). This will prevent multiple syncs tripping over each other writing to indexeddb (e.g. ConstraintErrors and friends).
>
> * updates are announced in the app (for now through a confirm dialog, but will use an in app notification once we have it)
> * fixes updates not installing on iOS, by having an update prompt. To get this update on iOS though, you'll need to unpin the app, and pinning it again. You'll need to login again after this. All future updates should be installable through the update prompt once you have 0.1.15 though, you won't have to do this again normally.
>
> * uses the hydrogen icon when pinning on iOS
I really recommend hitting <https://hydrogen.element.io/> - what great progress!
### Element Android
[benoit](https://matrix.to/#/@benoit.marty:matrix.org) reported:
> We are currently preparing the release of the version 1.0.9 of Element Android, which contain searching messages in clear rooms and a lots of other improvements and new features.
>
> The SDK 1.0.9 will also be released, with an updated readme, and a brand new sample app, written by Ganfra. It will help developers to start using the new SDK and can be found here: <https://github.com/matrix-org/matrix-android-sdk2-sample>. This sample app is able to let the user connects to an existing account on any homeserver with password login, display the room list, display a room timeline and send message to a room.
> a brand new sample app
YES! This is the best documentation
### Element-iOS
[Manu](https://matrix.to/#/@Manu:matrix.org) offered:
> This week, we released [1.0.16](https://github.com/vector-im/element-ios/releases/tag/v1.0.16) on the App Store (and TestFlight).
### Konheko (Sailfish client)
[Nico (@deepbluev7:neko.dev)](https://matrix.to/#/@deepbluev7:neko.dev) offered:
> I published the first preview of my Sailfish client called Konheko. While you can run Android applications on Sailfish, they usually are a subpar experience, since they really don't fit the platforms design and style and also usually don't properly send notifications.
>
> So about a year ago I started working on a Matrix client for SailfishOS, but I never really made much progress. Well, last weekend I did, and so it can now send plain text messages as well as various forms of media messages, I made a basic application icon and I've been using it this week already (for unencrypted rooms).
>
> It is still missing a lot of features, but if you want you can install it from [OpenRepos](https://openrepos.net/content/deepbluev7/konheko). Sources are available [here](https://nheko.im/nheko-reborn/konheko). Just be aware, that it currently stores all messages in RAM, so every restart will take forever to load your rooms and it may run out of RAM at some point. Storing messages in some database will come at some point. Also, a lot of menus may lead nowhere, since those are just placeholders for me atm.
![2020-10-16-QApie-Bildschirmfoto_20201016_002.png](/blog/img/2020-10-16-QApie-Bildschirmfoto_20201016_002.png)
![2020-10-16-I5_7M-Bildschirmfoto_20201016_007.png](/blog/img/2020-10-16-I5_7M-Bildschirmfoto_20201016_007.png)
### SchildiChat Web/Desktop
[SpiritCroc](https://matrix.to/#/@spiritcroc:matrix.org) announced:
> Recently, I tweaked Element-web to feature a few changes similar to SchildiChat for Android.
>
> For now, it's probably best seen rather as a proof-of-concept than a finished product, as there are still some layout bugs, and no settings available for the added features (I know some people prefer separate lists for direct and group chats).
> I consider it usable though.
>
> Particular changes compared to upstream Element-web include:
>
> * A common section for groups and direct chats in the overview
>
> * Message bubbles
> * Bigger items in the room overview
>
> * A different dark theme, similar to SchildiChat for Android
>
> I don't know how much I will work on this in the future, but I figured it might be interesting to share either way. Maybe even someone with more web-development skills than me might want to help improving it :)
>
> For further discussion, I have created [#schildichat-web:matrix.org](https://matrix.to/#/#schildichat-web:matrix.org) .
>
> The current version of SchildiChat-Desktop is available for Desktop [here](https://github.com/SpiritCroc/schildichat-desktop/releases), and I host the web variant [here](https://schildiweb.spiritcroc.de/).
> If you want to build it yourself, check out [this repo](https://github.com/SpiritCroc/schildichat-desktop).
![2020-10-16-oMUqR-1.png](/blog/img/2020-10-16-oMUqR-1.png)
### Element Web
[Neil](https://matrix.to/#/@neilj:matrix.org) offered:
> Element 1.7.9 has landed, highlights include
>
> * Many small fixes with edits and replies
>
> * Fixed a race during cross-signing key upload at registration time
> * Clarified when you have unsaved changes in profile settings
>
> Aside from that has been pushing on with the widgets project. A picture says a thousand words, so here you go.
![2020-10-16-wZher-image.png](/blog/img/2020-10-16-wZher-image.png)
> As you can see T3chguy was really pleased to have me interrupt him to take this picture. Expect the new design to merge next week.
>
> Finally we will most likely ship a new release next week to fix some Jitsi bugs.
## Dept of Ops 🛠
### Wake-on-LAN bot
[JCG](https://matrix.to/#/@jan.christian:gruenhage.xyz) announced:
> I wrote a [Wake-on-LAN](https://en.wikipedia.org/wiki/Wake-on-LAN) bot to wake up hosts by sending a matrix message. It is configurable with multiple hosts and has a list of users per host who are allowed to wake it up. It's using the matrix-rust-sdk, source is available over at <https://git.jcg.re/jcgruenhage/matrix-wol>, and if anyone has questions, feel free to join [#matrix-wol:jcg.re](https://matrix.to/#/#matrix-wol:jcg.re).
When I asked what this is used for:
> I have stuff on my workstation that I need access to most of the time, but keeping it running uses too much power (but I did it anyway so far), this is so that I can suspend it when I leave but can still power it on when I need something from there on the got
## Dept of Events and Talks 🗣️
### Talking about Bridges and Bots in Matrix (German)
[Oleg](https://matrix.to/#/@oleg:fiksel.info) announced:
> I was invited to the German Podcast MacMittwoch (no, it's not only about Macs) to talk about Bridges and Bots in Matrix. It was a very interesting and funny round.
>
> Here is the recording:
>
> * <https://macmittwoch.de/podcast/matrix_mit_bridges_und_bots>
## Dept of Interesting Projects 🛰️
### matrix-emoji-upload
[mewmew](https://matrix.to/#/@mewmew:blob.cat) offered:
> This is a script I've created for use with [MSC2545](https://github.com/matrix-org/matrix-doc/pull/2545/). It allows easy uploading of emoji packs to Matrix rooms. Feel free to check it out [on Gitea](https://git.blob.cat/mewmew/matrix-emoji-upload), or join the project room [#matrix-emoji-upload:blob.cat](https://matrix.to/#/#matrix-emoji-upload:blob.cat) if you have any questions/comments/issues.
### n8n.io support
[Matthew](https://matrix.to/#/@matthew:matrix.org) announced:
> n8n.io (FOSS extendable workflow automation) just added Matrix support! <https://n8n.io/integrations/n8n-nodes-base.matrix> and <https://github.com/n8n-io/n8n/pull/1046>
Exciting! Yet the saga that followed only adds to the excitement!
First [Oleg](https://matrix.to/#/@oleg:fiksel.info) noticed a problem:
> Could it be that only matrix.org as HS currently is supported?
>
> I'm getting `Matrix credentials are not valid!` and I see the request to matrix.org on a traffic capture.
[jaywink](https://matrix.to/#/@jaywink:federator.dev) discovered the alarming truth:
> the original PR mentions mantrixorg indeed: <https://github.com/n8n-io/n8n/pull/1024> .. sounds like PR time for someone :)
[Tulir](https://matrix.to/#/@tulir:maunium.net), like a coiled spring, provided a PR:
> well now it's a pull request <https://github.com/n8n-io/n8n/pull/1065>
...
> oh nice it got merged already
[Faelar](https://matrix.to/#/@Faelar:matrix.org) noticed:
> For those not following on github, n8n released a new version including the fix for homeserver
[Oleg](https://matrix.to/#/@oleg:fiksel.info), who brings us back to the start said:
> Just tested it.
>
> It works! 🥳
Open Source!
## Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by [pingbot](https://github.com/maubot/echo), a [maubot](https://github.com/maubot/maubot) that you can host on your own server. Join [#ping:maunium.net](https://matrix.to/#/#ping:maunium.net) to experience the fun live, and to find out how to add YOUR server to the game.
|Rank|Hostname|Median MS|
|:---:|:---:|:---:|
|1|imninja.net|466.5|
|2|midov.pl|531|
|3|mdpnd.ch|540|
|4|chatcloud.net|569|
|5|elcyb.org|801|
|6|fab.network|967|
|7|conduit.rs|1168|
|8|envs.net|1863.5|
|9|blob.cat|2660|
|10|aragon.sh|2743.5|
## That's all I know 🏁
See you next week, and be sure to stop by [#twim:matrix.org] with your updates!
[#TWIM:matrix.org]: <https://matrix.to/#/#TWIM:matrix.org>
[Open Tech Will Save Us #7]: https://www.youtube.com/watch?v=xJ2Tjc1Aj0A