matrix.org/content/blog/2021/11/2021-11-26-this-week-in-mat...

474 lines
24 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

+++
title = "This Week in Matrix 2021-11-26"
path = "/blog/2021/11/26/this-week-in-matrix-2021-11-26"
[taxonomies]
author = ["Thib"]
category = ["This Week in Matrix"]
[extra]
image = "https://matrix.org/blog/img/BRLaqyRUihVwexaSVGXnyMQK.png"
+++
### Matrix Live 🎙
{{ youtube_player(video_id="55P-NdDa-UI") }}
## Dept of Spec 📜
[anoa](https://matrix.to/#/@andrewm:amorgan.xyz) says
> Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
> ### MSC Status
>
> **New MSCs:**
> * [MSC3523: Timeboxed/ranged relations endpoint](https://github.com/matrix-org/matrix-doc/pull/3523)
> * [MSC3517: "Mention" Pushrule](https://github.com/matrix-org/matrix-doc/pull/3517)
> * [MSC3510: Let users kick/ban/demote each other](https://github.com/matrix-org/matrix-doc/pull/3510)
>
> **MSCs with proposed Final Comment Period:**
> * _No MSCs entered proposed FCP state this week._
>
> **MSCs in Final Comment Period:**
> * _No MSCs are in FCP._
>
> **Merged MSCs:**
> * [MSC2674: Event Relationships](https://github.com/matrix-org/matrix-doc/pull/2674)
>
> **Closed MSCs:**
> * [MSC3006: Bot Interactions](https://github.com/matrix-org/matrix-doc/pull/3006)
>
> ### Spec Updates
>
> [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) (serverside aggregations) is getting lots of updates from Bruno in order to align the proposal with what is currently implemented in the wild (as it's easier to iterate on incremental improvements from a starting point grounded in reality). This MSC is a bit of a special case though, as it was implemented with stable prefixes before the MSC landed (in the before times...).
>
> Regardless, thank you very much to Bruno for going through and finally untangling and help land aggregations in the spec! This MSC is one of four which describe how aggregations should work in Matrix, and it's great to see them finally being properly spec'd, especially as further features start to be built on top of them (such as [threading](https://github.com/matrix-org/matrix-doc/pull/3440/)!).
>
> And finally, a further thank you to community members [@ankur12-1610](https://github.com/ankur12-1610), [@Dominaezzz](https://github.com/Dominaezzz) for [an OpenAPI schema fix](https://github.com/matrix-org/matrix-doc/pull/3509)!
>
> ### Random Spec of the Week
>
> The random spec of the week is... [MSC3395: Synthetic appservice events](https://github.com/matrix-org/matrix-doc/pull/3395)!
>
> This proposal aims to allow appservices to get ever greater visibility into what is happening on the homeserver, while still maintaining full process separation.
>
> A lot of the time solutions to complex problems require knowing when a user has registered or logged in, with what name/3pids etc, when users change their emails, etc. Hooking into these non-room-based actions can help with developing useful features. Maybe you want an appservice that plays a sound effect whenever a user signs up to your homeserver!
>
> These days, that is often achieved by homeserver implementation-specific solutions, such as [modules](https://matrix-org.github.io/synapse/latest/modules/) in Synapse. Being able to notify of these events using a standard API shape would be hugely beneficial to generalisation of projects.
>
> So give the proposal a look over and review if that interests you!
> ![](/blog/img/db360c85bf5f0e1bef8eaf95481c1128ea51c61b.png)
Some exciting MSCs this week, I've been waiting for server-side aggregations for long!
## Dept of Servers 🏢
### Synapse [↗](https://github.com/matrix-org/synapse/)
Synapse is the reference homeserver for Matrix
[callahad](https://matrix.to/#/@callahad:matrix.org) announces
> As predicted last week, we [released Synapse 1.47.1](https://matrix.org/blog/2021/11/23/synapse-1-47-1-released) on Tuesday. This is a security release which fixes an issue with Synapse's built-in media repository. Admins are strongly encouraged to upgrade.
>
> Otherwise, relatively slow week: a handful of of the team have been away (Happy Thanksgiving, Americans!), but we *did* release [Synapse 1.48.0rc1](https://github.com/matrix-org/synapse/releases/tag/v1.48.0rc1). Most importantly, this release candidate includes changes to improve the efficiency of large background updates from past releases, which should significantly reduce database load when upgrading. It also adds support for the `/v3` APIs defined in version 1.1 of the Matrix specification.
>
> We'll talk more about 1.48 when it's formally released next week, but as always, we appreciate folks trying out the release candidates and letting us know how they behave.
Administrators, keep your users safe: update as soon as you can!
## Homeserver Deployment 📥️
### Helm Chart [↗](https://gitlab.com/ananace/charts)
Matrix Kubernetes applications packaged into helm charts
[Ananace](https://matrix.to/#/@ace:kittenface.studio) announces
> A bit of an earlier update this week, but I wanted to make sure to note that [my Hem Charts](https://gitlab.com/ananace/charts) have been updated to matrix-synapse 1.47.1 for the security fix - and element-web has also been bumped to 1.9.5
## Dept of Bridges 🌉
### mautrix-googlechat
[tulir](https://matrix.to/#/@tulir:maunium.net) reports
> [mautrix-googlechat](https://github.com/mautrix/googlechat) has seen lots of improvements over the past few days. New features include:
>
> * Bridging edits, deletions, reactions, formatting and read receipts in both directions (even `/rainbow` somewhat works from Matrix)
> * Bridging typing notifications and any types of files from Matrix to Google Chat
> * Bridging Google Meet links from Google Chat to Matrix
> * Syncing group members from Google Chat
>
> (edits and deletions are only available on Google Workspace accounts, not normal accounts. I have no idea why they did that, but that's just how Google Chat works 🤷)
>
> There's still a bug where it sometimes silently stops receiving messages, which I'm currently trying to solve (or work around). After that I'll make a v0.3.0 release. Backfilling history may also happen in the near future
### Matrix Webhook Receiver [↗](https://github.com/HarHarLinks/matrix-webhook-receiver)
An add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.
[kim](https://matrix.to/#/@kim:sosnowkadub.de) announces
> [Matrix Webhook Receiver](https://github.com/HarHarLinks/matrix-webhook-receiver)
>
> Do you like to receive notifications in matrix?
> Matrix Webhook Receiver (MWR) is an add-on for the [matrix-appservice-webhooks](https://github.com/turt2live/matrix-appservice-webhooks) bridge. [Webhooks](https://en.wikipedia.org/wiki/Webhook) are essentially web interfaces for applications to "push" data to.
> The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.
>
> This is where MWR comes in:
> It can receive any (JSON) content, optionally reformat it nicely (customizable!), and forward it to the webhooks bridge which will post it to a room for you. If you are running any software service, there is a good chance it can notify you via webhooks!
>
> Right now, several example configurations exist, ready for you to use:
> * GitHub
> * GitLab (including Community Edition/self hosted)
> * Ansible Tower/AWX
> * Grafana Alerts
>
> For example, here is the GitHub webhook as seen in [#matrix-webhook-receiver:matrix.org](https://matrix.to/#/#matrix-webhook-receiver:matrix.org):
> ![](/blog/img/webhook_receiver.png)
>
> More examples of apps currently in development: Prometheus Alertmanager, Jellyfin
>
> It is also easy to use to send messages from the commandline with standard tools (`curl`), e.g. for your cron jobs! See the README for an example.
>
> Some other webhook matrix things exist, but often require
> * admin access to server (appservices)
> * installing and running and maintaining a specific bot per service
> * create and set up accounts and passwords for each bot to use
>
> In contrast, MWR requires:
> * matrix-appservice-webhooks set up on your server by your admin
> * some knowledge of how to send POST requests for one time setup. I want to improve that, suggestions are welcome!
> * MWR can be installed and run by anyone who can access the bridge
> * one single MWR supports any amount of notifying apps!
> * you don't need admin access to the server to add more apps
> * no need to develop a bot/plugin to support new apps, just write a quick jinja template
> * multiple people can use a single MWR instance by sharing HTTP basic auth credentials
>
> Other features:
> * can post into encrypted rooms (see readme)
> * automatically generated API docs
> * message formatting presets including html markup or m.notice
> * arbitrarily customizable webhook URLs
> * list and manage currently installed webhooks in your browser ([screenshot](https://github.com/HarHarLinks/matrix-webhook-receiver/blob/main/examples/GUI_screenshot.png))
>
> Links:
> * [Repo](https://github.com/HarHarLinks/matrix-webhook-receiver)
> * [#matrix-webhook-receiver:matrix.org](https://matrix.to/#/#matrix-webhook-receiver:matrix.org)
What a massive update! At this rate I won't need to leave my Matrix client for anything!
### matrix-hookshot [↗](https://github.com/Half-Shot/matrix-hookshot)
A multi purpose multi platform bridge, formerly known as matrix-github
[Half-Shot](https://matrix.to/#/@Half-Shot:half-shot.uk) says
> Hey folks! Some exciting new news on the bridge front: I've renamed `matrix-github` to `matrix-hookshot` to better reflect it's not-just-GitHub-ness. That's not all though, as there are new features too:
>
> * The bridge now supports **Rust** as a companion language (we're aiming to rewrite critical sections in rust). Some parts of the formatting code have already been rewritten.
> * The bridge now supports JIRA (full puppeting!)
> * The bridge now supports generic webhooks too, with the ability to write **custom handling code** inside the state event to process these hooks into pretty messages.
> * Basic support for GitHub discussions.
>
> In the works:
>
> * A provisioning API to hook into integration managers
> * More GitLab support
> * Better GitHub discussions support
>
> We're not quite ready for a 0.2.0 release, but please check us out at https://github.com/Half-Shot/matrix-hookshot.
An interesting update, and Half-Shot even demoes it in today's Matrix Live!
## Dept of Clients 📱
### Nheko [↗](https://nheko-reborn.github.io)
Desktop client for Matrix using Qt and C++17.
[Nico](https://matrix.to/#/@deepbluev7:neko.dev) says
> After the 0.9.0 release last week, we have of course been busy fixing all the bugs different people reported. Messing around with the sticker pack editor and then leaving room should not make Nheko crash anymore. The problems where the flatpak has issues starting on Gnome systems are still under investigation. We thought we had a solution, but that seems to have broken other stuff! 💥
>
> Apart from that we have been doing some after release party cleanup. Apart from some refactorings, you can now filter your rooms on whether they are a direct chat or not in the sidebar. This is in addition to the filters we already had for favourites, spaces and your other personal tags. User colors should also now be much less biased towards blue and jdenticons should have more variance. Expect the next release to be a much more colorful experience!
>
> Speaking of colors, Twily made this awesome ZX Spectrum inspired logo after we changed our Gitlab bot to be more colorful! Check it out:
> ![](/blog/img/624b9e743f874ccf4431a5721a077d86e89d331e.png)
Gorgeous, I love it!
### Hydrogen [↗](https://github.com/vector-im/hydrogen-web)
Hydrogen is a lightweight matrix client with legacy and mobile browser support
[Bruno](https://matrix.to/#/@bwindels:matrix.org) says
> We're still distracted with SDK work and other things less visible for users, but this week we've also released [0.2.22](https://github.com/vector-im/hydrogen-web/releases/tag/v0.2.22) that fixes login on [Element One](https://one.element.io/) (and other servers using SSO login and not yet supporting the experimental dehydrated devices).
### Element [↗](https://element.io)
Everything related to Element but not strictly bound to a client
[kittykat](https://matrix.to/#/@kittykat:matrix.org) says
> **Threads**
> * On Web weve been working on Notifications and Badges, making sure no message goes unread.
> * On Mobile were building out the new Threads Panel so you can easily see all the Threads in a room.
>
> **Polls**
> * The Polls team is making great progress, focusing their efforts on creating and voting on all platforms.
> * User testing sessions are coming up!
>
> **Community Testing**
> * Bug squash session on Web on Tuesday 30 November at 17:00 UTC, join the call at [#element-community-testing:matrix.org](https://matrix.to/#/#element-community-testing:matrix.org)
> * Upcoming plans include testing sessions on Polls and IA.
### Element Web/Desktop [↗](https://github.com/vector-im/element-web)
Secure and independent communication, connected via Matrix. Come talk with us in [#element-web:matrix.org](https://matrix.to/#/#element-web:matrix.org)!
[kittykat](https://matrix.to/#/@kittykat:matrix.org) announces
> * Work continues on[ Information Architecture](https://github.com/vector-im/element-meta/issues/56): head over to Sidebar settings on develop.element.io to see what the team are currently working on.
### Element iOS [↗](https://github.com/vector-im/element-ios)
Secure and independent communication for iOS, connected via Matrix. Come talk with us in [#element-ios:matrix.org](https://matrix.to/#/#element-ios:matrix.org)!
[kittykat](https://matrix.to/#/@kittykat:matrix.org) announces
> * Were still working hard on replacing Matomo with PostHog to improve how we collect analytics data and making the MatrixKit obsolete.
> * Adding more features to Spaces and improvements to Spaces performance is [also happening](https://github.com/vector-im/element-ios/issues/5181).
### Element Android [↗](https://github.com/vector-im/element-android)
Secure and independent communication for Android, connected via Matrix. Come talk with us in [#element-android:matrix.org](https://matrix.to/#/#element-android:matrix.org)!
[kittykat](https://matrix.to/#/@kittykat:matrix.org) announces
> * Voice message drafts and other improvements to this feature are underway.
> * Were also looking to integrate PostHog on Android.
## Dept of SDKs and Frameworks 🧰
### simplematrixbotlib [↗](https://github.com/KrazyKirby99999/simple-matrix-bot-lib)
simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio.
[krazykirby99999](https://matrix.to/#/@krazykirby99999:matrix.org) says
> simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio. Version 2.4.0 provides several new features and a fix.
> ### New Features:
> * Newlines are now supported when sending markdown messages.
> * The msgtype of text and markdown messages can now be specified. Text and markdown messages can now optionally be sent as "m.notice" to avoid alerting everybody of the new message. The default msgtype will continue to be "m.text".
> ### New Fixes:
> * Fixed issue where the homeserver was hardcoded in an http request.
>
> Example usage is shown below:
> ```
> import simplematrixbotlib as botlib
>
> creds = botlib.Creds("https://home.server", "user", "pass")
> bot = botlib.Bot(creds)
> PREFIX = '!'
>
>
> @bot.listener.on_message_event
> async def echo(room, message):
> match = botlib.MessageMatch(room, message, bot, PREFIX)
>
> if match.is_not_from_this_bot() and match.prefix() and match.command(
> "echo"):
>
> response = " ".join(arg for arg in match.args())
> await bot.api.send_text_message(room.room_id, response, "m.notice") ## Uses the msgtype of m.notice instead of m.text
>
> bot.run()
> ```
>
> A thank you to HarHarLinks for their contributions to version 2.4.0!
>
> Request additional features [here](https://github.com/KrazyKirby99999/simple-matrix-bot-lib/issues/new?assignees=&labels=&template=feature_request.md&title=).
>
> [View source on Github](https://github.com/KrazyKirby99999/simple-matrix-bot-lib)
> [View package on PyPi](https://pypi.org/project/simplematrixbotlib/)
> [View docs on readthedocs.io](https://simple-matrix-bot-lib.readthedocs.io/en/latest/)
> https://matrix.to/#/#simplematrixbotlib:matrix.org
### jOlm [↗](https://github.com/brevilo/jolm)
Olm bindings for Java
[brevilo](https://matrix.to/#/@brevilo:matrix.org) reports
> This week saw three releases of [jOlm](https://github.com/brevilo/jolm) which fix a native memory management issue, an Olm API (buffer) issue and add a few other improvements. Everyone is strongly encouraged to update to the latest [release](https://search.maven.org/artifact/io.github.brevilo/jolm/1.0.4/jar).
>
> ### Notes:
> * ✅ Bugfix and maintenance releases
> * ✅ Up to date with Olm 3.2.6
>
> ### [Changelog](https://github.com/brevilo/jolm/compare/1.0.1...1.0.4):
> * Fixed the backing store retention for all Olm instances
> * Fixed a buffer issue in `InboundGroupSession.decrypt()`
> * Ensured conversions of variable native strings are trimmed
> * Centralized conversions to canonical JSON
> * Completed initial set of unit tests (effectively full coverage now)
>
> Cheers!
The only Java bindings of Olm to my knowledge, that's some very valuable work here!
## Dept of Ops 🛠
### synadm [↗](https://github.com/JOJ0/synadm)
Command line admin tool for Synapse (Matrix reference homeserver)
[jojo](https://matrix.to/#/@jojo:peek-a-boo.at) reports
> `synadm` v0.32 is out!
>
> My personal favorites of the new features are:
> * Dates and times in several subcommands are translate the admin API's UNIX epoch timestamps to a human readable format.
> * `synadm room list` now displays room aliases (#room:your.homeserver)
>
> Read the full release notes here: https://github.com/JOJ0/synadm/releases/tag/v0.32
synadm is very useful as a homeserver administrator. Thanks JOJ0
## Dept of Bots 🤖
### maubot [↗](https://github.com/maubot/maubot)
A plugin-based Matrix bot system.
[tulir](https://matrix.to/#/@tulir:maunium.net) reports
> maubot [v0.2.0](https://github.com/maubot/maubot/releases/tag/v0.2.0) was released last weekend. Highlights:
>
> * Enabling encryption should be much easier: the device ID can be entered in the web UI or you can just do `mbc auth --update-client` to automatically log in and store the access token and device ID in maubot.
> * `mbc auth` can now log in with SSO.
> * The standalone mode for running a single plugin with a static config is now mostly functional and [somewhat documented](https://docs.mau.fi/maubot/usage/standalone.html).
>
> Also, I finally took a day to figure out Sphinx/autodoc and made some decent-looking autogenerated [docs for mautrix-python](https://docs.mau.fi/python/latest/). I'll probably extend that to generate maubot-specific API references too eventually.
## Dept of Interesting Projects 🛰️
### MinesTRIX [↗](https://minestrix.henri2h.fr)
A privacy focused social media based on MATRIX
[Henri Carnot](https://matrix.to/#/@henri2h:carnot.cc) announces
> Quick update on [MinesTRIX](https://minestrix.henri2h.fr/) (a privacy focused social media based on MATRIX).
> This week was focused on performance and stability.
> * Changed database to use Fluffybox, this should greatly improve performances on web (thanks Famedly !)
> * Scrolling through the posts of a profile now properly request history.
> * Friend suggestions are now sorted according to the sum of user appearance in all rooms. Naïve, but it's the first step.
> * Chat page has also been redesigned. Now support replies and reactions. Chat settings now display room avatar and fetch user list from server.
> * Bug affecting MinesTRIX profile creation has been fixed. Login process should be way more stable now.
> * Minestrix rooms sync has been rewritten to take into account sync events to rebuild the list.
> * Debug page now allow forcing sorting rooms.
> * Various post display enhancement (links are now clickable, thanks kellya!)
>
> Come chat with us : [#minestrix:carnot.cc](https://matrix.to/#/#minestrix:carnot.cc)
> ![](/blog/img/BRLaqyRUihVwexaSVGXnyMQK.png)
That's one exciting project, I can't wait to see how far it's going!
### Sign in with Matrix
[Mish](https://matrix.to/#/@mishushakov:matrix.org) says
> Federated sign-in component for your web app (using Matrix)
>
> This week's update:
> * Gained 180 stars on GitHub since release (thanks!)
> * Added login states, accessible from the API
> * Sign out
> * Added CSS styling via variables
> * Updated demo
>
> more on https://github.com/mishushakov/signin-with-matrix
As last week, a note to keep in mind that this is a community project and that there is a MSC to make Matrix more OAuth2 friendly. More on that very soon!
## Dept of *Built on Matrix* 🏗️
### Matrix Forms
[Mish](https://matrix.to/#/@mishushakov:matrix.org) reports
> First release of "Matrix Forms", a project which redirects form submissions to designated Matrix rooms
>
> Features:
> * Server-side, no additional JavaScript
> * Rich formatting
> * Many forms on same instance
> * File uploads
> * Templates
> * CORS
> * Metadata accessible for bots
>
> Can be installed using NodeJS or Docker
>
> * demo: https://mishushakov.github.io/matrix-forms
> * code: https://github.com/mishushakov/matrix-forms
{{ youtube_player(video_id="R8HgMM1sp1M") }}
## Final Thoughts 💭
### Server_Stats
[MTRNord](https://matrix.to/#/@mtrnord:nordgedanken.dev) says
> Today is a good day for those calling me a spy, someone not wanting to care about privacy and for those who did publicly harass me for server_stats.
>
> People using the API likely already noticed it wasn't reachable for a while. Effective immediately I am currently leaving all rooms the bot is part of. This will take days or even months considering this are 6397 rooms at the time of writing. I am not going into the motives of why I am shutting it down. It comes down to personal reasons.
>
> There won't be any dump of the data. The source will be kept public. Note though if anyone ever tries to run it that you need about 600GB of space for synapse, a lot of CPU, a lot of RAM and plenty of workers as this can easily crash synapse.
Server_Stats was an incredibly useful project. It pains me a lot to see it go, but it pains me even further that its author got harassed. This is not an acceptable behaviour, and we are better than that as a community. Thanks for this incredible project MTRNord, it's been both exciting and useful.
## Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by [pingbot](https://github.com/maubot/echo), a [maubot](https://github.com/maubot/maubot) that you can host on your own server.
### [#ping:maunium.net](https://matrix.to/#/#ping:maunium.net)
Join [#ping:maunium.net](https://matrix.to/#/#ping:maunium.net) to experience the fun live, and to find out how to add YOUR server to the game.
|Rank|Hostname|Median MS|
|:---:|:---:|:---:|
|1|boba.best|413|
|2|kapsi.fi|508.5|
|3|maescool.be|528|
|4|envs.net|540.5|
|5|converser.eu|547|
|6|matrix.org|577|
|7|thesilentlink.org|838|
|8|matrix.markshorten.co.uk|1068|
|9|aria-net.org|1217|
|10|trygve.me|2137|
### [#ping-no-synapse:maunium.net](https://matrix.to/#/#ping-no-synapse:maunium.net)
Join [#ping-no-synapse:maunium.net](https://matrix.to/#/#ping-no-synapse:maunium.net) to experience the fun live, and to find out how to add YOUR server to the game.
|Rank|Hostname|Median MS|
|:---:|:---:|:---:|
|1|dendrite.neilalexander.dev|430.5|
|2|grin.hu|555.5|
|3|matrix.org|1027|
|4|dendrite.matrix.org|1056|
|5|matrix.awesomesheep48.me|1191|
|6|sspaeth.de|1251.5|
|7|0x1a8510f2.space|3350.5|
|8|spooks.cyou|10963.5|
### That's all I know 🏁
See you next week, and be sure to stop by [#twim:matrix.org](https://matrix.to/#/#twim:matrix.org) with your updates!