matrix.org/content/blog/2020/07/2020-07-10-this-week-in-mat...

387 lines
23 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

+++
title = "This Week in Matrix 2020-07-10"
path = "/blog/2020/07/10/this-week-in-matrix-2020-07-10"
[taxonomies]
author = ["Ben Parsons"]
category = ["This Week in Matrix"]
[extra]
image = "https://matrix.org/blog/img/2020-07-10-qiwpr-Screenshot2020-07-10at16.02.39.png"
+++
## Matrix Live 🎙
This week it's Open Tech Will Save Us!
{{ youtube_player(video_id="_2DQ_iYZi5U") }}
* **Jay Graber** ([@arcalinea](https://twitter.com/arcalinea), [@arcalinea:matrix.org](https://matrix.to/#/@arcalinea:matrix.org)), creator of [Happening](https://happening.net/) and [all-around decentralisation fan](https://medium.com/decentralized-web/decentralized-social-networks-e5a7a2603f53) gave an overview of the decentralized social media ecosystem.
* **Ania M. Piotrowska** ([@aniampiotrowska](https://twitter.com/aniampiotrowska)), known for her work on [Nym](https://nymtech.net/), Loopix, and at UCL presented "Building private future for the internet with the Nym mixnet".
* **Burak Nehbit** ([@nehbit](https://twitter.com/nehbit), [@burak:tomesh.net](https://matrix.to/#/@burak:tomesh.net)) gave a deep dive into his work on [Aether P2P](https://getaether.net/).
## Dept of Spec 📜
[anoa](https://matrix.to/#/@andrewm:amorgan.xyz) reported:
> Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at <https://matrix.org/docs/spec/proposals.>
>
>
> #### MSC Status
>
> **Merged MSCs:**
>
> * *No MSCs were merged this week.*
>
> **MSCs in Final Comment Period:**
>
> * [MSC2630: checking public keys in SAS verification](https://github.com/matrix-org/matrix-doc/pull/2630) (merge)
>
> **New MSCs:**
>
> * [MSC2677: Annotations and reactions](https://github.com/matrix-org/matrix-doc/pull/2677)
> * [MSC2676: Message editing](https://github.com/matrix-org/matrix-doc/pull/2676)
>
> * [MSC2675: Serverside aggregations of message relationships](https://github.com/matrix-org/matrix-doc/pull/2675)
> * [MSC2674: Event Relationships](https://github.com/matrix-org/matrix-doc/pull/2674)
>
> * [MSC2673: Notification Levels](https://github.com/matrix-org/matrix-doc/pull/2673)
> * [MSC2666: Get rooms in common with another user](https://github.com/matrix-org/matrix-doc/pull/2666)
>
> Note that a majority of those new MSCs are split out from [MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849) (the aggregations MSC) to make it easier to review 🙂
>
> #### Spec Core Team
>
> In terms of Spec Core Team MSC focus for this week, we're sticking to implementation work. anoa did have some time this week to make the graphs more useful though:
![2020-07-10-FFL8i-image.png](/blog/img/2020-07-10-FFL8i-image.png)
Check out that graph!
## Dept of GSoC 🎓️
### HTML embeddable client GSoC project
[arnav-t](https://matrix.to/#/@arnav-t:matrix.org) told us:
> This week for my HTML embeddable client GSoC project:
>
> * Added full markdown support and made replying and quoting fully functional
>
> * Added a lot more configuration options to the client, including a read-only mode
> * Fixed some bugs
### go-neb
[nikofil](https://matrix.to/#/@nikofil:matrix.org) said:
> * work is still being done on the bot's functionality to test different crypto functionalities of other Matrix clients when thrown in a room with them
>
> * slight improvements to mautrix-go (sql store can now store multiple Olm accounts and their sessions, key rotation params taken from encryption event, other minor things)
> * add some simple instructions to the readme for enabling e2ee for neb as well as integrate these mautrix changes (PR #330)
>
> * `room_key_request`s will probably be the next thing to be developed for mautrix-go
### Nheko (GSoC)
[Nheko](https://github.com/Nheko-Reborn/nheko) is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
[Chethan](https://matrix.to/#/@crdy2k1:matrix.org) said:
> * Added user cache which stores the encrypted user keys and updates them on sync
>
> * Added Verification Cache which stores verified devices of a user
> * Showing Verification Status of a device on Userprofile
## Dept of P2P 👥
### Riot iOS P2P Demo
[Neil Alexander](https://matrix.to/#/@neilalexander:matrix.org) announced:
> Over the last couple of weeks I have been working on the next P2P demo, using [Yggdrasil](https://yggdrasil-network.github.io/) as the transport instead of [libp2p](https://libp2p.io/). Although libp2p is arguably more featureful in many ways, Yggdrasil does have one major benefit in that it provides full overlay routing. Participants in the network can carry traffic on behalf of other participants, resulting in something much closer to a true mesh network.
>
> The mad science doesn't end there however. I've also built a custom Riot iOS build that includes the full Dendrite P2P demo (yes, that's a homeserver running locally on your phone). If you have an iPhone or iPad, it's available in [public TestFlight right now](https://testflight.apple.com/join/Tgh2MEk6) and you can play with it!
>
> The demo still is very experimental and has a number of bugs still (including but not limited to messages occasionally taking a while to deliver, the app crashing when going into the background sometimes etc.) but we'll be improving it further in the coming weeks and it's quite fun to play with, particularly if you can do so with other people nearby.
>
> Thanks to the same technology that powers AirDrop, the demo will automatically find and connect to other nearby devices running the demo and build up a network automatically - even if you are not on the same Wi-Fi network (or indeed connected to Wi-Fi at all)! If you don't have any other nearby peers to test with then fear not - you can still configure a public static peer in the app settings and join the wider network that way.
>
> With any luck the room directory will even work too, showing published rooms from other nearby devices. If not, try joining `#beachparty:b5ae50589e50991dd9dd7d59c5c5f7a4521e8da5b603b7f57076272abc58b374` from within the app.
>
> Questions, comments or feedback? Join us in `#p2p:matrix.org`!
## Dept of Servers 🏢
### A new way to test homeservers - [Complement](https://github.com/matrix-org/complement)
[kegan](https://matrix.to/#/@kegan:matrix.org) reported:
> Many of you know that I work on a Go homeserver called **Dendrite**, and so I've had to get familiar with **SyTest**: a black-box homeserver integration testing project. Unfortunately SyTest has a number of problems: from the dialect of perl, lack of documentation for federation bits, the inability to run a single test, and so on. Having a solid black-box integration testing project is crucial for the ecosystem to ensure spec compliance (and hence compatibility between server implementations) and for making all servers more stable and reliable.
>
> As a side project I've been working on a modern Go rewrite called **Complement** which is now ready for a bit more exposure. It currently only has a handful of SyTests converted but I hope to rapidly expand the number in the coming weeks. Complement makes heavy use of Docker to agnostically run homeservers, and already includes a Dockerfile for configuring a Dendrite instance. To try it out, clone the repo and run:
>
> ```
> (cd dockerfiles && docker build -t complement-dendrite -f Dendrite.Dockerfile .)
>
> COMPLEMENT_BASE_IMAGE=complement-dendrite:latest go test -v ./tests
> ```
>
> If you're interested in learning more and maybe adding some tests check out [ONBOARDING.md](https://github.com/matrix-org/complement/blob/master/ONBOARDING.md).
>
> My overall hope is that Complement will lower the barrier to entry when adding tests by so much that it encourages any bugfix/feature in any homeserver implementation to result in a new test. This will benefit everyone and create a feedback loop which will make Matrix even more reliable.
Asked about Sytest and SyTS, Kegan added:
> Complement currently implements the same tests as sytest, so tests which pass sytest will pass complement (though that will diverge as complement will guard against more race conditions by default). There's about 9 different kinds of tests currently testing CS-API and Federation (outbound to a dummy server and inbound via a federation client)
>
> The problem with SyTS was that the test tooling around it ultimately wasn't expressive enough. I was using Jest and kept finding myself fighting it by bumping up against issues like <https://github.com/facebook/jest/pull/8751> and the multi-process parallelisation stuff was too opaque. The assertion helper functions also lacked context because they rely on behavioural testing names like `describe("foo") { it("should say bar") { ... }` etc so when your assertion fails (eg wrong value for a JSON key using assert equals) it just says unhelpfully "foo != bar" where what I really want is to know what it actually does say then, and other contextual info around the object (maybe the key name was typod). When I realised that the assertion lib didn't have this and I'd need to add it, coupled with parallelisation concerns and head-desking against Jest, I found myself wishing I just used something else. I stuck with it though and then realised that all the federation stuff (signature checking, canonical JSON, etc) would need to re-implemented when I knew I had working code in Go. Both of these things combined and I thought "you know what, I'm just going to do this in Go" but you'll note that the architecture is identical, so SyTS lives on as an early prototype for Complement ;)
Find out more in the room: [#complement:matrix.org](https://matrix.to/#/#complement:matrix.org).
### Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
[Neil Alexander](https://matrix.to/#/@neilalexander:matrix.org) told us:
> While I've been busy distracted with the P2P demo, Kegan has been on a mission to implement some new features. Changes this week include:
>
> * Redaction is now mostly implemented
>
> * User-interactive authentication is now implemented
> * Device lists in the client API are now implemented
>
> * Media APIs are now available on both `/r0` and `/v1` endpoints which makes Riot iOS a bit happier
> * Some federation sender bugs have been fixed
>
> * Some database locks in the federation sender in SQLite mode have been fixed
>
> Spec compliance has improved a bit:
>
> * Client-Server APIs: 48%, up from 45% last week
>
> * Server-Server APIs: 51%, up from 50% last week
### Conduit
Conduit is a Matrix homeserver written in Rust <https://conduit.rs>
[timo](https://matrix.to/#/@timo:koesters.xyz) said:
> Conduit
>
> Hi everyone, here are some things we worked on this week:
>
> * Work on to-device sending improvements
>
> * Work on room tags (thanks to @gnieto)
> * Merged /logout/all endpoint (thanks to @CapsizeGlimmer)
> * Work on /joined_members (thanks to @CapsizeGlimmer)
>
> Last week we had a list of major features still missing. Here's a list of what already works:
>
> Registering, logging in, creating rooms, room visibility, join rules, basic permission management, public room list, inviting, creating DMs, e2e encryption, key backups, device verification, cross signing, notifications, uploading media/files (also user/room avatars), lower-resolution media thumbnails, voip calls and a few other thing's I'm forgetting right now.
> So it's useable for non-federating chats already.
>
> Thanks to everyone who supports me on [Liberapay](https://liberapay.com/timokoesters) or Bitcoin!
### Synapse
[Neil](https://matrix.to/#/@neilj:matrix.org) reported:
> In Synapse land this week, we shipped [1.16.0](https://matrix.org/blog/2020/07/08/synapse-1-16-0-released) and err [1.16.1](https://github.com/matrix-org/synapse/releases/tag/v1.16.1).
>
> Lifting from the blog post, the highlights are:-
>
> * An important performance fix to improve room state resolution.
>
> * An option to enable e2e by default for new rooms.
> * Ability to run multiple media repo workers side by side.
>
> * Ability to mark specific content as being safe from quarantine.
> * Bug fixes to make migrating from SQLite to Postgres more reliable - if you are running sqlite for anything other than evaluation purposes then please migrate!
>
> We also put out a [release candidate](https://github.com/matrix-org/synapse/releases/tag/v1.17.0rc1) for 1.17.0 which all being well well release on Monday. 1.17.0 is really a bug fix release the most notable being finally squashing a long standing bug that caused [locally rejected invites to get stuck client side](https://github.com/matrix-org/synapse/issues/2181).
>
> Continuing our matrix.org performance theme, having now got the CS API largely into a good state of responsiveness, we are looking at federation lag. Today we shipped a sharded Federation Reader to matrix.org which has reduced the average lag from seconds to milliseconds. We are just running two currently and are still tuning as we figure out how to get the best from it. Initial impressions seem promising. Watch this space.
![2020-07-10-qiwpr-Screenshot2020-07-10at16.02.39.png](/blog/img/2020-07-10-qiwpr-Screenshot2020-07-10at16.02.39.png)
> We also have a [PR out for review](https://github.com/matrix-org/synapse/pull/7798) to shard the Federation Sender, which will have much the same effect in the other direction.
>
> Once we have sending messages via the Client-Server API, much reduced federation lag and a sharded Pusher well take a look at room joins.
## Synapse Deployment 📥️
### dacruz21/matrix-chart
[Typo Kign](https://matrix.to/#/@david:typokign.com) reported:
> v2.3.0 of [dacruz21/matrix-chart](https://hub.helm.sh/charts/dacruz21/matrix) has been published with Synapse 1.16.
### Kubernetes
> Just pushed the 1.16 K8s-optimized Synapse image tags, this time done from a tablet in a server room between moving a bunch of hardware.
### New WireGuard-based deployment
[balaa](https://matrix.to/#/@balaa:pi2.noteworthy.im) announced:
> Hi everyone, my friend Patrick and I have been working on making Matrix more accessible in the context of personal overlay networks powered by WireGuard. Weve built a 1-click deployment solution for Synapse & Riot based on docker. The interesting part is that it gives you public addressability via a distributed proxy service that we have been developing. We are free and open source and welcoming contributors and testers ASAP! We are imagining an ecosystem for building collaborative intranets and see them as foundational to a freer, more equitable internet. Please join us at [#noteworthy:tincan.community](https://matrix.to/#/#noteworthy:tincan.community) See <https://www.patrickdlg.com/personal-messaging/> and <https://noteworthy.tech/start/>
## Dept of Bridges 🌉
### mautrix-whatsapp
[Tulir](https://matrix.to/#/@tulir:maunium.net) announced:
> mautrix-whatsapp got some bugfixes and improvements:
>
> * You can now create private chat portals by inviting a WhatsApp ghost user (e.g. from a group chat)
>
> * WhatsApp users in groups are now synced to Matrix properly, including kicking users who left the group without the bridge noticing
### matrix-sms-bridge
[Benedict](https://matrix.to/#/@benedict:imbitbu.de) offered:
> I fixed some bugs in [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) and added a feature that allows delayed sending of sms messages with the `sms send` command. It runs very fast and stable in my production environment.
## Dept of Clients 📱
### Nheko
[Nheko](https://github.com/Nheko-Reborn/nheko) is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
[Nico (@deepbluev7:neko.dev)](https://matrix.to/#/@deepbluev7:neko.dev) reported:
> Not much to talk about currently, since I'm breaking my Nheko to replace the entire event store at this moment, but I have some heads up on an annoying bug that I fixed on my branch:
>
> Sometimes we had a weird issue, where Nheko wouldn't load the language the user set on their system. This was especially annoying when I wanted to test a specific language, like Japanese, and I couldn't figure out the right LANG* variables to set the language to Japanese in Nheko. Turns out, this was a bug in Nheko and you wouldn't have guessed so, when looking at the examples in the Qt documentation. Anyway, KDAB wrote a nice blogpost explaining this issue: <https://www.kdab.com/fixing-a-common-antipattern-when-loading-translations-in-qt/>
>
> If you are a Qt developer, you may want to check your applications, if you are also affected.
### Riot-iOS
[steve](https://matrix.to/#/@stevio:matrix.org) offered:
> This week, we worked on the rebranding.
### RiotX / Riot-Android
[benoit](https://matrix.to/#/@benoit.marty:matrix.org) announced:
> We will release a beta (0.91.5) of Riot-Android this evening, including all the recent changes and lots of fixes. We are still working to prepare the great release!
### Daydream
[MTRNord](https://matrix.to/#/@mtrnord:nordgedanken.dev) told us:
> While not much is happening on the master branch some things happened on the Redesign branch:
>
> * Daydream does the sync now on a worker which is similar to using a thread
>
> * jplatte made a PR for optimizing multiple Parts of the Code which significantly improved the overall Daydream performance ( <https://github.com/daydream-mx/Daydream/pull/22> )
> * Some rework of the Login logic happened to prepare for well-known support
## Dept of SDKs and Frameworks 🧰
### mautrix-python
[Tulir](https://matrix.to/#/@tulir:maunium.net) offered:
> When I initially implemented end-to-bridge encryption in mautrix-python, I used matrix-nio in a slightly hacky way to get it done quickly. It worked fine in some cases, but also caused some undecryptable messages. I tried switching to using matrix-nio's crypto module directly, but the sans-I/O design made that difficult: there were too many parts that needed to be hooked up to the actual I/O and I wasn't able to find them all. In the end, I just decided to implement the basic e2ee stuff directly in mautrix-python the same way it's implemented in mautrix-go.
>
> All my bridges based on mautrix-python (Telegram, Facebook and Hangouts) have been switched to use the new crypto stuff. After a few initial bugs that are already fixed, it seems to work better than the old system. I'll probably add native e2ee to maubot soon too, which is my last main project that's still missing e2ee.
## Dept of Internet of Things 💡
### Mozilla IoT Matrix Adapter 0.3
[Christian](https://matrix.to/#/@christianp:vector.modular.im) reported:
> The [Mozilla IoT Matrix Adapter](https://gitlab.com/webthings/matrix-adapter) can send posts based on events in your home.
>
> Version 0.3 brings the ability to use room aliases instead of internal room ids.
> Furthermore, you can configure it to accept invites and follow room upgrades.
Due to utter negligence and malpractice by myself, we missed the update for this project last week. Of course, 0.2 is now superseded by 0.3, but here is the 0.2 update anyway:
> Released [v0.2.0 of the Mozilla IoT Matrix adapter](https://gitlab.com/webthings/matrix-adapter/-/releases/v0.2.0). It's now possible to post in multiple rooms (still one per post).
## Dept of Bots 🤖
### Cat Disruptor 7000
[Stephen D](https://matrix.to/#/@stephen:m.scd31.com) told us:
> I wrote a pretty simple Matrix bot in Rust which is loosely based on Cat Disruptor 6000. It will react with 🐈️ to any message containing the string "cat" \*. Since it is a small project, I hope it can be used as an example for other people interested in writing Matrix bots in Rust. It implements several important features (crypto/device store, auto-accepting invites, crafting custom events, etc.) You can host it yourself, or you can try it out by inviting the user "@catdisruptor:m.scd31.com" to your room! <https://git.scd31.com/stephen/cat-disruptor-7000>
>
> \* Cat Disruptor 6000 does not do this (it is used for disrupting monologues with cat pictures). However, many instances of Cat Disruptor 6000 also include a separate bot, which is what Cat Disruptor 7000 is mimicking.
### matrix-imposter-bot
[mr_johnson22](https://matrix.to/#/@mr_johnson22:matrix.org) said:
> **matrix-imposter-bot** - _A bot that uses your account to repeat other people's messages. This gives relay-bot capabilities to puppet-only bridges._
>
> Updates:
>
> * Can be much more easily deployed with a production-level WSGI server. It comes with [waitress](https://docs.pylonsproject.org/projects/waitress/en/stable/index.html) but it's possible to use any other server.
>
> * Member join/leave/rename now post messages
> * Shuts down when receiving a term/exit/quit signal, instead of hanging
>
> For more details, see <https://github.com/mrjohnson22/matrix-imposter-bot>
## Dept of Interesting Projects 🛰️
### cody
[carl](https://matrix.to/#/@carl:bordum.dk) offered:
> cody is a REPL for your matrix chat rooms.
>
> This week, the big new feature was support for Ruby with the message prefix `!rb`.
>
> Chat with cody: @cody:bordum.dk
>
> Read the source: <https://gitlab.com/carlbordum/matrix-cody>
## Dept of Jobs 💰️
### Famedly is recruiting
[sorunome](https://matrix.to/#/@sorunome:sorunome.de) announced:
> Heya, we ([famedly](https://famedly.com)) are looking for about 2-3 more flutter developers. We are a german startup which works on revolutionizing communication in the medical area. We build on top of matrix, so having some matrix-knowledge would be very beneficial.
>
> You might have seen us previously in TWIM with projects such as [famedly-email-bridge](https://gitlab.com/famedly/bridges/famedly-email-bridge) or the [famedly dart SDK](https://github.com/famedly/matrix-dart-sdk/).
>
> While we are based in Berlin we do allow remote work, and communication in english is fine, too.
>
> If you are interested or have more questions, please message [Niklas Zender](https://matrix.to/#/@larodar:famedly.de).
## Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by [pingbot](https://github.com/maubot/echo), a [maubot](https://github.com/maubot/maubot) that you can host on your own server. Join [#ping:maunium.net](https://matrix.to/#/#ping:maunium.net) to experience the fun live, and to find out how to add YOUR server to the game.
|Rank|Hostname|Median MS|
|:---:|:---:|:---:|
|1|fairydust.space|334|
|2|matrix.vgorcum.com|637.5|
|3|lossy.network|657|
|4|asra.gr|795|
|5|swag.industries|1407.5|
|6|lo.hn|1481|
|7|finallycoffee.eu|1804|
|8|nzbr.de|1885.5|
|9|halogen.city|2275|
|10|utzutzutz.net|2462.5|
## That's all I know 🏁
See you next week, and be sure to stop by [#twim:matrix.org] with your updates!
[#TWIM:matrix.org]: <https://matrix.to/#/#TWIM:matrix.org>