Zusam 0.4

As of today, the 0.4 version of Zusam is out !
Time to talk about the project, its goals and its roadmap.

What is Zusam and what it's not ?

If you're reading this, you probably already know what Zusam is. Let me reintroduce it from my point of view.
Zusam is a free and open-source solution to self-host private forums for you, your family and your friends. It's meant to be easy to install, to administer and to use. Its goal is also to be lightweight, stable and extensible.

It is not a synchronous chat platform: there is plenty of open-source projects that do a wonderful job at it.
It is not an open social network: the focus is on the group, not the individual. And everything is private by default.

What does that mean in practice ?

Since I want it to be extensible, I'm convinced that development should be focused.
I decided that the focus is on the web and that every feature should be added carefully. Let's take the time to make something that doesn't crumble under its own weight.
PHP is my main backend language but it's also a nice choice for stability, ubiquity and ease of installation.
That said, since the API is separated from the webapp, either could be reimplemented as soon as the API is specified.

The webapp is meant to be as lightweight as possible and with as little dependencies as possible.
That way, it's friendly to those who don't have a fast network. It also keeps compilation phase to a minimum, increases stability, ease of maintenance and lowers security issues from transient dependencies.

What was achieved in 0.4 ?

The 3 big visible achievements for the end user are search, subthreads and notifications.

Search

When there is a lot of messages in a group, it gets difficult to find an old one that has just a youtube link to a song you liked.
Maybe you want to search for all messages that have a certain keyword in it.
There are a lot of use cases for search and it was demanded by the users of my instance and on the linuxfr article of Zusam.

Search is a very complicated topic with multiple aspects: user interface, speed and accuracy.
I've done a first pass at it. It's nowhere final and will be improved upon following the next releases. But I'm happy that it works reasonably well for the simplicity of what was implemented.
For now, it accepts a list of keywords and search for occurences of them in the different areas of a message: title, author, text and links descriptions.
I want the implementation to stay as simple and straightforward as possible. The user should be able to know which results will come up: it shouldn't feel like magic.
A simple implementation allows it to be fast enough (less than 2s) for even the biggest groups that I have (more than 5k messages).

Improvements will probably happen on the user interface: I'm a believer that the major part in search is done when the search results are well presented. I'm not happy with the current one but it will be improved in the future.

Subthreads

This was another big request and something that I thought about for a while.
Subthreading was made difficult because it should:

Finally, I made the choice to handle it similarly to Slack. You can see the corresponding issue here.
The user interface could probably be improved, especially around message discovery. A big part is done by the new notification system.

Notifications

The new notification system centralizes news all around the different groups in one place.
this one was a long time coming but revealed itself necessary with the new subthreading feature. It was impossible to follow multiple subthreads in multiple groups without getting lost or missing some messages.
All new events (not only messages but users joining/leaving groups or groups name changes also) will be reported under the bell.

Miscellaneous

A lot of not visible things have been done to improve the core of Zusam.

Api-platform removal

Api-platform is a formidable tool to rapidly create functional and expressive APIs. I used it for Zusam and in other projects of mine.
So why did I decide to ditch it from Zusam ?
I want Zusam's code to be the easiest possible for others to apprehend. Api-platform was a not needed abstraction level given how simple the API is. It also was a bit slower than simply writing the controllers directly for Symfony and would often be in the way: I know how to handle Symfony but for complicated stuff, Api-platform was not as straightforward for me.
I took a day to rewrite all controllers and removed this dependency from the project.

Subpath handling

An important thing for me is to create a package of Zusam for Yunohost. Yunohost allows an app to be installed on a subdomain and/or a subpath. I wanted it to be the case for Zusam. After all, this will allow easier installations in a lot of situations and lower the technical level needed to have its own Zusam.
This only needed changes on the webapp and to find the correct configuration for Nginx. The process was long and not rewarding but I'm happy it's done.

Roadmap

Here are sme thoughts about the future of Zusam.
I'll force myself to use milestones/issues on github so that the process is more transparent in the future.

Goals for 0.4.x ?

I need to give some love to the webapp and consolidate what was already implemented.
There is a lot of small to medium issues that I want to tackle. Big features that need database changes can wait for now.
Here are some things that I want to address:

This will already take some time.
I will also focus on easying development if some want to help. For this I need to:

I'm also thinking about proposing some scripts to externally use the Zusam API and give some examples of what can be done.
If I continue alone on all this, it should take at least multiple months, probably the whole year.

Goals for 0.5 ?

The next big release will be 0.5. This will allow some changes in the database.

The first thing that comes to my mind is adding tags. This will tremendously improve the search if correctly used.
It will also enable users to structure their messages and make them escape the organic mess that is the state of current highly active groups.
In a certain way, this will improve the value of having a message history.

Another feature that would be awesome is the possibility to directly comment a picture in a photo album. It's pretty straightforward on the database side: just add a children attribute for files pointing at messages. But on the user interface side, there is a lot to think about. Depending on how the subthreading evolves, it could be a source of inspiration.

A revamp of the notifications will probably happen. I already see the limits of it in case of highly active groups. It's probably necessary to keep old notifications for later reference and organize them. A lot of thinking about the UI needs to be done.

Finally, I also want to enable rich messages alternating between text and medias. This is not possible with the current architecture (front and backend). Is this something that is really needed ? What is the best way to implement it ? Big questions.

Starting the management of extensions for the webapp and/or the API will also be worked on for this version.

0.5 will probably not be released before 2021.

What are the long term goals, for 1.0 ?

I don't want any database change without changing the major version number. So, for 1.0, the database structure should be stable.
This will also ease up another goal: to make it compatible with MySQL, MariaDB and PostGreSQL. I feel that this is important for environments that need more performance or that have high availability constraints. It's also nice for those who want to operate under these databases.

It's also very important that a group can be exported/imported for 1.0. I want users to be able to change server easily if the one they're on doesn't meet their wantings.

Another major goal is to allow à-la-wordpress installation. This means that Zusam should be able to operate on a shared web hosting platform.
This is complicated due to the dependency to ffmpeg. FFI and/or packaging ffmpeg with Zusam are ideas, but it's also possible to detect when ffmpeg is not there and let the API signal that video upload/conversion is not possible.
To be able to install Zusam as Wordpress is very important: it means that almost everyone can have his own Zusam, even without technical knowledge.

Final words

Thanks for your time.
I hope that this blog post gives you a better view of what was done with the project and in what direction I want to steer it.
Zusam is here to stay.