Daemon - a small and powerful daemon supervisor, with (e)logind support, for babysitting our PipeWire's user target daemons - Testing Wanted

GazL · 03-03-2021, 03:44 AM

Quote:

Originally Posted by LuckyCyborg

For what the heck you think we invented a daemon supervisor with elogind integration?

To try and Heath Robinson a solution to something that shouldn't need fixing in the first place?

igadoter · 03-03-2021, 04:07 AM

What does the phrase Heath Robinson mean?
British. : doing something simple in a very complicated way that is not necessary : Rube Goldberg a Heath Robinson contraption/device. https://www.google.com/search?q=heat...b55nEA82Bh4ApM

drgibbon · 03-03-2021, 05:08 AM

Nice work, cool to see!

ZhaoLin1457 · 03-03-2021, 05:16 AM

Quote:

Originally Posted by 0XBF

Okay I got the xdg autostart working properly. Not sure what the issue was before but I rebuilt to the latest post and set up that conf file in /etc/daemon.conf.d and its starting. Side question: Is there any reason thats a better method than using the flags for bind, respawn, and pidfiles? (i.e. "/usr/bin/daemon -f -B -r --pidfiles=~/.daemon /usr/bin/pipewire")

Anyways, now I that its starting properly I have run into a second problem: pipewire-pulse. Are you guys running this at all? It is meant to run in place of a pulseaudio server and allows programs that expect pulse to still run. The recommended way to start pipewire-pulse AFAIK is via the /etc/pipewire/pipewire.conf file, in the last stanza:

Code:

exec = {
    ## <program-name> = { [args = "<arguments>"] }
    #
    # Execute the given program with arguments.
    #
    # Start the session manager. Run the session manager with -h for
    # options.
    #
    # The bluetooth module is disabled by default because it causes
    # conflicts with PulseAudio. If you disable PulseAudio or don't
    # load its bluetooth module, you can enable it here with -e bluez5
    #
    "/usr/bin/pipewire-media-session" = { args = "bluez5"}
    #
    # You can optionally start the pulseaudio-server here as well
    # but it better to start it as a systemd service.
    # It can be interesting to start another daemon here that listens
    # on another address with the -a option (eg. -a tcp:4713)
    #
    "/usr/bin/pipewire-pulse" = { "#args" = "-a tcp:4713" }
}

Which I have uncommented and it starts pipewire-pulse fine. However daemon misses this one when exiting. Prior to last logout I get:

Code:

$ ps aux | grep pipewire
bob      23364  0.0  0.0   4112  2680 ?        S    20:06   0:00 /usr/bin/daemon -f -n pipewire /usr/bin/pipewire
bob      23369  0.0  0.1 101600  9560 ?        S<l  20:06   0:00 pipewire: /usr/bin/pipewire
bob      23374  0.1  0.2 147404 16756 ?        S<l  20:06   0:00 /usr/bin/pipewire-media-session bluez5
bob      23375  0.0  0.0  81468  5472 ?        S<l  20:06   0:00 /usr/bin/pipewire-pulse
bob      23678  0.0  0.0   3996  2152 pts/1    S+   20:07   0:00 grep pipewire

Then logout and back in, I get:

Code:

bob      23364  0.0  0.0   4112  2680 ?        S    20:06   0:00 /usr/bin/daemon -f -n pipewire /usr/bin/pipewire
bob      23369  0.0  0.0      0     0 ?        Z<   20:06   0:00 [pipewire] <defunct>
bob      23375  0.0  0.0  81468  5472 ?        S<l  20:06   0:00 /usr/bin/pipewire-pulse
bob      23765  0.0  0.0   4004  2128 tty1     S+   20:08   0:00 grep pipewire

If the pipewire-pulse line is commented out in /etc/pipewire/pipewire.conf then there is no left over programs running after logout (i.e. pipewire and pipewire-media-session exit fine, just not pipewire-pulse).

I was handling this problem with killing pipewire programs with wrapper scripts till now, and have been using the line "pkill --parent $PIPEWIRE_PID" before killing pipewire to get around this problem. Not sure how best to proceed with daemon, perhaps as another autostart service?

According with the daemon developer, it is a PipeWire issue or maybe it have a frugal handling of child processes,

https://github.com/raforg/daemon/iss...ment-789578812

Quote:

Oh, dear. Zombies are never fun. It's very tempting to blame someone else's software when things go wrong like this, but I'm never confident that that's a valid response. Without being able to replicate and investigate the problem, it's hard to know the real cause. But I think I can make a good guess.

A zombie process happens when a process terminates, and its parent process doesn't wait() for it, so as to obtain its termination status. Pipewire has terminated, but daemon hasn't wait()ed for its status yet.

Daemon wait()s for its client's termination status after it has both received the SIGCHLD signal to let it know that a child process and died, and after that child process's stdout file descriptor has closed (daemon wants to make sure that it reads any final output from the client).

But it's possible for the client's stdout file descriptor to stay open after it terminates: i.e. when the client has started its own child process (e.g. pipewire-pulse) which shares its parent's process's stdout, and that other process is still running, and is still keeping stdout open.

Daemon is effectively waiting for pipewire-pulse to terminate as well. That will close the shared stdout. But pipewire hasn't terminated pipewire-pulse. It probably assumes that whatever sent it a SIGTERM will also send a SIGTERM to pipewire-pulse. But daemon won't do that because it doesn't know anything about pipewire-pulse. Daemon assumes that the client process will handle the termination of its own child processes when it receives a SIGTERM signal.

So, it looks like this happened:

- daemon started pipewire
- pipewire started pipewire-pulse
- the user logged out
- daemon sent pipewire SIGTERM
- pipewire didn't propagate SIGTERM to pipewire-pulse to terminate it
- pipewire-pulse and pipewire are sharing the same stdout which stayed open
- daemon is still waiting for the pipewire's stdout to close before it finishes up and wait()s for the status
- so pipewire-pulse stays around as a zombie

If pipewire and pipewire-pulse do share the same stdout file descriptor, that would prevent pipewire's stdout from closing (from daemon's point of view), and this would result. There are several approaches to fixing this:

1. pipewire, upon receiving SIGTERM, could send SIGTERM to pipewire-pulse
2. daemon could be started with --ignore-eof to not wait for eof on the client's stdout
3. pipewire-pulse could be started by daemon --bind rather than by pipewire (as suggested by 0XBF/LuckyCyborg)

Option 1 requires getting pipewire changed.

Option 2 should prevent pipewire becoming a zombie, and cause it and the daemon process to disappear from the process list, but it won't do anything to terminate pipewire-pulse. That's still running.

Option 3 is probably the quickest and easiest approach that should get both the pipewire and pipewire-pulse processes terminated cleanly.

Please ask 0XBF to try option 3 and let me know if it works.

So, looks like the proper solution is the @LuckyCyborg's idea of running the pipewire-pulse in a separate instance supervised directly by daemon.

ZhaoLin1457 · 03-03-2021, 07:30 AM

Quote:

Originally Posted by chris.willing

Is the xdg autostart mechanism OK for non-graphical logins? I guess the video side of pipewire implies there is a graphical login but if the daemon supervisor is supposed to be generic, then other non-graphical daemons need to be supported; hence a way to autostart the supervisor too.

chris

This daemon supervisor is of course generic, as it is just a console application with abilities to daemonize itself and the client program.

Probably when executing a pure console client, for an user target it could be launched from "/etc/profile" or "~/.profile" as the daemon go on background by default (we use the "foreground" option explicitly for the XDG autostart) and giving a name on command line, the daemon will handle an unique instance per user, no matter how many subsequent logins happens, then it can quit on user logout, if instructed.

Nothing stops, of course, to use it also on init scripts, and in its source code you will find even an script example for the init.

ZhaoLin1457 · 03-03-2021, 07:39 AM

Quote:

Originally Posted by LuckyCyborg

I've managed to configure the PipeWire and is PulseAudio compat audio server, by using a separate (autostart) services for them.

/etc/daemon.conf.d/pipewire.conf

Code:

pipewire-main  bind,respawn,pidfiles=~/.daemon
pipewire-pulse bind,respawn,pidfiles=~/.daemon

/etc/xdg/autostart/pipewire-main.desktop

Code:

[Desktop Entry]
Version=1.0
Name=PipeWire Media System
Comment=Start the PipeWire Media System
Exec=/usr/bin/daemon -f -n pipewire-main /usr/bin/pipewire
Terminal=false
Type=Application
X-GNOME-Autostart-Phase=Initialization
X-KDE-autostart-phase=1

/etc/xdg/autostart/pipewire-pulse.desktop

Code:

[Desktop Entry]
Version=1.0
Name=PipeWire Media System
Comment=Start the PipeWire Media System
Exec=/usr/bin/daemon -f -n pipewire-pulse /usr/bin/pipewire-pulse
Terminal=false
Type=Application
X-GNOME-Autostart-Phase=Initialization
X-KDE-autostart-phase=1

So, the PipeWire daemons works for me both as audio and video and no matter if I login and logout as root or unprivileged user, the result is always:

Code:

vanya@darkstar:~# ps xa | grep pipewire
 2978 ?        S      0:00 /usr/bin/daemon -f -n pipewire-pulse /usr/bin/pipewire-pulse
 2985 ?        Sl     0:00 pipewire-pulse: /usr/bin/pipewire-pulse
 2990 ?        S      0:00 /usr/bin/daemon -f -n pipewire-main /usr/bin/pipewire
 2992 ?        Sl     0:00 pipewire-main: /usr/bin/pipewire
 2995 ?        Sl     0:00 /usr/bin/pipewire-media-session bluez5
 3372 pts/7    S+     0:00 grep pipewire
vanya@darkstar:~#

BTW, I've disabled the PulseAudio globally on /etc/pulse/client.conf for "autospawn = no" and adding "Hidden=true" on /etc/xdg/autostart/pulseaudio.desktop

Well, I tried to setup also myself the PulseAudio compat server in the hope to make the Bluetooth audio to work, but even I followed your notes (as probably it's the best way) and physically to arrive to same processes handled, unfortunately on my MinisForum Z83-F the native PulseAudio works much, much better.

So, I chosen to use only the video part, aka the "main" in your design, for the Wayland/Plasma5.

Again, I truly believe that that happens not because you missed something, but looks like the PipeWire audio side has some issues on my mini-PC.

So, the moral of this story is that I still have a functional Bluetooth audio only on Windows 10.

No Bluetooth audio for me on Slackware.

0XBF · 03-03-2021, 08:08 AM

I set up an xdg autostart file to get daemon to start and supervise pipewire-pulse. That takes care of the problem and everything exits on logout properly. I guess that would be the closest solution to the pipewire dev's idea of "starting with systemd", as LC pointed out. I'll keep testing using daemon instead of my scripts to see if I run into any other problems.

ZhaoLin1457 · 03-03-2021, 08:20 AM

Quote:

Originally Posted by GazL

To try and Heath Robinson a solution to something that shouldn't need fixing in the first place?

There permit me to disagree...

1. Excluding this elogind integration and the associated features, this daemon supervisor is just a small program useful to daemonize various programs and to supervise them. Its main usage is probably on init scripts, where I am confident that it will do a brilliant job.

And there are many similar things, like daemontools or that "daemonize" which is on SBo.

https://slackbuilds.org/repository/1...tem/daemonize/

So, it is not overly complicated.

Honestly, a Heath Robinson would have been if we tried to add the elogind integration to daemontools.

2. There is an issue needing a fix on Slackware, as the PipeWire is used by Wayland/Plasma5.

That's WHY it was added to Slackware, as hard dependency of Wayland/Plasma5 which use it for various things.

However, the Slackware does not setup the PipeWire to run its daemons in a way or other. Nothing.

This results at least on functionality issues on Wayland/Plasma5, like the utterly visible lack of taskbar thumbnails.

On my mini-PC, the Wayland/Plasma5 works significant better than the X11 sessions, so for me to make the PipeWire daemons running is about a set of issues to fix on Plasma5 for Wayland.

True, there's another and simpler way: just to launch the PipeWire daemon via XDG autostart and to instruct elogind to kill the user processes on logout.

However, as you know, the elogind will kill also programs like screen or tmux, which will result in a way more complicated way of management for the people who will want both PipeWire running and the screen leaved alone.

0XBF · 03-03-2021, 08:30 AM

Quote:

Originally Posted by chris.willing

Is the xdg autostart mechanism OK for non-graphical logins? I guess the video side of pipewire implies there is a graphical login but if the daemon supervisor is supposed to be generic, then other non-graphical daemons need to be supported; hence a way to autostart the supervisor too.
chris

IMO this is the problem with the proposed xdg autostart set up; it doesn't cover the console use case. The main problem is that pipewire requires a session dbus at startup or it will exit. If you want to run pipewire for console audio you can use a dbus starting command, like:

Code:

dbus-run-session pipewire &

And if you have /etc/pipewire/pipewire.conf set to also start pipewire-pulse, then everything starts up and you can play audio in the console (just remember to kill it after).

However, now you are running an independent dbus-session and another one will get launched by xinit scripts, since they start with a "dbus-launch" wrapper. Things start to look messy. One solution that comes to mind is to start a dbus session at the very first login via a cue from elogind to keep everything under one user dbus, followed by pipewire and friends. However, now we are talking about starting multiple user targeted services at login, which is getting closer and closer to systemd imo.

Now back to testing. Hope I didnt de-rail things too much

GazL · 03-03-2021, 10:03 AM

Quote:

Originally Posted by ZhaoLin1457

There permit me to disagree...

I think you're misunderstanding me. I'm not complaining about your new daemon thingy. I'm lamenting the fact that pipewire is designed in such a way as to make it necessary. Anyway, I don't wish to derail the thread any further, so I'll leave it at that.

tramtrist · 03-03-2021, 12:18 PM

Quote:

Originally Posted by ZhaoLin1457

This results at least on functionality issues on Wayland/Plasma5, like the utterly visible lack of taskbar thumbnails.

On my mini-PC, the Wayland/Plasma5 works significant better than the X11 sessions, so for me to make the PipeWire daemons running is about a set of issues to fix on Plasma5 for Wayland.

This is very interesting point. I wonder if Pat is planning to fully integrate PipeWire to fix things like this.

bassmadrigal · 03-03-2021, 12:50 PM

Quote:

Originally Posted by tramtrist

This is very interesting point. I wonder if Pat is planning to fully integrate PipeWire to fix things like this.

I'd imagine he's watching this thread to see how well things progress and what ideas are proposed.

However, it's possible that he may not try and push anything major right now as Wayland is still relatively new and most users will probably use X11 for 15.0. But at some point, Wayland will likely take over as the dominate display server protocol, so there will need to be something in place to ensure Wayland based desktops function properly (including launching and ending pipewire on user login and logout, respectively).

I would guess Wayland was initially added to -current to allow this type of testing to occur without requiring users to recompile a bunch of core packages (like Plasma5) if they want to test it.

ZhaoLin1457 · 03-03-2021, 01:37 PM

Quote:

Originally Posted by bassmadrigal

I'd imagine he's watching this thread to see how well things progress and what ideas are proposed.

However, it's possible that he may not try and push anything major right now as Wayland is still relatively new and most users will probably use X11 for 15.0. But at some point, Wayland will likely take over as the dominate display server protocol, so there will need to be something in place to ensure Wayland based desktops function properly (including launching and ending pipewire on user login and logout, respectively).

I would guess Wayland was initially added to -current to allow this type of testing to occur without requiring users to recompile a bunch of core packages (like Plasma5) if they want to test it.

About what "anything major" you talk?

Did you know what is needed to fix the main PipeWire daemon running?

1. To add to Slackware this daemon supervisor - that's a 100KB package which contains a single binary of 200KB size, a man page and a config file. Yes, it's about adding of a very small program.
2. To put on the PipeWire package a XDG autostart file like the ones presented on this thread, at least as example. OK, there probably will be nice to have also an example for a pipewire-pulse autostart file.

That's all. There are no such major changes like the ones you talk.

However, while the PipeWire would probably be the first client of this daemon supervisor, this PipeWire is not the single "user target" daemon on the World. There are tons of them.

This daemon would be useful to run any of them.

The systemd is widely available, as it is used by all major distributions excluding us and partially Gentoo - they have the systemd as setup option.

For example, how about the dnscrypt-proxy mentioned on the first page of this thread?

This dnscrypt-proxy is also an "user target" daemon, just like PipeWire.

Or how about those proprietary VPN clients, which are almost all of them "user target" daemons?

In the wild are tons of "user target" daemons and this daemon supervisor with elogind support offers a way to run them.

bassmadrigal · 03-03-2021, 05:12 PM

Quote:

Originally Posted by ZhaoLin1457

About what "anything major" you talk?

Did you know what is needed to fix the main PipeWire daemon running?

1. To add to Slackware this daemon supervisor - that's a 100KB package which contains a single binary of 200KB size, a man page and a config file. Yes, it's about adding of a very small program.
2. To put on the PipeWire package a XDG autostart file like the ones presented on this thread, at least as example. OK, there probably will be nice to have also an example for a pipewire-pulse autostart file.

That's all. There are no such major changes like the ones you talk.

The program is small, but it introduces big changes on how people can use Slackware.

I'm not sure of the answer to this next question, but can the daemon supervisor only launch pipewire when started in a Wayland session or will it be started with any Plasma5 session (or is it started with any Wayland/X11 session)? If it is started in an X11/Plasma5 session, are there potential issues with it or does it "just work"?

The real question though, are pipewire and using daemon to launch it ready for primetime? Is it going to work for most or all cases? Look at when pulseaudio was added. By the time it was added to Slackware, it was seamless for *most* users. Is pipewire and daemon at that point? If not, can they get to that point before 15.0 is released? Considering how long pulseaudio stayed out of the OS, it might take some time for Pat to look at making pipewire primetime.

LuckyCyborg · 03-03-2021, 06:18 PM

Sorry, but you say some enormities that I can't stay shut-up...

Quote:

Originally Posted by bassmadrigal

The program is small, but it introduces big changes on how people can use Slackware.

Yeah, they would have a daemon capable to supervise programs! Fantastic! A breakthrough invention... 15 years ago.

BTW, you know which daemon supervisor is most famous in the world?

Hint: it was invented 10 years ago and many people hate it for the init part included in that particular software collection.

Quote:

Originally Posted by bassmadrigal

I'm not sure of the answer to this next question, but can the daemon supervisor only launch pipewire when started in a Wayland session or will it be started with any Plasma5 session (or is it started with any Wayland/X11 session)? If it is started in an X11/Plasma5 session, are there potential issues with it or does it "just work"?

You are kidding, right?

A daemon supervisor is just a daemon, a console application which capable to go on background, like are also Apache, MySQLd, FTPD and many others. It does NOT depend on a graphical environment.

So, the "daemon" would work the same for Wayland/Plasma5, X11/Plasma5, XFCE, CDE, WhateverDM and Linux console. It cares only to supervise its client program. You know, PID, input, output and error consoles, etc...

Like @ZhaoLin1457 said, we can make it even to supervise a PHP script on the init system.

Quote:

Originally Posted by bassmadrigal

The real question though, are pipewire and using daemon to launch it ready for primetime? Is it going to work for most or all cases?

You put equal between PipeWire and this daemon.

I do not know so many things about PipeWire, BUT according with the daemon's homepage, the first release of this daemon supervisor was on 1999, October 20.

It was NOT written yesterday by @ZhaoLin1457, but created by @raforg on 1999!

http://libslack.org/daemon/

BTW, those are the supported platforms today:

Code:

    Linux 3.x/4.x (i386, x86_64, debian-{7,8,9,10}, ubuntu-{14.04,16.04}, ...
    Linux 2.6 (i386, x86_64, debian-5.0.4, ubuntu-10.04, fedora-13)
    Solaris 10 10/09 (i386, amd64)
    OpenSolaris 2009/06 (i386, amd64)
    OpenBSD 4.7 (i386, amd64), 6.6 (amd64)
    FreeBSD 8.0 (i386, amd64), 12.1 (amd64)
    NetBSD 5.0.2 (i386, amd64), 8.1 (amd64)
    MacOSX 10.{4,5,6,11,14,15}, 11 (ppc, i386, x86_64, arm64)
    kFreeBSD 20090729 (i386)
    GNU/Hurd (i386)

What the heck you want more?

This daemon supervisor wasn't written yesterday.

Yesterday it got only the support for elogind integration, after @ZhaoLin1457 requested it and the daemon's developer agreed to add it.

And the systemd integration, as I understand that the author developed this on Debian.

So we have the guarantee that it respects the systemd API. Hundred percents.

Quote:

Originally Posted by bassmadrigal

Look at when pulseaudio was added. By the time it was added to Slackware, it was seamless for *most* users. Is pipewire and daemon at that point? If not, can they get to that point before 15.0 is released? Considering how long pulseaudio stayed out of the OS, it might take some time for Pat to look at making pipewire primetime.

WHY you talk about PipeWire and daemon?

The PipeWire is a fairly new invention, while the daemon has 11 years since its first release. So, I guess that this daemon is rock solid. At least since several days I use it, it worked like a charm.

However, also I used the PipeWire since Plasma5 was still on KTown, stopping it with elogind.

Yeah, looks like the PipeWire can run its video business, tagging along while PulseAudio runs its audio business.

And worked fine as video server. I can say that at least for me, it works better than PulseAudio, BUT I've just started to use it, so I cannot comment much.

BUT, like OP said, the subject of this thread is not about proposing and testing PipeWire (as it is already on Slackware -current), BUT about proposing and testing this daemon.

If you are afraid of PipeWire, feel free to test the daemon with dnscrypt-proxy or another user target daemon designed for systemd.

And please do not put an equal sign between PipeWire and this daemon supervisor. Because the PipeWire is only one of possible clients to be supervised by it.

From what I understand, this thread is about managing the systemd's user target daemons, as appropriate of systemd as possible, while having a minimal impact on Slackware.