LinuxQuestions.org - [SOLVED] Can't detach from screen session which runs java application

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Can't detach from screen session which runs java application (https://www.linuxquestions.org/questions/programming-9/cant-detach-from-screen-session-which-runs-java-application-4175449692/)

Can't detach from screen session which runs java application

Hi there,

I am trying to run a long-running java command line application on a remote host through ssh and screen to be able to disconnect and reconnect later.
The application provides the option to open a gui (in a JFrame) that shows current status and reclose it without actually exiting the application.
However, if I do this, I cannot detach from the screen session anymore. More precisely, I can detach using ctrl-a-d, but when I then type exit to close the ssh session, it will say "Disconnected", but not give me the prompt of the local host back.

Code:

user@remotehost:~$ screen

[detached from 27685.pts-0.remotehost]

user@remotehost:~$ exit

Abgemeldet

If I close the terminal window the application on the remote host is aborted, which is the key problem I need to fix. I have to be able to shut down the local host completely without killing the application running on the remote host.

I have been able to reproduce the issue with a stripped down java program. The source and additional info on OS etc... can be found below, I am running it as a jar on the remote host.

Any ideas on what to do to fix this would be great.

Code:

import java.awt.event.*;

import javax.swing.*;



public class Main 

extends WindowAdapter

implements Runnable

{

        public Main() {

        }



        public static void main(String[] args) 

        {

                // opens the JFrame in different thread. Commenting out the following two lines stops the problem from reproducing

                Thread t = new Thread(new Main()) ;

                t.start();

                while(true)

                {

                        // in the actual program here is where stuff happens

                        Thread.yield();

                }

        }



        public void windowClosing(WindowEvent event)

        {

                // closes the window without exiting the application

                event.getWindow().setVisible(false);

                event.getWindow().dispose();

        }

        

        public void run()

        {

                JFrame f = new JFrame();

                f.addWindowListener(this);

                f.setLocation(100, 100);

                f.setSize(100, 100);

                f.setVisible(true);

        }

}

Setup:
Localhost running Linux Aptosid

Code:

~$ uname -r

3.7-5.slh.2-aptosid-amd64

Code compiled on Java opendjk 6:

Code:

~$ java -version

java version "1.6.0_27"

OpenJDK Runtime Environment (IcedTea6 1.12.0) (6b27-1.12-1)

OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

ssh

Code:

~$ ssh -V

OpenSSH_6.0p1 Debian-3, OpenSSL 1.0.1c 10 May 2012

Remote host running debian squeeze:

Code:

~$ uname -r

2.6.32-5-amd64

Screen version

Code:

~$ screen -version

Screen version 4.00.03jw4 (FAU) 2-May-06

Java

Code:

~$ java -version

java version "1.6.0_18"

OpenJDK Runtime Environment (IcedTea6 1.8.13) (6b18-1.8.13-0+squeeze2)

OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

ssh

Code:

~$ ssh -V

OpenSSH_5.5p1 Debian-6+squeeze2, OpenSSL 0.9.8o 01 Jun 2010

Anything else missing? Let me know.
Many thanks in advance to everyone looking at this :-)

I've run into this before but I just spawned the java app with nohup, my java app ran in the background and didn't have a GUI so a little different but it worked as a workaround. I didnt have any problems with the screen session not going away. If you spawn the java app with nohup it would allow you to keep it running while you see if you can diagnose the screen issues.

I would imagine it has to do with the way the GUI is starting and forking you off so that you aren't actually on the original terminal session. Thats just a thought, you should be able to check that with some ps commands and checking PPIDs if you have another terminal up at the same time.

Hi Kustom. Thanks for your reply. Unfortunately nohup does not meet my needs as I have to be able to interact with the application while I am still logged in and I have to be able to reattach to it.

Quote:

I would imagine it has to do with the way the GUI is starting and forking you off so that you aren't actually on the original terminal session. Thats just a thought, you should be able to check that with some ps commands and checking PPIDs if you have another terminal up at the same time.

Hmm. My first idea was also that it is somehow related to the multi-threading, but I have not been able to reproduce the issues with just some other thread forking off. It apparently has to be a JFrame. Or at least some gui object.
In my real application I am starting the app on command line, and then only bring up the gui with a dedicated command that is read from stdin.
If I never bring up the gui the detaching works smoothly. However the output of ps looks the same before and after bringing it the gui up.

Just experimented around with this further and found I can strip it down a lot more. In fact the problem still occurs with the following java code:

Code:

import javax.swing.*;



public class Main 

{

        public static void main(String[] args) 

        {

                new JPanel();  // commenting this out stops the issue from reproducing.

                while(true)

                {

                        Thread.yield();

                }

        }

}

So no multi-threading at all. GUI is not even brought up. Just instantiating a swing component is already enough!
Note that non-swing objects to not cause problems. The following code lets me detach gracefully:

Code:

public class Main 

{

        public static void main(String[] args) 

        {

                new String();  // this does not cause any problems

                while(true)

                {

                        Thread.yield();

                }

        }

}

I don't get it. Anyone at least an idea whether this problem is a java problem, an ssh problem or a screen problem? Or how I could diagnose this? Based on that I might turn to the respective support channels / mailing lists...

I think I am getting closer to it. So when I ssh to the remote server without X11 forwarding I am getting the following exception when running the jar that instantiates a JPanel:

Code:

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Class.java:186)

        at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:82)

        at sun.awt.X11.XToolkit.<clinit>(XToolkit.java:112)

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Class.java:186)

        at java.awt.Toolkit$2.run(Toolkit.java:849)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:841)

        at sun.swing.SwingUtilities2$AATextInfo.getAATextInfo(SwingUtilities2.java:121)

        at javax.swing.plaf.metal.MetalLookAndFeel.initComponentDefaults(MetalLookAndFeel.java:1564)

        at javax.swing.plaf.basic.BasicLookAndFeel.getDefaults(BasicLookAndFeel.java:147)

        at javax.swing.plaf.metal.MetalLookAndFeel.getDefaults(MetalLookAndFeel.java:1599)

        at javax.swing.UIManager.setLookAndFeel(UIManager.java:530)

        at javax.swing.UIManager.setLookAndFeel(UIManager.java:570)

        at javax.swing.UIManager.initializeDefaultLAF(UIManager.java:1320)

        at javax.swing.UIManager.initialize(UIManager.java:1407)

        at javax.swing.UIManager.maybeInitialize(UIManager.java:1395)

        at javax.swing.UIManager.getUI(UIManager.java:991)

        at javax.swing.JPanel.updateUI(JPanel.java:126)

        at javax.swing.JPanel.<init>(JPanel.java:86)

        at javax.swing.JPanel.<init>(JPanel.java:109)

        at javax.swing.JPanel.<init>(JPanel.java:117)

        at Main.main(Main.java:7)

So apparantly just instatiating swing components is enough for the JVM to connect to the graphical environment.
I played around a bit with garbage collection etc. and it seems that destroying the JPanel instance is not sufficient to release this connection again.
I have found this thread where they describe how dbus-launch is preventing ssh from exiting normally, and I could even reproduce this with e.g. thunar, but it does not seem to apply to my problem. I have compared the full list of processes (using ps -ef) before and after running the java code and there is no difference, except the jars processor scheduling value, i.e.:

Code:

user 11066 10998 22 22:38 pts/4 00:00:01 java -jar jars/my_test_jar.jar

turns into

Code:

user 11066 10998 30 22:38 pts/4 00:00:07 java -jar jars/my_test_jar.jar

I am guessing it is something similar to the dbus-thing though.

I have also tried to run the ssh session with the verbose switch.
When I am trying to exit it is hanging here:

Code:

user@remotehost:~$ exitdebug1: client_input_channel_req: channel 0 rtype exit-status reply 0

debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0



Abgemeldet

debug1: channel 0: free: client-session, nchannels 2



######## session hangs here until I kill java process from other terminal ############



debug1: channel 1: FORCE input drain

debug1: channel 1: free: x11, nchannels 1

Connection to remotehost closed.

Transferred: sent 14992, received 9656 bytes, in 54.2 seconds

Bytes per second: sent 276.5, received 178.1

debug1: Exit status 0

user@localhost:~$

Still hoping for someone who reads this and can give me a hint...

If the GUI component is using your local X server, you cannot sever the SSH connection without closing the X client GUI. It is probably the SSH server recognizing that the X tunnel is still active, and keeping the connection open until the X tunnel is unused. If you create a separate X connection that is not tunneled in the SSH connection, you will be able to close the SSH connection. It would require you to expose the X server to TCP connections, which is normally not the standard configuration.

--- rod.

Quote:

Originally Posted by theNbomr (Post 4893801)

Hi Rod. Thanks for the reply. This makes sense to me. The things is that by the time I want to close the ssh connection I actually don't need the X tunnel anymore. I'd be happy to stop using it, but don't know how to do it. I would like to keep the application running and only close the gui. Disposing the swing components does not seem to be sufficient. Any ideas?

On the X client host, run lsof, and see what TCP streams are open on ports 6000 to 6020 or so. These are very likely to be the ssh daemon listening for or maintaining an TCP connection that it uses to tunnel the X traffic. Coupled with the PID that the listing will provide, and the value of $DISPLAY in the shell session, you may be able to identify the process that is holding the TCP connection alive. From there you can make some choices about how to deal with it.
The value of $DISPLAY on a ssh-connected session with an X tunnel will be some smallish integer value. That value gets added to 6000 to form the TCP port number used by the ssh tunnel. X clients will use that port to try to connect to an X server on localhost. Finding the PID and nature of the X client that is (probably) holding open the connection may lead to a solution to your problem.
--- rod.

Quote:

Originally Posted by theNbomr (Post 4894522)

Hello Rod. Thank you for the detailed response, this is helpful stuff.
So my $DISPLAY is 10, and when I launched the gui part of my java app I can see these two new line items in the output of LSOF, I guess that is what you were referring to.

Code:

COMMAND    PID      USER  FD      TYPE            DEVICE  SIZE/OFF      NODE NAME

sshd      23163      user  12u    IPv4            7969358        0t0        TCP localhost:6010->localhost:52908 (ESTABLISHED)

java      23221      user  12u    IPv4            7969357        0t0        TCP localhost:52908->localhost:6010 (ESTABLISHED)

I have tried killing one or the other process, but unfortunately this does not leave the non-gui part of the application running. My conclusion is that the part of the java virtual machine that is using the ssh connection to listen to the x-server is running in the same process as the non-gui parts of the application, and can hence not be killed on OS level without killing everything. In other words I should adress this within the java code.
So this question should really have been posted in a Java forum. Unless you disagree with the above I think I will post a new question in a java forum, possible linking to this thread here for background info, and report back if someone there can tell me how to do it.

Again many thanks for the help so far.

Hmmm. Something is not ringing true with that.

Quote:

The application provides the option to open a gui (in a JFrame) that shows current status and reclose it

I think you need to focus on what you do, exactly, to 'close' the GUI. Not displaying any window doesn't necessarily mean that the connection to the X server has been closed. There is probably some way to fully close all X communications that you haven't done. I don't know enough Java or it's GUI components to advise you further on that.

--- rod.

Quote:

Originally Posted by theNbomr (Post 4894628)

Hmmm. Something is not ringing true with that.

I think you need to focus on what you do, exactly, to 'close' the GUI. Not displaying any window doesn't necessarily mean that the connection to the X server has been closed. There is probably some way to fully close all X communications that you haven't done. I don't know enough Java or it's GUI components to advise you further on that.

--- rod.

I think you are right. I posted this question on the oracle forums, since I have no idea what to do to fully close all X communications.
I case you are interested you may find the thread I opened here:

https://forums.oracle.com/forums/thr...00228&tstart=0

Quote:

Originally Posted by joe_2000 (Post 4894669)

No movement at all over there. That forum looks very quiet. Can't believe that nobody out there knows the answer to this question. You'd think it is a pretty basic one. But I am afraid I will have to work around it. Very unsatisfying...

I finally solved it. I don't think it can be solved inside of java as long as one wants to keep using swing. (I believe swing does not support running in headless mode at all).

So it had to be on OS level. I found this nice tool called xpra.

It allows to move an x application between different x displays.
With that I can create a display, run my java application on it, and attach to that display when I want to look at the GUI.
I can also detach from it again, all of that without killing the running process.

For Debian Squeeze xpra does not seem to be in the standard repos, so I installed it using the instructions given on their website:

http://xpra.org/debian.html