[SOLVED] Failed miserably to execute bash script via PATH variable after FS migration.

kopatops · 08-18-2010, 01:49 AM

Hi,

I just wiped my old big NTFS partition, where I stored all kinds of user-important stuff such as /home/. I started getting strange errors, like the system freezing to a halt every now and then, and input/output errors when writing to my ~/.mozilla directory.

Hoping it was not my Intel 80GB SSD already starting to fail, I backed everyting up on an external magnetic drive, deleted the NTFS and formatted the partition with XFS. (I heard it might be good for big files, but whatever.)

After copying everything back, and setting stuff to order, changing permissions and ownership, and so on, I can't execute bash scripts in my bash script folder ~/bin/.

~/bin/ is in the PATH variable.

What's weird (and what I suppose justifies posting on LQ) is that I can execute a script ~/bin/script fine, either by typing

Code:

$ bash ~/bin/script

or standing in the bin directory and typing

Code:

$ bash script

-------------------------------------------------------

All files in bin have 'executable' permissions for everyone, but most are run using sudo anyway. And when I run a script called wifil using sudo, I get

Code:

[user@computer /]$ sudo wifil
Password: 
sudo: unable to execute ~/bin/wifil: Permission denied
*** glibc detected *** sudo: double free or corruption (out): 0x00000000011ce340 ***
======= Backtrace: =========
/lib/libc.so.6(+0x716b6)[0x7f41e43be6b6]
/lib/libc.so.6(cfree+0x6c)[0x7f41e43c33ac]
/lib/libpam.so.0(+0x2779)[0x7f41e48c3779]
/lib/libpam.so.0(pam_end+0x22)[0x7f41e48c4242]
sudo[0x40429a]
sudo[0x40d887]
sudo[0x40f2b9]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7f41e436bc4d]
sudo[0x4036c9]
======= Memory map: ========
00400000-00427000 r-xp 00000000 08:01 33711                              /usr/bin/sudo
00626000-00629000 rw-p 00026000 08:01 33711                              /usr/bin/sudo
00629000-0062d000 rw-p 00000000 00:00 0 
011cc000-011ed000 rw-p 00000000 00:00 0                                  [heap]
7f41dc000000-7f41dc021000 rw-p 00000000 00:00 0 
7f41dc021000-7f41e0000000 ---p 00000000 00:00 0 
7f41e2920000-7f41e2935000 r-xp 00000000 08:01 6380                       /usr/lib/libgcc_s.so.1
7f41e2935000-7f41e2b35000 ---p 00015000 08:01 6380                       /usr/lib/libgcc_s.so.1
7f41e2b35000-7f41e2b36000 rw-p 00015000 08:01 6380                       /usr/lib/libgcc_s.so.1
7f41e2b36000-7f41e3d05000 rw-p 00000000 00:00 0 
7f41e3d05000-7f41e3d11000 r-xp 00000000 08:01 2306                       /lib/libnss_files-2.12.so
7f41e3d11000-7f41e3f10000 ---p 0000c000 08:01 2306                       /lib/libnss_files-2.12.so
7f41e3f10000-7f41e3f11000 r--p 0000b000 08:01 2306                       /lib/libnss_files-2.12.so
7f41e3f11000-7f41e3f12000 rw-p 0000c000 08:01 2306                       /lib/libnss_files-2.12.so
7f41e3f12000-7f41e3f1a000 r-xp 00000000 08:01 2301                       /lib/libcrypt-2.12.so
7f41e3f1a000-7f41e4119000 ---p 00008000 08:01 2301                       /lib/libcrypt-2.12.so
7f41e4119000-7f41e411a000 r--p 00007000 08:01 2301                       /lib/libcrypt-2.12.so
7f41e411a000-7f41e411b000 rw-p 00008000 08:01 2301                       /lib/libcrypt-2.12.so
7f41e411b000-7f41e4149000 rw-p 00000000 00:00 0 
7f41e4149000-7f41e414b000 r-xp 00000000 08:01 2373                       /lib/libdl-2.12.so
7f41e414b000-7f41e434b000 ---p 00002000 08:01 2373                       /lib/libdl-2.12.so
7f41e434b000-7f41e434c000 r--p 00002000 08:01 2373                       /lib/libdl-2.12.so
7f41e434c000-7f41e434d000 rw-p 00003000 08:01 2373                       /lib/libdl-2.12.so
7f41e434d000-7f41e44a0000 r-xp 00000000 08:01 2351                       /lib/libc-2.12.so
7f41e44a0000-7f41e469f000 ---p 00153000 08:01 2351                       /lib/libc-2.12.so
7f41e469f000-7f41e46a3000 r--p 00152000 08:01 2351                       /lib/libc-2.12.so
7f41e46a3000-7f41e46a4000 rw-p 00156000 08:01 2351                       /lib/libc-2.12.so
7f41e46a4000-7f41e46a9000 rw-p 00000000 00:00 0 
7f41e46a9000-7f41e46c1000 r-xp 00000000 08:01 6166                       /usr/lib/libz.so.1.2.5
7f41e46c1000-7f41e48c0000 ---p 00018000 08:01 6166                       /usr/lib/libz.so.1.2.5
7f41e48c0000-7f41e48c1000 rw-p 00017000 08:01 6166                       /usr/lib/libz.so.1.2.5
7f41e48c1000-7f41e48cd000 r-xp 00000000 08:01 6470                       /lib/libpam.so.0.82.2
7f41e48cd000-7f41e4acc000 ---p 0000c000 08:01 6470                       /lib/libpam.so.0.82.2
7f41e4acc000-7f41e4acd000 rw-p 0000b000 08:01 6470                       /lib/libpam.so.0.82.2
7f41e4acd000-7f41e4acf000 r-xp 00000000 08:01 2331                       /lib/libutil-2.12.so
7f41e4acf000-7f41e4cce000 ---p 00002000 08:01 2331                       /lib/libutil-2.12.so
7f41e4cce000-7f41e4ccf000 r--p 00001000 08:01 2331                       /lib/libutil-2.12.so
7f41e4ccf000-7f41e4cd0000 rw-p 00002000 08:01 2331                       /lib/libutil-2.12.so
7f41e4cd0000-7f41e4cee000 r-xp 00000000 08:01 2372                       /lib/ld-2.12.so
7f41e4d0e000-7f41e4ec1000 r--p 00000000 08:01 14030                      /usr/lib/locale/locale-archive
7f41e4ec1000-7f41e4ec6000 rw-p 00000000 00:00 0 
7f41e4eed000-7f41e4eee000 rw-p 00000000 00:00 0 
7f41e4eee000-7f41e4eef000 r--p 0001e000 08:01 2372                       /lib/ld-2.12.so
7f41e4eef000-7f41e4ef0000 rw-p 0001f000 08:01 2372                       /lib/ld-2.12.so
7f41e4ef0000-7f41e4ef1000 rw-p 00000000 00:00 0 
7fff104d3000-7fff104f4000 rw-p 00000000 00:00 0                          [stack]
7fff10592000-7fff10593000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted
[user@computer /]$

The script runs *perfectly* whenever bypassing the PATH variable. Please help me. I think I'm going crazy. That backtrace is too much for a humble shellscript programmer. x)

pingu · 08-18-2010, 02:17 AM

Could this be a problem with sudo?
What happens, what errors, when you try to run a script without sudo?

i92guboj · 08-18-2010, 02:25 AM

Check the output for "which sudo" and "ldd sudo", maybe you have some leftover from the previous installation. You should also check the contents of $PATH with "echo $PATH", for the very same reason.

It seems like either sudo is broken or you are using a different version of glibc than the one that was used to compile your sudo binary. It could also be a problem in glibc, but then your whole system would probably be unusable.

kopatops · 08-18-2010, 04:33 AM

Running a script without sudo simply gives me

Code:

[user@computer /]$ wifil
bash: ~/bin/wifil: Permission denied
[user@computer /]$

Permissions for that particular script:

Code:

[user@computer /]$ ls -l ~/bin/ | grep wifil
-rwxr-xr-x 1 user users    1340 Aug 18 08:12 wifil
[user@computer /]$

After a reboot (and possibly other things) I now get

Code:

[user@computer /]$ sudo wifil
Password: 
sudo: unable to execute ~/bin/wifil: Permission denied
Segmentation fault
[user@computer /]$

I'm sorry, but I really can't say what I did to avoid the backtrace output. But from what I understand (segmentation fault), something is wrong with a binary, and not the script? Sudo and bash are the only binaries involved.

Quote:

Check the output for "which sudo" and "ldd sudo", maybe you have some leftover from the previous installation. You should also check the contents of $PATH with "echo $PATH", for the very same reason.

Code:

[user@computer /]$ which sudo
/usr/bin/sudo
[user@computer /]$

Code:

[user@computer /]$ sudo ldd /usr/bin/sudo
Password: 
	linux-vdso.so.1 =>  (0x00007fff92bff000)
	libutil.so.1 => /lib/libutil.so.1 (0x00007f72e42af000)
	libpam.so.0 => /lib/libpam.so.0 (0x00007f72e40a3000)
	libz.so.1 => /usr/lib/libz.so.1 (0x00007f72e3e8b000)
	libc.so.6 => /lib/libc.so.6 (0x00007f72e3b2f000)
	libdl.so.2 => /lib/libdl.so.2 (0x00007f72e392b000)
	libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007f72e36f4000)
	/lib/ld-linux-x86-64.so.2 (0x00007f72e44b2000)
[user@computer /]$

Code:

[user@computer /]$ echo $PATH
/opt/wine/bin:/bin:/usr/bin:/sbin:/usr/sbin:/opt/java/jre/bin:/opt/kde/bin:/usr/lib/perl5/vendor_perl/bin:/usr/bin/perlbin/vendor:/usr/lib/perl5/core_perl/bin:/opt/qt/bin:~/bin/:~/bin/scriptscript:~/bin/: ~/bin/scriptscript
[user@computer /]$

It does seem like I have duplicate entries in PATH. Is this a problem? Also I'm not sure why. ~/bin and ~/bin/scriptscript are added in the .bashrc file at logon.

Quote:

It seems like either sudo is broken or you are using a different version of glibc than the one that was used to compile your sudo binary. It could also be a problem in glibc, but then your whole system would probably be unusable.

The system doesn't exactly feel rock solid with those random (halts/kernelpanics/whatever) [EDIT: those haven't occurred since XFS migration] but sure it is usable... Also, sudo works flawlessly in all other contexts. Actually,

$ sudo ~/bin/wifil [EDIT: *need* to use 'sudo bash (path)'. Sorry for this typo.]

connected me so I could post this.

Note, that all I ever did was
- moving ~/ (let's call it "DIR") from the "storage partition" sda2, to an external disk
- deleting sda2
- creating 70GB XFS system on new sda2
- moving DIR back to sda2
- creating new symlink /home/user --> [sda2_mount_point]/DIR

user's home directory is /home/user/ ,where /home/user is a symlink to a directory on the "storage" partition, which has in effect had a change of filesystem.

I should mention that there was definitely corrupted data on the old filesystem, some of which resided in ~/. However, now I can fsck /dev/sda2, (can't with an NTFS) and the checks show no errors.

I find it hard to beleive that certain programs really cause the problem, because they reside on their own 10GB ext4 partition. EDIT: As do all shared libraries.

pingu · 08-18-2010, 04:48 AM

Quote:

Originally Posted by kopatops

Running a script without sudo simply gives me

Code:

[user@computer /]$ wifil
bash: ~/bin/wifil: Permission denied
[user@computer /]$

But you also said in first post:

Quote:

Originally Posted by kopatops

I can execute a script ~/bin/script fine, either by typing

Code:

$ bash ~/bin/script

And then:

Code:

$ sudo ~/bin/wifil
connected me so I could post this.

So the problem only occurs sometimes?
I'm beginning to believe you have a somewhat corrupted system, you say yourself it isn't rock solid, with frequent halts/kernelpanics/whatever.
How about free space?
What mount options for your /home?
Try moving the scripts to some location outside this new filesystem see if that works better.

kopatops · 08-18-2010, 05:24 AM

[QUOTE=pingu;4070012]But you also said in first post:

And then:

Code:

$ sudo ~/bin/wifil
connected me so I could post this.

I'm terribly sorry for making such typos. I'll try to get it right:

I do need to use

Code:

[user@computer /]$ sudo bash ~/bin/script

or

Code:

[user@computer ~]$ sudo bash script

But you *are* saying what I'm thinking right now; Something is broken. The errors are sort of random.

fstab entry:

Code:

/dev/sda1    /               ext4      defaults,noatime,discard         0  1
/dev/sda2    /mnt/0          xfs       rw,noatime,user                  0  0

Code:

[user@computer /]$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
udev                     10240       212     10028   3% /dev
/dev/disk/by-uuid/d39a9c41-fece-4837-9f5e-b32bdbe5131e
                       4198236   3257516    727464  82% /
none                    960052         0    960052   0% /dev/shm
tmpfs0                  960052       160    959892   1% /tmp
tmpfs1                  960052     12664    947388   2% /var/cache/pacman
/dev/sda2             73846860  44940616  28906244  61% /mnt/0
/dev/loop0             7527102   7527102         0 100% /mnt/2
[user@computer /]$

I hoped to fix stuff by reformatting the affected partition, but there might be some physical damage on the disk.
I don't know what to do. I'm terribly keen on keeping the data in ~/.

i92guboj · 08-18-2010, 05:24 AM

fs corruption or hardware problem could be also causing this. If the problem is in a script you should run it with '#!/bin/bash -x' so you can get verbose output and tell what exactly inside the script is causing the problem.

If the fs got corrupted you should fsck it first, and then reinstall all the packages. Your package manager should provide an easy way to do that.

pingu · 08-18-2010, 05:30 AM

Now wait a minute: Is it so, that you cannot execute any script unless you explicitly runs it with bash - that is, typing "bash scriptfile" works but not "scriptfile" alone?
If so, what is your default shell, is it really bash?
(You can check that with
# cat /etc/passwd |grep user
You will see what is default shell, like mine here is bash:

Code:

pingu@edgar:~$ cat /etc/passwd |grep ping
pingu:x:501:501:P,,,:/home/pingu:/bin/bash

i92guboj · 08-18-2010, 05:35 AM

That shouldn't matter as long as the script has a correct header as it should. It's worth checking though. It's also worth checking what does /usr/bin/sh points to with ls -l.

kopatops · 08-18-2010, 05:38 AM

Quote:

Originally Posted by pingu

Try moving the scripts to some location outside this new filesystem see if that works better.

Eureka!!!

Code:

[user@computer /]$ sudo cp ~/bin/wifil /mnt/3
Password: 
[user@computer /]$ sudo ~/bin/wifil
Password: 
sudo: unable to execute ~/bin/wifil: Permission denied
Segmentation fault
[user@computer /]$ sudo /mnt/3/wifil 
Password: 
/mnt/3/wifil: line 14: [: ==: unary operator expected
Local wifi connection script: START
Make sure you are root, I won't check this for you
killing all client apps...
wpa_supplicant: no process found
dhcpcd: no process found
DONE
NIC down...
DOWN
NIC up...
UP

The script is currently configured for:
PG
Security: wpa

WPA security enabled. If this is wrong,
kill the script, it's useless. 
ioctl[SIOCSIWAP]: Operation not permitted
ioctl[SIOCSIWESSID]: Operation not permitted
WPS-AP-AVAILABLE 
Trying to associate with 00:11:6b:44:97:70 (SSID='bono' freq=2462 MHz)
Associated with 00:11:6b:44:97:70
WPA: Key negotiation completed with 00:11:6b:44:97:70 [PTK=CCMP GTK=CCMP]
CTRL-EVENT-CONNECTED - Connection to 00:11:6b:44:97:70 completed (auth) [id=0 id_str=]
dhcpcd[12452]: version 5.2.6 starting
dhcpcd[12452]: wlan0: rebinding lease of 193.11.239.42
dhcpcd[12452]: wlan0: acknowledged 193.11.239.42 from 193.11.239.1
dhcpcd[12452]: wlan0: checking for 193.11.239.42
dhcpcd[12452]: wlan0: leased 193.11.239.42 for 300 seconds
dhcpcd[12452]: forked to background, child pid 12493
Call script itest:
/mnt/3/wifil: line 47: ~/bin/itest: Permission denied

Local wifi connection script: END
[user@computer /]$

What on earth?? I would never have thought of that. THANK YOU!

But why?

kopatops · 08-18-2010, 05:41 AM

Quote:

Originally Posted by i92guboj

That shouldn't matter as long as the script has a correct header as it should. It's worth checking though. It's also worth checking what does /usr/bin/sh points to with ls -l.

The header points as follows:
# ! /bin/bash
I changed it from /bin/sh after the errors started to appear.

Code:

[user@computer /]$ ls -l /bin/ | grep "sh \-"
lrwxrwxrwx 1 root root      4 May 17 02:40 sh -> bash
[user@computer /]$

i92guboj · 08-18-2010, 05:41 AM

Did you already try fsck?

kopatops · 08-18-2010, 05:49 AM

Quote:

Originally Posted by i92guboj

Did you already try fsck?

Yes, first thing I did before copying back ~/ was

-logon as root
-umount /dev/sda2
-fsck /dev/sda2 (no output)
-mount /dev/sda2 /mnt/0 (no -t specification)
-exit
-logon as user

pingu · 08-18-2010, 05:53 AM

My thought was that you might have /home mounted with options/flags not allowing you to execute.
But then, it shouldn't work with "bash scriptfile" either?
I have never used this possibility myself, but maybe it's worth digging into?
It could be that different filesystems have different default options.

i92guboj · 08-18-2010, 06:02 AM

Quote:

Originally Posted by pingu

My thought was that you might have /home mounted with options/flags not allowing you to execute.
But then, it shouldn't work with "bash scriptfile" either?

He should check the output from "mount" without arguments and make sure that neither of "noexec" and "users" are between the mount options ("users" implies "noexec" by default).

When you open a script doing "bash <filename>" there's virtually no difference between that and doing "oowrite my_file.doc". Bash will launch a new session and start parsing the file. You can quickly check by setting -x on any random script and then launching it with bash or sh.