Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Yesterday, after a restart of the server, we had to make the horrifying experience that our home folder had been rollbacked for almost half a year!
We immediatly contacted our server Provider (Hetzner) and they told us that the raid md127 didn't start up correctly and that we should reload the raid manually.
I wanted to get help from this forum, since i am new to this whole subject and since there is a high risk in losing all your data if approaching falsely to this issue.
To our problem:
When typing in "cat /proc/mdstat": http://puu.sh/2dtXE (Screenshot)
We can see, that md127 marks a _U. As far as i understood that means sdba4 can't be loaded but sdb4 is loaded.
When having a closer look into md127 with "mdadm -D /dev/md127": http://puu.sh/2du19 (Screenshot)
We see, that partition number 0 has been removed and 1 is running
I would have also given you the etc/raidtab but for some reason its missing on our root!
As mentioned above i don't really know how to approach in such a case, do i just reactivate the raid with commands, copy it over to another, or do we even have to get the disk swapped?
I would be very thankful for any kind of help and advice you can give me.
I am kind of scared to lose our important data, thats why i am asking here :S I hope you have comprehension for that
3. It looks like a strange setup; you appear to have 4 RAID1 (mirror) sets, but only 2 physical disks: sda, sdb.
This is not a good idea if one disk goes bad, all RAID sets would be affected.
What you appear to have is md3 has split into 2 single disk RAID1 sets; md3 & md127.
Can you check the conf file or somehow other check how the RAID sets were built eg ask your Provider ?
2. actually, there's no '-' in my 'cat' cmd; its deliberate so that it usually works on most distros.
3. It certainly looks like it, given the other RAID arrays and their numbering.
That's why its important to find out if they are 2 halves of the same RAID set, or if you've broken 2 sets. My money is on the former.
Whoever built the sets should know... and you NEED to know before you try fixing anything.
Incidentally, if you can avoid using those 2 and ideally unmount them, that should stop any further drift in content.
2. actually, there's no '-' in my 'cat' cmd; its deliberate so that it usually works on most distros.
3. It certainly looks like it, given the other RAID arrays and their numbering.
That's why its important to find out if they are 2 halves of the same RAID set, or if you've broken 2 sets. My money is on the former.
Whoever built the sets should know... and you NEED to know before you try fixing anything.
Incidentally, if you can avoid using those 2 and ideally unmount them, that should stop any further drift in content.
We haven't really touched anything on the raids it must have happened automatically.
Do u have any other ways of finding out how everything looked like before the incident since the conf files dont provide anything for some reason :/
Would the provider know?
EDIT: I might have found something that could support your assumption
When typing in "mdadm --detail --scan >> /etc/mdadm/mdadm.conf" i get the following + 1 error message:
I unmounted md127 to see what would happen, i restarted the server and the md127 entry was gone, also the md127 file in /dev/. The /home directory is empty now
Is that normal? Or did we screw up here?
also we get this error:
Quote:
mount: wrong fs type, bad option, bad superblock on /dev/md3,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
EDIT: Sry i didn't want to spam that hard it seemed like i was lagging or the webpage so i may have accidently clicked post several times
3. I have read somewhere that if a raid can't be loaded the system tries to create a new raid set as a mirror of the broken one and all of these automatic created raids start with the number 127-. So in this case i would assume that the setup was md0,md1,md2,md3. Also if you check on the hetzner wiki about repairing a broken raid they are also talking about this setup. However if you think this is not yet enough information we can contact the provider again to be 100% sure.
4. Hmm... More or less. We had most of our backups in the home directy which has been rollbacked for 5 months (i honestly dont understand why 5 months) and we only have backups that are 2-3 months old on external devices. We are kind of in problematic situation.
EDIT: Btw if we can't manage to get the raids mounted again, do you know any way of extracting or bumping the content of a raid1 file to a directory or isn't this possible? We are planning on formatting the whole system IF we can get the files back
mount: none already mounted or /dev/pts busy
mount: according to mtab, devpts is already mounted on /dev/pts
mount: wrong fs type, bad option, bad superblock on /dev/md3,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
mount: wrong fs type, bad option, bad superblock on /dev/md3,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
EDIT:
Here the dmesg tail:
Quote:
[ 19.551459] INFO-xpp: FEATURE: with sync_tick() from DAHDI
[ 19.643844] INFO-xpp_usb: revision Unknown
[ 19.644024] usbcore: registered new interface driver xpp_usb
[ 20.307779] dahdi: Registered tone zone 0 (United States / North America)
[ 21.759758] eth0: no IPv6 routers present
[ 34.032136] [drm] Initialized drm 1.1.0 20060810
[ 34.635433] lp: driver loaded but no devices found
[ 34.771899] ppdev: user-space parallel port driver
[ 142.734088] EXT4-fs (md3): VFS: Can't find ext4 filesystem
[ 142.742375] EXT4-fs (md3): VFS: Can't find ext4 filesystem
EDIT2:
md3 details give this:
Quote:
/dev/md3:
Version : 1.2
Creation Time : Sat Jun 23 13:47:29 2012
Raid Level : raid1
Array Size : 1822442815 (1738.02 GiB 1866.18 GB)
Used Dev Size : 1822442815 (1738.02 GiB 1866.18 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Mar 8 12:58:03 2013
State : clean, degraded
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1
Name : rescue:3
UUID : 45958b4b:1024b8cb:30a98470:705d7110
Events : 3101840
Number Major Minor RaidDevice State
0 8 4 0 active sync /dev/sda4
1 8 20 1 spare rebuilding /dev/sdb4
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.