How dangerous to shrink system LVM partition a little bit with gparted? [CentOS 7]

ohmster · 05-21-2015, 04:24 AM

Okay I have been working on a a nice, older Pentium 4 and prepping it to be my new Linux Server. I wanted something new and upped the system w/new CPU that is x64 and 3.6GHZ instead of 3GHz. The system has a 1TB SATA hard drive. Only using about 200Gb of the drive right now.

I got a great rsync script to run in cron that backs up the entire disk to the external 1TB USB drive. Since the rsync works so well and easily backs up the entire 1TB hard disk to a small amount of space and preserves all ownerships and permissions, this is a great way to back things up. But to restore the entire disk from this backup would be nearly impossible. I need a system disk image. I will use the 750Gb USB drive for rsync daily and the 1TB USB disk for manual disk imaging with HDClone 5, True Image, Ghost, Clonezilla or something else.

I have a real serious problem. Although the internal SATA and external USB drives are both 1TB, the internal system drive with boot and / partitions is just a wee bit larger than the external USB drive. In HDClone 5, the source drive is just a tad over 1TB in size, but the USB drive is showing 931.5Gb, even after wiping it with gparted, making a full sized ext4 partition, HDClone 5 will NOT allow me to proceed because the source drive is larger than the target drive. I need to trim the / partition from 1,002Gb down below the 930GB of the target backup drive. Then, I can save an image of the system drive in the event something really bad happens to restore and still have my data backed up with rsync.

Shrinking partitions is not new, it is supposed to be safe with Partition Magic or gparted. But this is an LVM partition and I am really scared to shrink it without "Expert Advice" because I learned the hard way what happens when you mess up an lvm drive.

How safe is it to shrink an LVM partition in Linux and how would you recommend it be done? Gparted? Commercial software?

Experts please chime in! Thank you very much.

beachboy2 · 05-21-2015, 06:15 AM

ohmster,

Do make sure all personal data is backed up before trying any of the suggestions here:

http://askubuntu.com/questions/19612...hysical-volume

syg00 · 05-21-2015, 06:39 AM

If you have a rsync copy of the whole system, why do you need an image ?.
And why aren't you using an open source tool ?.

You seem insistent on Rube Goldberg solutions.

beachboy2 · 05-21-2015, 07:50 AM

Quote:

You seem insistent on Rube Goldberg solutions.

Or even Heath Robinson ones

ohmster · 05-21-2015, 02:05 PM

Thank you beachboy2. I will look into this article.

Quote:

If you have a rsync copy of the whole system, why do you need an image ?.
And why aren't you using an open source tool ?.

Because I do not know how to restore an rsync backup of an lvm drive back to a new hard disk and actually get the system to boot and run fine. rsync is great for data backup and even the system is backed up, all perms and ownership intact. But I know of no way to restore that and re-create the system disk if it goes down.

Images can be restored to a new disk, turn the PC on, and it starts right up like nothing ever happened. I have to do it manually and use yum update to make sure the restored image is up to date. Then I have my data backup to restore data.

It took weeks to install and setup this system for me. A hard drive image can be restored in an hour or so.

Quote:

You seem insistent on Rube Goldberg solutions.

What in the world is so "Rube Goldberg" about an rsync backup script to cron or a hard disk image? You guys use shell scripts all the time and imaging a disk is just free or commercial software.

If you guys have a better idea, I am all ears and that *is* what I am asking for. Thank you fellows!

onebuck · 05-22-2015, 09:05 AM

Moved: This thread is more suitable in <CentOS> and has been moved accordingly to help your thread/question get the exposure it deserves.

rknichols · 05-22-2015, 10:32 AM

First, I believe you are mixing binary GiB and decimal GB in your numbers. The USB drive is probably 931.5 GiB, which is 1000.19 GB. That's a reasonable number for a nominal "1 TB" drive, but is indeed just a tiny amount smaller than your other "1 TB" (1002 GB) drive. In hindsight (which, of course, is always 20/20), that's a good reason for not using more than the nominal capacity of any drive.

I believe the CentOS 7 installer defaults to the XFS filesystem, and one of the disadvantages of that filesystem is that it cannot be shrunk. What filesystem(s) are you using inside that LVM partition? Other filesystems can be shrunk, though I don't think you will find a tool to do that automatically through all the levels of containers (partition -- LVM PV -- LVM LV -- filesystem).

dt64 · 05-26-2015, 01:29 PM

it's not that hard to reduce the size of a LVM partition

lvresize -r is your friend.

the only hook here is whether you can unmount the partition. If you can't (since it's a system partition) you should boot from a livecd and run the command from there.

ohmster · 05-27-2015, 02:38 AM

Quote:

Originally Posted by rknichols

First, I believe you are mixing binary GiB and decimal GB in your numbers. The USB drive is probably 931.5 GiB, which is 1000.19 GB. That's a reasonable number for a nominal "1 TB" drive, but is indeed just a tiny amount smaller than your other "1 TB" (1002 GB) drive. In hindsight (which, of course, is always 20/20), that's a good reason for not using more than the nominal capacity of any drive.

I believe the CentOS 7 installer defaults to the XFS filesystem, and one of the disadvantages of that filesystem is that it cannot be shrunk. What filesystem(s) are you using inside that LVM partition? Other filesystems can be shrunk, though I don't think you will find a tool to do that automatically through all the levels of containers (partition -- LVM PV -- LVM LV -- filesystem).

Everything is xfs, a quick peek at fstab should show us what is up.

Code:

#
#
# /etc/fstab
# Created by anaconda on Tue Apr 28 04:54:46 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos_linux--server-root /                       xfs     defaults        0 0
UUID=787b75ae-64e1-4d33-bc8c-8a8b8af52a09 /boot                   xfs     defaults        0 0
/dev/mapper/centos_linux--server-home /home                   xfs     defaults        0 0
/dev/mapper/centos_linux--server-swap swap                    swap    defaults        0 0
LABEL=USB-Backup        /mnt/backup              ext4    noauto,x-systemd.automount      0 0
#LABEL=IMAGE-BU         /mnt/backup              ext4    noauto,x-systemd.automount      0 0

Yes, I was right. I also think your size on the backup drive is correct, if I access it, it should auto-mount.

NOTE: You see two identical lines at bottom of fstab, one commented out. I NEVER plug these drives in at the same time. Depending on what I plan to do. Right now, the imaging is out of reach. Either one would grab the first /dev mount available, /dev/sdb1. This P4 will not, under any circumstances boot if any external USB drive is plugged in. So I must plug it in once booted. I took "auto" out of the label so that the CentOS machine would not have a fit and dump me into an emergency shell when it did not find /dev/sdb1 plugged in. The joy of having an almost ten year old Asus P5GD1 motherboard.

gparted shows the disk sizes as follows:
750GB USB = 698.6GB
1Tb USB = 931.51GB
Internal /dev/sda1 = 500Mb /boot Ext4
Internal /dev/sda2 = 931.51GB xfs

fdisk:
1Tb USB = 1002GBGB
Internal /dev/sda = 1002GBGB

Makes no sense.

Thank you rknichols. Your explanation on why two "identical size" drives are not identical at all seems to make sense. I have the rsync script to backup the drive and data to ext-usb with perms and ownership retained. Great for data, near impossible to "restore the system to a replacement drive if need be. My plan was to make one or two system images a year, just so that I have something to restore in the event of a catastrophic disk or system failure. In my PC experience, a hard drive image is ideal. One can restore the image to the same or a different drive, plug it in, turn it on, and it works as it did when the backup image was made. Then, I could restore any missing data from rsync and use yum to update the system.

I spent a LOT of time setting up all the servers and daemons that I need for this system to work. I would love to restore it in an hour or two rather than spend weeks trying to put it all back by hand, THEN restoring the data. It is just my luck that "the bigger drive" is unconventional and the one that requires imaging. So far, nothing has worked. Not even dd because that would also require the same target space and the source to work, even with compression added like zip, gunzip, or anything else. Not even newer commercial software will touch it. I thought for sure I would be good with HDClone 5 Enterprise Edition but no. I cannot click "Next" because I am a few hundred megs short on the target. This is very depressing.

Other than purchasing a very large backup drive, 2Tb or larger, I see no way to get this done and with SSI and disability from working my whole life providing a very slim income, I can't budget for it unless I get a really good deal on one or earn extra money somehow. Just this week I picked up an extra $300 to move two Godaddy websites from Economy Linux Hosting to combined account with Deluxe Linux hosting with cPanel. Nice, but things like web form mail and blogs are slowing me way down. And the money is ½ up front, the other ½ when done. That is why I have been MIA for a while. My trusty Pentium 4 3GHz, 32 bit, CentOS 6.6 saw me through this site work and has never given me issues for decades. This new (older) CentOS 7 x64 P4, 3.6GHz machine was supposed to replace it. But I never get it stable and backed up enough to make the swap. Main thing now is thermal, I need to try a different CPU heat sink. Pulling the one back off that I mounted is not making me happy with the spread area of the Z9 heat sink compound applied to the heat sink. Should be a perfect mirror as to what was applied to the heat sink on the CPU and there is some bare metal showing. ...another day, another thread.

My current machine is not backed up at all. I can always pull the sites back down in FTP if I had to. But an image sure would make me one happy man!

I really have to keep my eyes open for a 2TB or larger drive. Thank you my friend, I do appreciate the effort on your part and suggestions!

ohmster · 05-27-2015, 02:47 AM

Quote:

Originally Posted by dt64

it's not that hard to reduce the size of a LVM partition

lvresize -r is your friend.

the only hook here is whether you can unmount the partition. If you can't (since it's a system partition) you should boot from a livecd and run the command from there.

You know what dt64? This actually sounds "doable". Yes of course, I will use a live CD. At this point, the machine is no way going into service and I am in no trouble as my CentOS 6.6 lives on man. If I kill it, well, no huge loss, a learning experience but I do like for them to pay off, if possible. I will look into it when I have time and come back to thank and update when I can. I do use the Linux machine for a Dreamweaver testing server and file host. Right now I got a nice $300 job to move to sites from Economy Linux to Deluxe Linux Hosting with cPanel and consolidate the two separate sites and accounts into one. I need the money so I must stop this for now as I really need the money. Moving the sites from one host to another is no big deal. But the form mail and blog make it more difficult. Form mail done last night with a simple anti-spam php form and script. But the blog on the other site will be a super PITA as LifeType blog is like 10 years old and I doubt I can import it to WordPress, not to mention setting it all up and teaching it to him.

I am keeping this as a text file to investigate when I have time. Thank you dt!

dt64 · 05-27-2015, 04:27 AM

Quote:

Originally Posted by ohmster

Everything is xfs, a quick peek at fstab should show us what is up.

This is a problem. You can't shrink xfs. Read http://xfs.org/index.php/XFS_FAQ#Q:_..._or_smaller.3F

I know xfs is the new standard file system in RHEL/CentOS, but the above is one of the reasons I still prefer to use ext4 on my systems.

rknichols · 05-27-2015, 02:05 PM

How much space do you have allocated to the swap LV? If you could trim 2GB from that, you could make everything fit. I'd need to see the contents of /etc/lvm/backup/linux--server and the output from "fdisk -lu" to see how difficult/dangerous that might be.

ohmster · 05-27-2015, 04:32 PM

Quote:

Originally Posted by rknichols

How much space do you have allocated to the swap LV? If you could trim 2GB from that, you could make everything fit. I'd need to see the contents of /etc/lvm/backup/linux--server and the output from "fdisk -lu" to see how difficult/dangerous that might be.

rknichols, thank you for replying. We may have insurmountable issues with this if what I read in another thread is true in post #6: Shrink LMV2 root partition

syg00 says that XFS is bad and cannot be shrunk. That is the filesystem in use with the LVM partitions. Only the swap filesystem is different.

There is no "/etc/lvm/backup/linux--server" but there is a /etc/lvm/backup/centos_linux-server file. I will put the output in a code box because it is confusing to me. You must be a genius to understand this file my friend.

The machine has 4, 1Gb stick of ram, making a total of 4Gb physical RAM.

I simply wanted to copy the entire hard drive, raw, over to the USB drivie as an image, shrunken down by compression so as not to backup empty space. I have done this many times with "ordinary hard disks" like Windows many times using Ghost or True Image. I was very surprised to find out that I cannot use this method with the new Linux, then it became apparently because these "tools" cannot mount or read LVM, or so I suspect. So the only imaging option I have is sector-by-sector, data present or not. Meaning the "Entire Disk" would have to be imaged because these tools cannot read unmounted LVM to determine what is data and what is not? Yeah, or "something like that"?

Simply put, the target backup drive must be larger than the source drive, even if it is only one bit.

/dev/sdb IS the 1Tb USB-Backup drive. It contains a single ext4 partition as /dev/sdb1 and that partition fills the entire disk. For whatever reason, there is always some "sliver" of unused space when making full disk partitions. Maybe it is due to block size.

Code:

[root@ohmster backup]# cat centos_linux-server
# Generated by LVM2 version 2.02.115(2)-RHEL7 (2015-01-28): Tue Apr 28 01:45:05 2015

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'pvscan --cache --activate ay 8:2'"

creation_host = "Linux-Server.localdomain"      # Linux Linux-Server.localdomain 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64
creation_time = 1430199905      # Tue Apr 28 01:45:05 2015

centos_linux-server {
        id = "d9OGt2-fedC-v7PW-6TV1-bpTv-deHJ-Xjveyy"
        seqno = 4
        format = "lvm2"                 # informational
        status = ["RESIZEABLE", "READ", "WRITE"]
        flags = []
        extent_size = 8192              # 4 Megabytes
        max_lv = 0
        max_pv = 0
        metadata_copies = 0

        physical_volumes {

                pv0 {
                        id = "VGWTyw-dO64-uogu-JKI3-tby4-rVv5-96uFjc"
                        device = "/dev/sda2"    # Hint only

                        status = ["ALLOCATABLE"]
                        flags = []
                        dev_size = 1952497664   # 931.023 Gigabytes
                        pe_start = 2048
                        pe_count = 238341       # 931.02 Gigabytes
                }
        }

        logical_volumes {

                swap {
                        id = "K9zJoo-iCH0-Ir3D-Glgh-dpme-OEAH-D3Xx8R"
                        status = ["READ", "WRITE", "VISIBLE"]
                        flags = []
                        creation_host = "localhost.localdomain"
                        creation_time = 1430196877      # 2015-04-28 00:54:37 -0400
                        segment_count = 1

                        segment1 {
                                start_extent = 0
                                extent_count = 800      # 3.125 Gigabytes

                                type = "striped"
                                stripe_count = 1        # linear

                                stripes = [
                                        "pv0", 0
                                ]
                        }
                }

                home {
                        id = "isFTrt-d7hX-sobo-ROPs-di5u-bska-bJUjlE"
                        status = ["READ", "WRITE", "VISIBLE"]
                        flags = []
                        creation_host = "localhost.localdomain"
                        creation_time = 1430196878      # 2015-04-28 00:54:38 -0400
                        segment_count = 1

                        segment1 {
                                start_extent = 0
                                extent_count = 224725   # 877.832 Gigabytes

                                type = "striped"
                                stripe_count = 1        # linear

                                stripes = [
                                        "pv0", 800
                                ]
                        }
                }

                root {
                        id = "dCzqTE-VkEV-Gt1Y-QHNW-2P4T-YIg7-Wd3or2"
                        status = ["READ", "WRITE", "VISIBLE"]
                        flags = []
                        creation_host = "localhost.localdomain"
                        creation_time = 1430196882      # 2015-04-28 00:54:42 -0400
                        segment_count = 1

                        segment1 {
                                start_extent = 0
                                extent_count = 12800    # 50 Gigabytes

                                type = "striped"
                                stripe_count = 1        # linear

                                stripes = [
                                        "pv0", 225525
                                ]
                        }
                }
        }
}
[root@ohmster backup]#

Code:

[paul@ohmster ~]$ sudo fdisk -lu

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x00020c65

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048  1953523711   976248832   8e  Linux LVM

Disk /dev/mapper/centos_linux--server-swap: 3355 MB, 3355443200 bytes, 6553600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos_linux--server-root: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos_linux--server-home: 942.6 GB, 942564966400 bytes, 1840947200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x01110956

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  1953523711   976760832   83  Linux

Not sure why df does not show the swap file but I will include it anyway.

Code:

[root@ohmster backup]# df
Filesystem                            1K-blocks    Used Available Use Mounted on
/dev/mapper/centos_linux--server-root  52403200 8876828  43526372  17 /
devtmpfs                                1509848       0   1509848   0 /dev
tmpfs                                   1522828      72   1522756   1 /dev/shm
tmpfs                                   1522828    9108   1513720   1% /run
tmpfs                                   1522828       0   1522828   0% /sys/fs/cgroup
/dev/mapper/centos_linux--server-home 920024152 4556176 915467976   1% /home
/dev/sda1                                505580  252836    252744  51% /boot
/dev/sdb1                             961297424   77848 912365152   1% /run/media/paul/IMAGE-BU
[root@ohmster backup]#

Using ssh to gather this info and since I cannot find where "swap" is, I want to run to the Mate desktop and find it with GUI tools. Surprise, x comes up, black screen, but I can rotate the Compize desktop cube and see the cube tops and background. The desktop manager is not working. Thermal issue perhaps or the fact I put in a used nVidia card that may have been pulled years ago because it had a problem. Rebooting. Man that "black face rotating cube" is the strangest thing I've ever seen. Wish I could have got a screencap you you guys. Rebooting now to see if I can find swap space on a GUI tool and give a screenshot.

Ack, Caja File Manager is not working after a reboot "starting Caja" and nothing happens. Terminal does not open. I have other problems, most likely overheated right now. Even samba is not working right. Gets me into the root but the shares are "hanging". Will pull what I can ssh before shutting down.

Disk Utility shows that swap is a 3.4Gb LVM partition. (See screenshot attachment.)

Code:

[paul@ohmster ~]$ cd /dev/mapper
[paul@ohmster mapper]$ ls -la
total 0
drwxr-xr-x.  2 root root     120 May 27 17:12 .
drwxr-xr-x. 20 root root    3560 May 27 17:13 ..
lrwxrwxrwx.  1 root root       7 May 27 17:12 centos_linux--server-home -> ../dm-2
lrwxrwxrwx.  1 root root       7 May 27 17:12 centos_linux--server-root -> ../dm-1
lrwxrwxrwx.  1 root root       7 May 27 17:12 centos_linux--server-swap -> ../dm-0
crw-------.  1 root root 10, 236 May 27 17:12 control
[paul@ohmster mapper]$

That's all she wrote. The machine has been on for 2 days in run level 3 and is hanging up, I got problems. Chances are I will never be able to use this machine.

Reboot to BIOS for Hardware Monitor thermal realdings:
CPU: 63 C or 149 F
That should not disable the machine. The highest recommended CPU temp for this CPU is 70 C. Probably the video card I "reused" is bad. I need find a cheap replacement as it does not pay to spend lots of money that I cannot ever use.

I will deal with the hardware issues. Any help you have for the backup would be great! Thanks rknichols!

EDIT:
Okay, I shutdown the machine, took a shower, and rechecked it. Once it cools for 20 min or so, all is fine. Either CPU cooler or I need a test nVidia video card. There is no build in video on this board. I will deal with it. I got the GUI image of the swap space and posted it as an attachment, HTH!

rknichols · 05-27-2015, 10:38 PM

I think the USB drive actually has enough space to hold the copy. First, your drive is 1000.2 GB (not 1002 GB), and the USB drive is 1000.19 GB. There is 64 MiB of unused space between the end of the last LV and the end of the sda2 partition, so trimming 0.02 GB off the end of the partition shouldn't hurt.

Here is what I'd do. First, on the running system (not booting from a rescue disk) use pvresize to shrink the physical volume to eliminate some of the extra space at the end:

Code:

pvresize --setphysicalvolumesize 1952456704S /dev/sda2

That is 20 MiB smaller than its current size. Now you have some wasted space at the end of the partition. It should be safe to do that -- pvresize will refuse to do the operation if the end of the LV was getting cut off. You do this on the running system so that /etc/lvm/archive will have a record of the state prior to the operation.

Everything else need to be done while booted from a rescue disk or other separate media.

Next, use dd to copy the whole drive to the USB drive, and don't despair when it runs out of space right at the end.

Code:

dd if=/dev/sda of=/dev/sd{X} bs=256K

Replace "{X}" with the correct designation for the USB drive. Be sure you get that right! Copying in the wrong direction will destroy everything on your source drive. (I always look at the output from "cat /proc/partitions" to make sure the current partitioning makes sense for what I believe to be the source and destination drives.)

Now, run "fdisk -u /dev/sd{X}". It will complain that partition 2 extends past the end of the disk. Print the partition table. Then carefully delete partition 2 and re-create it with the same starting location and using the fdisk default for the ending location. Print the partition table again before writing it out and confirm that the starting location is exactly the same as it was before. Then, use the "w" command to write out the partition table and exit.

In order to check this, you really need to disconnect the original drive. What you have is two LVM structures with identical names and UUIDs. With both connected, you have no way to control which one the system will use. With just the USB drive connected, you should be able to mount the XFS filesystems successfully. You might first want to run "xfs_repair -n /dev/mapper/centos_linux--server-root" to verify that all is well with the LV at the end of the disk.

ohmster · 05-28-2015, 12:51 AM

I will reply topside, lot to read. First of all, I LOVE this reply and answer! Brief, to the point, written with confidence.

This took me over an hour to write, I am nervous, a bit scared, and do not have the confidence to do this immediately. I will do it but have some questions first. Take your time, this is not "Mission Critical" so you can take days to answer if you like. I will get your reply. Thank you so much! (If I ever get past this, I will need a restore procedure, if you can help with that. You are pretty smart with this stuff, fellow!

EDIT Thursday, May 28, 2015:
Near bottom post is mention that large USB drives will not allow my older Asus PC to boot when plugged in. I thought perhaps it was the ext4 format as the PC hangs at boot "Auto-Detecting USB Mass Storage Devices". Formatting to ext3 did not make a difference. Only small USB drives allow machine to boot. Linux and Windows mange large disks and large memory well. I don't think this machine can manage large USB drives in BIOS so I will put up with it, plugging in disks when necessary.

You know, it is midnight and almost always I would just go to the machine and get started. I figure an hour or two for this, bed by 2:30, and I'm good. But recent experience has led me to unexpected contingencies that I work through, looking at the clock (Oh man, 3 AM!), keep working (Gawd, 4:30 AM!) and get done by sunup or simply have to stop. (Please excuse me rknichols. I must talk to you like my friend, I find it easier this way.)

Not this time. For one, it scares the crap out of me, but only at first read. Your Senior Member status, and the way you speak, friendly, with confidence, almost in a "matter of fact" way, really helps give me confidence.

I will print this out and take it with me, the machine is in the next room on a LAN Ethernet cable. I will read it, again and again, maybe do some "dry runs" on some steps, and see what questions I have before I am "knee deep in the muck". Let me start the machine so I can gather info by ssh. PuTTY is a marvelous program, doncha think?

I cannot tackle this immediately, I must build up my confidence level a bit first.

Basic Questions:

Question #1:
I want to image the drive, not necessarily the partitions. As the entire reason for doing this is to restore the entire hard disk in the event of a catastrophe. So how would shrinking a partition "reduce the size of the source disk"? Isn't that what these backup programs want? A target disk as big or bigger than the entire drive, partitions and all, to contain the entire source disk? Question continues below CODE:

Relevant drives viewed with parted:

Code:

[paul@ohmster ~]$ sudo parted -l
Model: ATA ST1000DM003-9YN1 (scsi)
Disk /dev/sda: 1000GB
Sector size (logical/physical): 512B/4096B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  525MB   524MB   primary  xfs          boot
 2      525MB   1000GB  1000GB  primary               lvm


Model: Seagate Desktop (scsi)
Disk /dev/sdb: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1000GB  1000GB  primary  ext4

So internal disk Disk /dev/sda: 1000GB when addressed as /dev/sda is the entire disk with both partitions. /dev/sda1 = /boot partition and /dev/sda2 = / (root) partition. Do I have that right? (I think that LVM screws up this simple explanation of disk structure with reference to '/dev/sda2'.)

Question #2:
If this backup is to be made "sector by sector" because nothing can read this unmounted lvm partition, wouldn't that mean that the entire disk media be backed up, regardless of partition amounts and sizes? Even if the target disk is smaller by ONE BIT no backup software will allow me to advance.

I am missing something here, this is jacking me up.

Question #3:
Ghost 11 Will do the job but warns me:

LVM volumes are cloned as physical partitions. This will probably mean you will need to edit your boot loader and fstab entries to get a bootable system. Proceed with Image File Creation? [OK] [Cancel]

Okay that scares the crap out of me, what the heck is Ghost 11 talking about, rknichols?

Checklist:

We are shrinking the root LVM partition to leave unused space at the end of the drive. It should not hurt anything.
Boot to LIVE CD for rest of this process!
Next we are going to use dd to copy the entire internal drive to external USB drive, it WILL run out of space, but this is "okay"?
We are going to use on /dev/sdb fdisk to show the sector units on the USB Drive.
While still in fdisk, we tap the p key to "print the partition table" to screen.
While still in fdisk on USB Drive, we use the d key to delete partition 2, the root lvm partition.
While still in fdisk on USB Drive, we use the g key (Upper or lower case, it makes a difference!) to create a new partition and use the same starting location as the printed table above and fdisk default to end of disk.
Still in fdisk we print the partition table again with the p key to BE SURE start locations are identical.
Still in fdisk, press w key to make it happen and commit changes to the disk.

I do not quite understand this part fully. I get it, one or the other disk plugged in but not both because they will be identical. Unplug the internal drive and what, boot from the USB drive? Then once booted to the newly created, identical backup drive, I can use "xfs_repair -n /dev/mapper/centos_linux--server-root" to fix the lvm partition and make sure it is working well. I don't think there are two dashes in the name like you say. Let me check with mount command:

Code:

/dev/mapper/centos_linux--server-root on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

Okay, you are right. That will create a working backup. How to restore this back to another internal disk would be nice. I would imagine it would be this same procedure in reverse, but how do you do that? I do need the how to restore back to internal hard drive information. But still overwhelmed at this point.

This computer uses an old Asus BIOS and if ANY large USB drive is plugged in, it will not boot. It hangs after the first post screen, before it "Checks for IDE drives". THAT is a serious problem. I have been all through the BIOS for an Asus P5GD1 motherboard and cannot find any reason for this.

Let me test with a small bootable flash drive. I got Hiren's boot disk 14 on a small 8GB flash drive. BIG drives totally stop the boot process. But, maybe it won't if it is a bootable USB disk. Let's find out!

No problem. It booted right to the grub screen. I tried again with boot popup screen and booted off USB2 to Hiren's Boot Disk.

But with either the Passport 750Gb or Seagate 1Tb linux ext4 formatted drives, the PC hangs at:

Code:

Auto-Detecting USB Mass Storage Devices..
Device #01:

And that is all she wrote. I don't think that this Asus board likes ext4 maybe. Suggestions?

* Since this 1TB USB drive is freshly formatted with ext4 and has only a lost+found directory on it, I will format it again in ext3 to see if this PC will allow me to boot with it plugged in. Maybe that will help me get past this "Cannot boot with Linux backup USB drive plugged in" issue. It is 2:00 in the morning, I will continue with the format experiment for booting another day, too late now.

Dude, I GREATLY appreciate your time, experience, knowledge, and PATIENCE with me. It is not going to waste. It took me well over an hour to write this. I do not expect an immediate answer but an answer when you can spare some time and brain power for it. Thank you. Thank you very much.

Quote:

In order to check this, you really need to disconnect the original drive. What you have is two LVM structures with identical names and UUIDs. With both connected, you have no way to control which one the system will use. With just the USB drive connected, you should be able to mount the XFS filesystems successfully. You might first want to run "xfs_repair -n /dev/mapper/centos_linux--server-root" to verify that all is well with the LV at the end of the disk.

Quote:

Originally Posted by rknichols

I think the USB drive actually has enough space to hold the copy. First, your drive is 1000.2 GB (not 1002 GB), and the USB drive is 1000.19 GB. There is 64 MiB of unused space between the end of the last LV and the end of the sda2 partition, so trimming 0.02 GB off the end of the partition shouldn't hurt.

Here is what I'd do. First, on the running system (not booting from a rescue disk) use pvresize to shrink the physical volume to eliminate some of the extra space at the end:

Code:

pvresize --setphysicalvolumesize 1952456704 /dev/sda2

That is 20 MiB smaller than its current size. Now you have some wasted space at the end of the partition. It should be safe to do that -- pvresize will refuse to do the operation if the end of the LV was getting cut off. You do this on the running system so that /etc/lvm/archive will have a record of the state prior to the operation.

Everything else need to be done while booted from a rescue disk or other separate media.

Next, use dd to copy the whole drive to the USB drive, and don't despair when it runs out of space right at the end.

Code:

dd if=/dev/sda of=/dev/sd{X} bs=256K

Replace "{X}" with the correct designation for the USB drive. Be sure you get that right! Copying in the wrong direction will destroy everything on your source drive. (I always look at the output from "cat /proc/partitions" to make sure the current partitioning makes sense for what I believe to be the source and destination drives.)

Now, run "fdisk -u /dev/sd{X}". It will complain that partition 2 extends past the end of the disk. Print the partition table. Then carefully delete partition 2 and re-create it with the same starting location and using the fdisk default for the ending location. Print the partition table again before writing it out and confirm that the starting location is exactly the same as it was before. Then, use the "w" command to write out the partition table and exit.

In order to check this, you really need to disconnect the original drive. What you have is two LVM structures with identical names and UUIDs. With both connected, you have no way to control which one the system will use. With just the USB drive connected, you should be able to mount the XFS filesystems successfully. You might first want to run "xfs_repair -n /dev/mapper/centos_linux--server-root" to verify that all is well with the LV at the end of the disk.