First, note that my post originally had an error in the pvresize command. I had omitted the "S" (sectors) units specifier. Since the default unit is "megabytes", that would be requesting a huge PV size, provoking a warning from pvresize. When you quoted my message, you picked up the uncorrected version.

Any semi-intelligent imaging tool will ignore space that is neither in a partition nor used for the partition table or boot loader. A dumb tool like dd will complain, but the result will still be fine since the missing space is unused.
/dev/sda2 holds the LVM container, which in turn holds the root and /home filesystems and the swap space.
As I said above, any imaging tool that is aware of partitioning should not complain. All of your partitions will fit.
Your boot partition is the only one where that could be important, and your boot partition is not in the LVM volume. Don't worry about it. It does mean, unfortunately, that every bit of the LVM volume will be copied, including the content of all the free space in the filesystems. C'est la vie.
That would be the "n" key to create a new partition, then you specify the type ("p" for primary), and then the starting location (sector number). (I don't know where you are getting that "g" key from. It is not part of fdisk.)
I would boot from the rescue CD/USB (with neither drive connected, since your BIOS does not like large USB drives), then hotplug the USB drive and see if all looks well there.
The "-" sign is special in the /dev/mapper names. It separates the VG name from the LV name. It is doubled to indicate a literal "-" in the VG name.
Copying the whole disk back with dd would work, with no other changes needed.

Once you have verified that all looks well on the backup drive you just created, go through the same partition table changes on internal drive. From that point on, any imaging tool should work in both directions. Because of the BIOS limitations, you will have to boot that tool without the large USB drive connected, and then hotplug it later.

Inline quoting! You really *did* earn them "senior member stripes", rknichols!

...man, these "older women" Viagra commercial women are BABES! ...sorry. I have a dual monitor setup with TV Tuner card installed on my desktop. That is a distraction I have to pause for. (Single guy and all.)


NOTE: Out of the entire post, this syntax question/confirmation might be THE most important. I have instructions that are well written but if there is an error in the pvrsize command to my live system drive, I NEEED to get this right. I have the entire Post #14 saved as "instructions" and if there is a mistake, can you please correct it by editing and let me know so that I can save a proper copy, rknichols?

Alright, well let's get it right, then. I was "smart" to be "scared". This is the code I have from you. Is this correct or would you please correct it? This is the actual internal active drive we are working on and I do not see a "corrected version".


Agreed. Ghost offered to do it, with warnings we discussed. Acronis offered to do it, but only if I use "Sector by sector". That sounds like backing up blank space, no? HDClone 5 Enterprise Edition will not let me advance after picking a smaller target. Perhaps after shrinking the partition it will become an option.

/dev/sda2 holds the LVM container, which in turn holds the root and /home filesystems and the swap space.

Got it. Not worth a quote box for it.

Yes.

"The GhostLVM volumes are cloned as physical partitions." warning.

So Ghost 11 is "out" because it will copy free space. Trying it will not hurt, just have to reformat or something and there is no time limit on this. But AFTER shrinking the LVM partition. I am using my old server for my current Linux Hosting swap client job anyway. It is just as old and keeps on ticking. Never tried external USB drive or backing it up. So this "Lesson you are giving me" is of utmost importance. Cannot thank you enough my friend.

Where am I getting this g key from? I dunno, let me open an SSH window, run fdisk, and find out.

...oh. It is in the internal help. I missed the command to "add a new partition" altogether and was looking at "new partition table". Sorry 'bout that!



Thanks for the mini-LVM primer. I actually learned all this stuff when I hosed my Red Hat LVM disk, playing with webmin and knowing *nothing* about lvm, but that was SO long ago that I do not remember it, at all. It took me 10 days of intense study and research to actually "fix" the lvm drive and make the contents visible again. I swear, you would think this is an empty hard drive but I *know* my stuff was there, I had to fix the LMV structure to make it visible again. I used dd to save the first section of the drive to a file because the LVM ID numbers and structure were there in a buffer and each repair attempt would push it back and off the buffer! dd captured the LVM ID numbers and names, but not the structure, they got pushed out the back of the buffer. I used a "sample structure" and plugged in the values I got from dd. I just about fainted when I did an ls on the drive and instead of empty, EVERYTHING was back!

Oh yeah! That is an awesome idea. That's what I get for writing this stuff so late at night. I should have thought of that idea but being overwhelmed does tend to impede an individual a wee tad at times.

"Copying the whole disk back with dd would work"

Man I would kill for a sample command with proper syntax. Either for this machine or in general to fit this particular situation. Might you find one off the top of your head my good man? (I actually have and use scripts, command lines, and useful Linux tips from Usenet, from 5 to 20 years ago to this day!) Those Usenet guys were pros like you guys! Here is an example script that I still find *very* useful, from Usenet years ago. Your work will also help me "forever" and be kept. Stanly below is only the top of a very useful python script that takes a pure text FAQ, a HUGE one that I made, with index, and converts it to html, with click-able headers and a real click to section index. Wow, you guys are brilliant, man.


So you recommend doing this procedure twice. Once on the internal drive and then again on the hotplugged USB drive once backup is made to enable any sort of imaging tool to work. Brilliant!

Thank you very much, rknichols. I am in the middle of a time sensitive html hosting server swap out and normally this would be easy. But there is an outdated Gdaddy form mail script to be replaced with something new, with spam control, and instant delivery instead of buffered, and worse, the other site has a LifeType blog I must copy over if possible. I can copy it over but unless the MySQL database is exactly setup right and file/folder perms are set right, I will see "script error" until my eyes glaze over. I did backup the database, whether I can import it and reconnect it to the php blog remains to be seen. But I have to get it done before the grace period ends and I lose my source server.

It might be a while but I will post back, give heaps of praise, and mark this thread as "solved". Goodnight.

That is the corrected version. I corrected it before you had posted your response, but you had already picked up the original version in your quote. Using the original version would have been relatively harmless despite the ominous "You may lose data" warning from pvresize. Running pvresize again with the correct argument, or even without any "--setphysicalvolumesize" argument at all, would have fixed it.

I think you will find that any imaging tool is going to copy the entire LVM physical volume without examining the structures inside. I don't know of any tool that will, when it sees an LVM container, look inside to see what it contains and copy those parts intelligently. If you happen to find one, please let me know.

Ahhh, yes -- the newer GPT-aware version of fdisk. I'm using an older one that does not have that feature.

LVM is actually pretty robust. There are files in /etc/lvm/backup and /etc/lvm/archive that hold the current and older versions of the configuration, and running vgcfgrestore with one of those files will generally repair whatever you just did to mess up the configuration. Those files are a lot easier to work with than a dd dump of the LVM header.

That's just a swap of the "if=" and "of=" parameters in the command I showed for copying the internal drive to the USB drive:Again, be damn sure you get the drive designations correct, else you could destroy your source. BTW, the "bs=256K" is a fairly arbitrary blocksize. For I/O efficiency you want something larger than the default 512 bytes, but once you get beyond 64K or so, the further gains are minimal.

The procedure could have been done just once -- on the source drive. But, the part that is critical to get right is the partitioning change. It is safest to do that on the backup drive and not do the source until you've confirmed that you got it right, and besides, now you have a known good backup.

1 members found this post helpful.

I usually go with "bs=1M" since most drives out there should have at least 1Meg of cache.

In case you want to make it a bit more intelligent you could use this one (found in my toolbox):
What happens:
- GNU dd (disk dump) copies any block device to another block device or file. It's really useful for disk cloning, but its usual invocation isn't as fast as it could be. These settings, or settings like them, often improve copying speed by more than double.
- Piping the input of dd into the output of another instance seems to always improve copying speed.
- /dev/sda refers to your input device, which may vary. Check yours with fdisk -l.
- image.dd refers to the copy stored in the current working directory. You can also use another block device, such as /dev/sdb. WARNING! Be sure you know what you set the output file to! A mistake here could do irreparable damage to your system.
- The entire hdparm subshell sets dd's input block size to the buffer size of the source medium. This also usually improves copy speed, but may need adjustment (see limitations below).
- conv=noerror tells dd to ignore read errors.

Unless combined with the "sync" conversion, that is a really bad idea since everything that follows in the stream will be shifted in the destination by whatever blocksize dd is using. The result will be a thoroughly trashed filesystem. And with the "sync" conversion, you will get bit if any OS hiccup results in returning a short block and subsequently continues with no gap in the stream. You will have inserted zeros where none should have gone, and again shifted everything that follows. That latter case is extremely unlikely, but not impossible.

If you have a disk drive that is giving read errors, you should be using ddrescue to do the copying. It will behave sensibly in the presence of errors, and later try to go back and fill in the gaps.

[EDIT] And BTW, for several of my disk drives "hdparm -i ... | grep BuffSize" returns "BuffSize=unknown". Others return really odd numbers like "15001kB" -- not something I would ever choose as a transfer blocksize.

OMG! I just answered this post, in detail, with inline quoting, and posted it. Now I look for my post and it is "gone" as if it never posted and I cannot find in by going "back" in my browser! Now I have to do it all over again and cannot include the nice chatter I had in the first version.

That is good! I make my own guides by isolating the useful post by clicking on the Post # to open in a new tab, then make a pdf file out of it with working URLs that I can read later or print. If the post cannot be isolated, I save the page and use Dreamweaver to trim it, save it, and covert to pdf. Thank you Nicholes!

I get it. LVM really "does not exist" unless it is mounted and the software can read/write on it. Like a CD.iso file, you cannot do anything with it unless you mount it or burn it to disc. The backup program will "see" the partition and "knows" it contains data. Since the backup program cannot read LVM, it cannot decide what is data and what is blank space, so it backs up the entire thing, sector by sector.

Almost all backup tools can read standard file system types and make those decisions because they are physical and present, do not need to be mounted with tools to read. A backup that could really do LVM intelligently and omit blanks space really would be "A Holy Grail"!

Yep. I wanted the most up to date OS for this machine and even purchased the 64 bit CPU for such and old machine. I started seriously with Redhat and then they dumped the "free stuff" to Fedora. Then Fedora went "bleeding edge" with a new version every so many months. I wanted Enterprise Linux and seriously considered pirating RHEL rhetorically in a public Linux Usenet group, expecting to get my arse kicked for saying it. To my surprise, the members (All experienced sysadmins and veterans) were nice and told me to just get CentOS, same thing as RHEL without the support. Perfect!

That was a LONG time ago when lvm 1 first hit the scene. I could not understand why they did this, but it appeared seamless and worked well. I then read you could combine disks and partitions to make a bigger drive so the benefits became more clear. I love webmin for stuff like cron jobs and decided to peek at the new LVM stuff. You could make the group or volume size bigger or smaller. "Oh great, I can make my disk bigger out of thin air by increasing the group size" (or something like that) and made the change. Since there is no "free lunch", energy, or anything else, this failed miserably and my data disk was now totally blank. I *knew* my stuff was still there as not enough time went by for a format, but I could see nothing, not even lost+found.

There were some repair tools but not knowing how to fix it, I made one or two feeble attempts to recover that failed. I then read each time I attempt this, I push vital group and volume ID information as well as structure off the disk buffer and stopped. I then read how to use dd to capture that buffer to a text file for restore use. I was bummed out because although I did get the ID #'s, I did not get the structure information. I did use the restore tools with an empty LVM skeleton file, plugged in my ID numbers, and applied it to the disk, not expecting much. To my surprise, *everything* came back! That took about 10 days of intense Googling and studying.

I am not as scared now as I was then and your instructions are very clear, explained, and make sense. I cannot thank you enough.

This is *exactly* what I wanted. A working sample command that I can use. I will be super careful for the 'if' and 'of' tags. I really thought that "if" was a condition as used in script files. Input file/Output file. Got it.

This is a great idea. Change the partition size on the system disk, fairly safe, backup to USB, then do the partitioning change on the copy instead of the system disk to be safe. All great stuff!

I am in the middle of doing a dual website host server migration to new host and only have a couple of weeks to get the stuff from one hosting server to another. Normally this is not too big a deal but the form mailer was a bit challenging because I am no longer offered that crappy Godaddy webformmailer.php that buffers the messages and needed to come up with a new form mail script. Worse is the LifeType blog on the 2nd site. It contains almost 10 years of stuff, all php scripts that rely on a MySQL database. I have to see if I can export the data and import it on the new server and do the setup again. LifeType setup used a wizard to check as you go for permissions on files and folders, plus database setup. Then you delete it. I renamed it so I still have it. But will it work? Will it wipe out the entire thing? Do I have to give him a new, empty blog? The client will NOT be pleased about that but I did mention it may be the only option we have. Importing LifeType into Wordpress scripts that I found are 7 years old and the domain they resided on no longer exist and the zip file was not popular enough to propagate to other servers.

So my hands are very full for the next couple of weeks. But there is nothing worse than helping someone, going out of your way to help them, and then not finding out if your help made a difference or so much as get a "thanks" in return. I will NOT do this to you or anyone else. It might take some time but I will do it and mark this thread as solved.

Thanks again rknichols. You are the first member to take me seriously instead of giving me the "Rube Goldberg" rub. *THAT* I really don't need. I have to gather up my help guide and get back to this website now. "I'll be back!"

Make that, "Change the PV size on the system disk..."It's not all that strange to want have an image for disaster recovery as long as that's not your only backup. I don't do that, but then again, I have gone through the exercise of doing a "bare metal" recovery** from my backups and thus know what would be involved.

** Remove the system disk from the machine and replace it with a brand new drive. Then, starting with nothing but your backups and a recovery disk downloaded from the internet, see what it takes to get your system running again. There are generally a few surprises along the way. http://www.taobackup.org/testing.html