Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Hey all-
I'm trying to compress a huge (2.3 GB) .raw file down as much as possible so that when I download it later it doesn't take absolutely forever. I tried using bzip2 as it seems to be more effective (if a bit slower) than gzip, but it only knocked the size down to 2.1 GB. Do you guys know of any tools that might work better than bzip2 for this? Speed of compression isn't a huge deal, as my main hangup at the moment is downloading a bunch of these files at the end. Thanks in advance
Maybe I missed something, but the use of that extension isn't consistent enough for me to know.
Is it some format of digital video? If you are compressing digital video, you want to use a compression method that knows it is compressing digital video.
Additionally, if that file is already pretty random - the most obvious way for this to happen is if the data is already compressed by something else - you're not going to see good gains with anything.
Please allow me to clarify, per the context of the OQ:
Generally speaking, *IF* anything is going to give you better compression than you are getting with bz2 or gz, it would be LZMA, with the exception perhaps being wrt what johnsfine wrote about digital video. I say 'perhaps' because I personally don't know either way about digital video compression.
Do let us know what you use in the end, and/or/if you use something besides bz2/gz, how it turns out. I'm curious now!
Well!
Thanks for the tips guys. For the record, the .raw file is a displacement field generated from comparing 2 sets of lung CT images. It's essentially a giant data file that our software overlays onto the images.
gzip did basically nothing, maybe a 100MB reduction. bzip2 was only a little better, reducing the size by about 200MB. LZMA though... WOW! It reduced the files all the way down to 1.45GB. I'm currently toying around with the settings, though, since at the default quality setting, LZMA took over 3 hours to run!
And apparently I spoke too soon. I just ran LZMA at a reduced quality setting of -4 instead of -7, and it only took 10 minutes. Huzzah!
I'll update this thread if I decide to head down another road.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.