Compression questions

[ecookman]

On my EXHDD I need more space for a new backup...as in another 62gb, Its not really as simple as deleting files I don't need or want because I only use my external drive as a backup holder, my 2 towers + my laptop. This time around I don't have enough space. Is there a compression program that can really shrink files very small, I used Winrar on the best setting and it only shrunk the other backups by about 20gb, I need them to go way smaller

Edit - time isn't an issue, It took Winrar almost 30 hours to shrink the other backups...I can wait lol

[DemonWasp]

7zip usually gets better compression (in the 7z format), and bzip2 does too (in the bz2 format). RAR is a pretty awful compression format. However, none of these will compress your files small enough (though all will be faster than WinRAR).

You should consider either a new hard disk, or a smarter backup system (it sounds like you're doing it manually; there are tools that will store backups much more efficiently than you can on your own, usually by storing a "base" file and some smallish "difference" sets).

[ecookman]

my laptop is a manual backup along with an image the other computer backups are all images.

[mirhagk]

Again, if you image the harddrive without any regard to previous images, then you will be wasting space.

Also consider this, how many files are on your desktops and laptop? I'm sure there is music/programs/pictures that are on all three?

Dunno if any back-up programs take that into account, but there's gotta be something that does.

[rdrake]

Compressing most large things is a waste of time anyway. If you're trying to compress media then don't bother, buy more storage space.

If it's text then knock yourself out with one of the ones suggested before. XZ/LZMA are also good choices.

[mirhagk]

Yeah as rdrake said, movies and music are already compressed. Changing their format could help, but any generic compression won't really help.

One thing I can suggest is if you have a camera, look for picture files. Sometimes cameras are retarded with compression, and opening pictures up with GIMP or something, and saving them again can save you like 90% of space, while losing basically no quality. (once my brothers camera produced a 1.5gb file. Yeah that's right 1.5gb PICTURE! I have no idea how it happened, but it was pathetic).

[ecookman]

Multimedia stays uncompressed [because yes it is pointless] on the drive so yeah but I do shoot in RAW/jpeg, so that grinds space [32gb card is almost full again] so I'm sure there's tonnes of space I could save there, and then go to town compressing the heck out of everything, though it doesn't look promising..... *sigh* anyone know of any current deals on WD 1TB externals? 1 doesn't seem to be cutting it >_>. Or does anyone have a review on a redundant backup system, might as well go big or go home.

[mirhagk]

2TB HDD's are relatively cheap nowadays, you can find one for about $50 easily, however you'll need a case.

[chrisbrown]

I highly recommend the following: buy a new 2TB (only $10-20 more than a 1TB) and transfer everything onto it. Get a subscription to CrashPlan Plus: $3/month per system for unlimited backup space. Use it to backup everything locally to the old 1TB, and also to backup essential files online in case of fire/flood/theft etc. You can also use it for version control, i.e. snapshots of your data every X minutes. Trust me, this is very valuable, especially when coding.

This is what I use and it's saved my ass more than once. CrashPlan has a free version as well but the Plus features are easily worth a few bucks a month.

[ecookman]

chrisbrown @ Mon Oct 17, 2011 10:10 am wrote:

snapshots of your data every X minutes. Trust me, this is very valuable, especially when coding.

My god I never thought of that D: hours of my life wasted because undo doesn't go back far enough or the program crashes. O_O thanks!

[Tony]

chrisbrown @ Mon Oct 17, 2011 10:10 am wrote:

i.e. snapshots of your data every X minutes. Trust me, this is very valuable, especially when coding.

This assumes just a single dev branch (so no stable code base) with none of the typical source control features. Why not use an actual source control system such as git?

[rdrake]

Tony @ Mon Oct 17, 2011 8:57 pm wrote:

chrisbrown @ Mon Oct 17, 2011 10:10 am wrote:

i.e. snapshots of your data every X minutes. Trust me, this is very valuable, especially when coding.

This assumes just a single dev branch (so no stable code base) with none of the typical source control features. Why not use an actual source control system such as git?

I second this.

If you want to keep your stuff private, BitBucket offers better plans in that regard than Github and supports Git now too. If you're fine with keeping most of your stuff public, then GitHub is a fantastic option (look into this offer especially).

[chrisbrown]

It's certainly no substitute for proper source control; I use it as more of a supplement. I like being able to go on a tangent for half an hour and know that I have, say, six "restore points" to fall back on, without committing what may or may not turn out to be useless code.

Plus, it can give rudimentary version control to personal documents and anything that wouldn't otherwise go into a repository. I guess code was probably a bad example, but it's nice to have on the occasions you might need it.

[Tony]

Those tangental trips would normally happen on an "experimental" branch. Which could continue going in parallel to main, without needing to be thrown out. A typical source control branch list might look something like:

- v1 stable release
- v2 stable release candidate
- v2 dev
- v2 experimental

Should there happen to be some bug in the core, it could be fixed in v2 and backported to v1 (e.g. v1.1).

Though this is somewhat of an idolized view for long-term / commercial projects. This isn't necessary the approach one might take to school assignments (although I still keep them on GitHub).