• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Noulakaz

The blog of Avinash, Christina, Anya and Kyan Meetoo.

  • Home
  • About
  • People
    • Christina & Avinash Meetoo
    • Avinash Meetoo
    • Christina Meetoo
    • Anya Meetoo
    • Kyan Meetoo
  • General
    • News
    • Mauritius
    • Politics
    • Education
    • Business
    • Travel
  • Computing
    • Apple
    • Linux
    • LUGM
    • Programming
    • Web
    • Technology
    • Knowledge Seven
  • Entertainment
    • Music
    • Movies
    • Photography
    • Sports

Preserving digital memories

23 June 2008 By Avinash Meetoo 18 Comments

Like most (technology-oriented) parents, Christina and I have recorded hours and hours of Digital Video (DV) of Anya and Kyan growing and we’re still doing it now.

Digital Video (DV) requires a lot of space (of the order of 15Gb per hour!) Consequently, we edited the raw footage like mad and discarded all the crap to come up with beautiful movies (for us at least — I guess that most of you will fall asleep if you were ever subjected to them…)

Initially we only had one way to archive the movies: on VCD (No! Not DVD!) The reason is that we didn’t have DVD burners then. Of course, the quality is not great (MPEG-1 / 352×288 / 1150 kbit/s) but we personally do not care. Watching Anya discovering the world for the first time is what is important!

Then we acquired a DVD burner and shortly after we were producing our own DVDs. The quality level rose abruptly (MPEG-2 / 720×576 / around 2500 kbit/s) and we did that for years until H264 came.

H264 is the new MPEG-4 Part 10 video compression standard which has become pervasive since its adoption by Apple, Sony, Nokia and the Blu-ray Disc Association. Consequently, this year, Christina and I stopped creating DVDs and started rendering our movies to H264 at 720×576 and 1500 kbit/s.

The big decision

Two weeks ago, Christina and I decided that having VCDs, DVDs and H264 Quicktime movies on my MacBook was going to hurt us in the long run. I did a little bit of investigation and realized that we had to convert all our VCDs and DVDs to H264. We settled on using HandBrake to do the conversion because, well, it’s excellent and open source.

As we are planning to buy an AppleTV some day, I used the AppleTV preset in HandBrake with the following changes:

  • H264 Video / 720×576 / 1500 kbit/s
  • AAC Audio / Stereo / 48kHz / 128 kbit/s
  • 2-pass encoding
  • Turbo first pass
  • Anamorphic: no
  • Keep aspect ratio: yes
  • Crop: manual
  • Deinterlace: yes, slow.

The result

Christina and I are now the proud owners of 48 home made H264 movies taking 17Gb of space (more than 24 hours.)

We have bought an external hard disk (a Lacie) to store those 48 files and I’ve copied them on the hard disk of one of the desktop computers I have access to. I have also copied them on a second external hard disk I normally use to backup my MacBook. All in all, I’m happy… for the time being.

Amazon S3

I have been looking at Amazon S3 for some time now as a means to keep my most important files in a data-center somewhere. And those 48 movies certainly qualify as my most important files ever! Unfortunately, sending 17Gb of data to Amazon from Mauritius is not practical (and this is one of the most massive understatements I’ve ever made) due to the pathetic bandwidth we have here.

Let’s see, at 128 kbit/s, 13 days would be required to upload all the 17Gb of video. Maybe it’s more sensible paying an air ticket to Réunion Island and doing it from there… Anyway, the cost to keep the data on Amazon S3 would then be:

  • $4.25 (Rs. 113) for the first month (to account for the initial data transfer)
  • $2.55 (Rs. 68) per month thereafter.

This is extremely cheap! One added benefit is that the videos will all be accessible online. Surely, Kyan and Anya will be able to stream the movies from S3 to their mobile phones when they’ll be big enough to have phones :-)

A question

Have you had the same problems? What strategy have you settled on for archiving your precious digital memories?

Filed Under: Apple, Movies, Technology, Web

Reader Interactions

Comments

  1. Eddy Young says

    23 June 2008 at 13:50

    The only files that I preserve are my pictures. I make several copies of them on DVDs in a staggered manner. For example, collections 1 to 35 on DVD #1, collections 15 to 50 on DVD #2, and so on. It’s a poor-man’s RAID, but that will have to do until we get ZFS in Mac OS X, at which point, I will back up to an external HDD — I have seen too many good HDD fail to trust them fully.

    Eddy.

  2. avinash says

    23 June 2008 at 14:33

    I backup my iPhoto collection on an external hard disk and I keep the best online (albeit at a much lower resolution) using Flickr.

    Unfortunately, video files still require too much bandwidth to be stored online for the time being. Of course, I could resize them to, say, 352×288 (VCD size) but then I believe the quality will be degraded too much… I guess I’ll have to wait for some additional years. Meantime, I’ll rely on multiple hard disks… Thanks for the tip on ZFS. It seems that the next version of Mac OS X will be ZFS-based.

  3. Eddy Young says

    23 June 2008 at 15:56

    ZFS looks very promising. If you haven’t read the details, this testimonial from James Gosling should be convincing enough.

    Mac OS X Snow Leopard Server will bundle ZFS apparently, but nothing has been said if the desktop Mac OS X will.

    Eddy.

  4. Eddy Young says

    23 June 2008 at 16:34

    A good backup plan without an equally good restoration plan is worthless.

    Eddy.

  5. carrotmadman6 says

    23 June 2008 at 17:33

    All my photos & videos are on dvd… awaiting to be burned on Blu-ray (when commercially available & viable). :P

    As for online backup storage, I would recommend Xdrive – 5GB & free! ;)

    As for x264, I believe that it won’t be long before it will replace xvid as the main video format.
    I’ve been toying around with MeGui, & the results were great. Half-size of xvid & greater quality. :)
    I’ll check out Handbrake…

  6. avinash says

    23 June 2008 at 20:48

    What do you mean Eddy?

    Files copied to an external hard disk are as easy to restore (just access them) as files found on your internal hard disk. As for Amazon S3, it’s possible to access a file using a URL. I’ll say that restoration is pretty much taken care of.

  7. Eddy Young says

    23 June 2008 at 20:58

    @Avinash: I was referring more to 4lyf’s post than yours, what with the reliable-but-complex ancestry backup method.

    External HDD are convenient but prone to failure, so HDD backups require frequent verification to ensure that the files do not go corrupt without you noticing.

    Eddy.

  8. Patrick Ng says

    24 June 2008 at 02:27

    @Avinash

    Backing up on a remote site is good because in the event of a catastophe (knock on wood), you won’t lose all your invaluable data. However, there is no guarantee that a company won’t go out of business. Zero risk does not exist.

    I am currently evaluating different strategies to backup my files. One way for me to have a remote backup would be to burn my most important data to DVDs and them store a copy at work.

    @Eddy
    You’re right. A good backup means nothing if files cannot be restored properly. Time machines and iBackup failed some of my test in the past. That’s why I don’t trust them and am currently leaning towards Carbon Copy Cloner. You’re also right to say that HDD are fragile. I, too, have seen so many of them went south.

  9. avinash says

    24 June 2008 at 09:17

    Hi Patrick,

    I have been using SuperDuper! for two years now and I’ve been very satisfied.

    It has even saved my ass once when I lost the internal hard disk of my MacBook.

    It is free for basic use. But you only get incremental backups when you register.

  10. Eddy Young says

    24 June 2008 at 13:12

    Am I the only one who hands it all to fate? *grin*

    Seriously, I cannot wait for ZFS to be part of Mac OS X. One cool thing I would be able to do is making a storage pool with my internal and external HDDs as mirrors. Then, whenever I connect the external drive, ZFS will sync it to the internal one automatically.

    Eddy.

  11. Eddy Young says

    24 June 2008 at 13:13

    @Avinash: It looks like Askimet flagged one of my comments as spam wrongly.

    Eddy.

  12. Patrick Ng says

    24 June 2008 at 16:13

    Hello Avinash,
    I tried SuperDuper once, but did not like it very much. It’s funny, because SuperDuper and Carbon Copy Cloner are almost the same, except the first one is not free, whereas the second is. I also like the fact that Carbon Copy Cloner’s writer wants you to only donate when you completely trust the product and it does not block some of the features in the free version (there’s only one version).

    Does SuperDuper do a real incremental backup or a differential backup? CCC calls its backup “incremental backup,” but it seems to be doing a differential backup where it will backup only files that have changed, unless of course, I have not clearly understood the difference between differential and incremental backups.

    @Eddy
    Some like to live dangerously :-) I’ve been pretty serious about backing my data for a long time, but the most important reason is probably of what happened to me in Dec. 2004. Two days after having submitted my last assignments, my HDD died.

  13. Patrick Ng says

    24 June 2008 at 16:13

    @Avinash
    I think Askimet flagged my last comment too.

  14. Eddy Young says

    25 June 2008 at 13:14

    @4lyf: I hand it all to fate, remember?

    Seriously, if the Apple computer burned, I’d 1) buy a new one, or 2) recover the contents of the HDD from anothor ZFS-enabled operating system.

    @Patrick: Most of my works, except my CV and PGP keys, are in the cloud. My code is in CVS. My MP3s can be ripped again or downloaded. Same for movies. There is very little for me to back-up actually.

    Eddy.

  15. selven says

    25 June 2008 at 22:57

    hell you are kinda picky on quality!
    I usually convert everything to VCDs and save em on DVDs, save space and i usually save two copies of everything and back whatever is classified as highly important in several 3 DVDs, 1 on my normal computer, 1 on my file server and 1 copy where i host my website [300GB i’ve got to use that one day or the other] :p.
    if it is even more important than that, then i encrypt it and save it in the above locations.

    since i upload immediately after i find sometihng important [reason being: i am parano and unlucky when it comes to things blowing up], i never really feel the load big.. :p though after that, i never really would want to download what i uploaded, then i would prefer to waste the entire day searching dvds that looking for it on my host.

    definitely, you are sick to be that crazy about quality for backup’ed data.

  16. avinash says

    26 June 2008 at 08:53

    I was referring to “archiving” in my post. Not “backup” :-)

  17. Eddy Young says

    26 June 2008 at 13:26

    @Avinash: Somehow the backup-related issues that are being discussed also apply to archiving.

    I know archiving focuses more on accessibility whereas backup is mostly about safeguarding, but the reliability of the archival media is relevant in both cases.

    Eddy.

  18. David Channe Vy says

    2 July 2008 at 22:31

    I usually achieve all my documents, photos, videos and audio on DVD-Rs. As a safety measure, I make two copies on two different brands. It’s not the most practical and fastest solution, but it works for files less than 4.2GB (or less than 8GB for DVD-DL).

    I tend not to trust hard drives for long term storage. In the past, I’ve had several drives failing after less than a year of moderate usage, leading to gigabytes of unrecoverable data…The prices for mass storage have considerably gone down, and unfortunately so has the quality/reliability in my opinion.

    Just like video, uncompressed audio data and multitrack audio takes an awful lot of storage space; especially when working in stereo/24-bits/48kHz or 96kHz…In such cases, I usually have a second external hard drive solely as backup. After each session, it’s much faster and easier to copy all the contents from my dedicated audio drive to the second drive. I don’t have to worry about synchronizing the data contained on each volume or spliting its contents on multiple discs. At the end of a project, I usually burn them to multiple DVD-Rs (in both native and AIFF format).

    I think the most problematic issue when archiving data is that of changing formats, standards and media in consumer electronics. Will the videos encoded today in whatever current MPEG standard be available in 20-30 years? Will the documents I’ve created in my favorite word processor (or any other proprietary format) in 1990, be readable in 2010? Will files compressed or split to fit multiple discs be still be retrievable in the next ten years? Or will the optical drives we use today be slowly be replaced by solid state drives (SSDs)…just like the audio CDs replaced vinyls and compact cassette tapes.

    Over the last ten to fifteen years or so, de-facto standards and open formats like PDF, RTF, JPEG, WAV, AIFF, MPEG etc…have worked pretty well. Let’s hope the programs we’ll use in the next decades will still support those formats on which we’ve come to rely…

Leave a Reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

Our Personal Websites

Avinash Meetoo
Christina Meetoo
Anya Meetoo
Kyan Meetoo

You may also like

Default ThumbnailSynchronizing two computers using Amazon S3 Default ThumbnailMy next LCD TV, possibly? Default ThumbnailNoulakaz Podcast (16 September 2006) Default ThumbnailChina Video Disc

Random Posts

Nine Gmail Labs features I cannot live without

I have my dream HiFi setup: Audiolab and Elac

I like Haskell a lot...

Recent Comments

  • Memento Mori by Depeche Mode on I have my dream HiFi setup: Audiolab and Elac
  • Avinash Meetoo on High-End Hi-Fi Audio Companies
  • Gerard Chavagnan on High-End Hi-Fi Audio Companies

Archives

  • March 2023 (5)
  • February 2023 (1)
  • December 2022 (1)
  • November 2022 (1)
  • October 2022 (4)
  • August 2022 (3)
  • July 2022 (3)
  • June 2022 (5)
  • May 2022 (5)
  • January 2022 (3)
  • December 2021 (2)
  • November 2021 (1)
  • October 2021 (1)
  • September 2021 (4)
  • August 2021 (2)
  • July 2021 (14)
  • May 2021 (2)
  • April 2021 (4)
  • March 2021 (9)
  • February 2021 (2)
  • January 2021 (1)
  • October 2020 (1)
  • September 2020 (1)
  • August 2020 (2)
  • July 2020 (5)
  • June 2020 (3)
  • May 2020 (5)
  • April 2020 (6)
  • March 2020 (2)
  • February 2020 (2)
  • January 2020 (2)
  • October 2019 (1)
  • September 2019 (2)
  • July 2019 (2)
  • June 2019 (1)
  • May 2019 (3)
  • April 2019 (2)
  • March 2019 (1)
  • February 2019 (1)
  • January 2019 (3)
  • December 2018 (1)
  • October 2018 (3)
  • August 2018 (2)
  • July 2018 (2)
  • June 2018 (1)
  • May 2018 (1)
  • April 2018 (1)
  • February 2018 (1)
  • December 2017 (1)
  • October 2017 (1)
  • September 2017 (1)
  • August 2017 (1)
  • July 2017 (1)
  • May 2017 (4)
  • April 2017 (3)
  • March 2017 (4)
  • February 2017 (5)
  • January 2017 (3)
  • October 2016 (1)
  • September 2016 (1)
  • August 2016 (4)
  • July 2016 (1)
  • June 2016 (1)
  • March 2016 (3)
  • February 2016 (3)
  • January 2016 (1)
  • December 2015 (1)
  • November 2015 (2)
  • September 2015 (1)
  • August 2015 (3)
  • March 2015 (1)
  • December 2014 (1)
  • November 2014 (4)
  • October 2014 (1)
  • March 2014 (2)
  • February 2014 (3)
  • December 2013 (1)
  • October 2013 (1)
  • September 2013 (1)
  • August 2013 (1)
  • July 2013 (1)
  • June 2013 (2)
  • May 2013 (1)
  • March 2013 (3)
  • January 2013 (2)
  • December 2012 (3)
  • November 2012 (4)
  • September 2012 (3)
  • August 2012 (2)
  • July 2012 (3)
  • June 2012 (2)
  • May 2012 (1)
  • April 2012 (2)
  • February 2012 (1)
  • January 2012 (4)
  • December 2011 (2)
  • November 2011 (1)
  • October 2011 (4)
  • September 2011 (2)
  • August 2011 (1)
  • July 2011 (2)
  • June 2011 (4)
  • April 2011 (7)
  • March 2011 (2)
  • February 2011 (1)
  • January 2011 (3)
  • November 2010 (3)
  • October 2010 (1)
  • September 2010 (2)
  • August 2010 (4)
  • July 2010 (2)
  • June 2010 (1)
  • May 2010 (3)
  • April 2010 (4)
  • March 2010 (3)
  • February 2010 (3)
  • January 2010 (5)
  • December 2009 (2)
  • November 2009 (3)
  • October 2009 (1)
  • September 2009 (5)
  • August 2009 (3)
  • July 2009 (1)
  • June 2009 (3)
  • May 2009 (2)
  • April 2009 (7)
  • March 2009 (12)
  • February 2009 (10)
  • January 2009 (5)
  • December 2008 (4)
  • November 2008 (11)
  • October 2008 (6)
  • September 2008 (7)
  • August 2008 (3)
  • July 2008 (8)
  • June 2008 (6)
  • May 2008 (5)
  • April 2008 (7)
  • March 2008 (6)
  • February 2008 (3)
  • January 2008 (6)
  • December 2007 (11)
  • November 2007 (10)
  • October 2007 (7)
  • September 2007 (9)
  • August 2007 (3)
  • July 2007 (7)
  • June 2007 (8)
  • May 2007 (14)
  • April 2007 (11)
  • March 2007 (18)
  • February 2007 (14)
  • January 2007 (15)
  • December 2006 (16)
  • November 2006 (10)
  • October 2006 (7)
  • September 2006 (8)
  • August 2006 (8)
  • July 2006 (6)
  • June 2006 (4)
  • May 2006 (13)
  • April 2006 (10)
  • March 2006 (11)
  • February 2006 (7)
  • January 2006 (14)
  • December 2005 (8)
  • November 2005 (6)
  • October 2005 (7)
  • September 2005 (2)
  • August 2005 (6)
  • July 2005 (2)
  • June 2005 (6)
  • May 2005 (15)
  • April 2005 (12)
  • March 2005 (3)
  • February 2005 (8)
  • January 2005 (3)
  • December 2004 (1)
  • November 2004 (2)
  • October 2004 (2)
  • September 2004 (3)
  • August 2004 (3)
  • July 2004 (3)
  • June 2004 (3)
  • May 2004 (6)
  • April 2004 (10)
  • March 2004 (12)
Creative Commons License This work is licensed by Avinash Meetoo under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 Unported License.