Archiving in the 21st Century…..

[Justin]

“Tape archiving is dead!! Long live tape archiving!”

Hello everyone,
Having just installed a new speedy, huge, central data server at our small post house, I am thinking about only doing file archiving for all our jobs. About half our work is still being done at PAL res. Ditching the VTR tape archive is obviously the way of the future.

We have 2 flame suites (2009,1.3 Terabyte stones, HP XW8400), a 12 terabyte x-server and a 30 terabyte LTO4 box (which backs up the whole company).

I know there are numerous ways to get a job off flame and safely filed away, but was wondering how the rest of you (especially the smaller post houses) went about it. We currently file archive directly to USB mounted hard drives hanging off the front of flame. When they get full we print-out the names of the jobs archived on them, stick that list to the side of the drive and put it on the shelf.

With this new x-server having (now) plenty of space, I was planning to file archive directly to it, then back-up that data to the LTO4 (after leaving the archives sitting on the server for a month or so in case of revisions).

One alternative might be to install some 1 terabyte SATA drives directly into the HP xw8400. This I assume would be quicker archiving to than a networked server. Would still need to back-up to the LTO4 of course.

Or maybe just purchase a smaller LTO4 and hang it off the back of the flame?

Does wire tap central help in any of this?

Any thought’s, do’s and dont’s?

cheers JB

[Ramazan]

Maybe it’s not an issue with the latest drives, but there certainly was a problem the last time I looked at this with filing away on a harddisk. If the harddisk is left sitting on the shelf too long without ever spinning up, there is a definite risk that the parts stop working, in particular, the bearings can seize.

I know of a case of someone who filed archives this way, who two years later had to pull something back in and lo and behold the drive was dead. He then decided to check the other disks, and had a problem on around 40% of the disks that hadn’t been touched in more than a year.

Also USB and Firewire drives have had issues with blowing the connection bridge on either or both the harddrive and motherboard, due to power issues.

Cheers

Paul

[Kjell]

i second the recommendation to avoid archiving to hard disk for long-term storage. it seems like a great idea when you think of cost and speed, but you’ll change your tune when one of them fails and you lose a job you need to recover. we looked into it as a viable way of keeping things nearline, but all the anecdotal evidence said that was asking for trouble. archiving to a more stable format (disc, tape) is infinitely safer.

the LTO4 should be super speedy for you; we use it and it seldom takes more than half a day to move a job off/on the framestore through the attached NAS. the bottleneck is usually the gigE connecting the flame/smoke to the network not the NAS itself.

[filip]

We installed a LTO4 one month ago and I’m very happy so far.

[king koret]

So I will stay clear of the USB external drives. Thanks for that info pgill.

Martincito…. Do you archive directly to the LTO4? Or is it via a server?

Cheers Justin.

[filip]

I archive directly to the LT04. Is faster than a digibeta and you keep in the same place the setups and the clips. You can also append projects in the same media. The only bad thing is if you are appending a project and the application crashes the LTO4 tape get corrupted and you can’t open it anymore. What I am doing is archiving two or three project to a tape, close it, delete the projects from Flame, and never write again in that tape.
You can use LTO3 also in the same machine(400 Gigs instead of 800, and cheaper)
You need to buy a couple of extra cables. One goes into the HP and the other is to connect the LTO4 to the HP.
The fxguide member VictorWol sent me an email with a link to these cables if you need them I can post them here.

[Ramazan]

we have also been having corrupt archive problems and we are using sait2….we are moving over to lto 4 in the hope of reducing the problem…but it sounds like it is the same issue when appending to archives.

This didn’t seem to happen with the dtf’s so much…

is their a solution to avoiding these corrupt archives?

paul

[Isaac]

when i hear about these corrupt archives, it makes me feel better about archiving to hard drives. we have a sled we plug sata drives into and copy file archives onto. we have been doing this for over a year now…. haven’t had any problems yet opening old archives. (knock on wood) i’m sure that after 3yrs or more the chance of drive stiction is more threatening, but how long do you really need to keep these archives? when was the last time you opened an archive from 5 years ago? additionally, i’m hopeful it wont be long before we are all archiving to hard drives with no moving parts at all.

tim

[Kjell]

@pgill 25679 wrote:

we have also been having corrupt archive problems and we are using sait2….we are moving over to lto 4 in the hope of reducing the problem…but it sounds like it is the same issue when appending to archives.

This didn’t seem to happen with the dtf’s so much…

is their a solution to avoiding these corrupt archives?

paul

actually we had several DTF2 archives get corrupted. and true to form those are the tapes that have projects that clients came calling on later. DTF2 was a horrible format in my opinion. very slow and expensive.

i think corrupted archives are a danger no matter what format you write to. i recall having resort to the ATOC several times on digibeta archives in the past.

k

[Ramazan]

kevin,

do you reckon the lto or sait tapes would be a more solid archive system if you archive to file and then just write the file to tape in the command line rather than through the autodesk archiving system. (we currently use conform).

[Kjell]

@pgill 25757 wrote:

kevin,

do you reckon the lto or sait tapes would be a more solid archive system if you archive to file and then just write the file to tape in the command line rather than through the autodesk archiving system. (we currently use conform).

that is exactly what we are doing. our Conform system *was* our archive system until we decided having to tether our tape machine directly to a discreet box (that may or may not be supported in the future and that needs frequent software & hardware updates) was a Very Bad Idea.

our IT guys wrote some custom scripts to automate the process and integrate it into a database backend so searching & retrieving archives is a lot easier than it used to be. it leaves all archived jobs on the RAID as long as possible, but when the capacity gets 80% full it automatically deletes the oldest jobs first that have been verified as written to LTO. actually a handy policy because there have been more than a few occasions when a job has suddenly reactivated and i’ve found it still on the archive RAID instead of having to restore it from tape.

k

[Author]

Posts