29 October 2015
Andrew K. Bobo and Robert D. Marcello
Many visitors to the Center for the Study of New Testament Manuscripts’ (CSNTM)website are familiar with the front end of the digitization process. We travel across the world—this past summer we were in Athens, Greece for three months—taking our equipment with us to photograph manuscripts page by page. What most people do not realize is the amount of work that continues after an image is taken, the part of the process we refer to as “post-production.”
Above is an image of the server room where CSNTM stores its archive.
The post-production process of safely storing our data has developed rapidly over the past decade with the advent of newer and better technologies for every stage of the digitization process. The quality of digital cameras has increased exponentially, and so has the size of each image file. As we planned to switch to 50 megapixel cameras last summer, we knew we would significantly increase our storage needs as we increased the quality of every image. We archive files in RAW, JPEG, and TIFF formats. So a single manuscript page digitized with the newest cameras requires more than 425 MB of storage space. Because we have recently committed to capture 150,000 images in 15 months, this demanded a more robust way to manage our data.
The Center purchased and installed a professional-grade server system, which tripled the capacity of our previous server. The system can be scaled to hold over 600 TB of data, ensuring that CSNTM has room to grow for the next decade of expeditions. The new server runs a RAID6 backup protocol. This process is designed to automatically rewrite all data to multiple locations within the system, and would allow for two individual drives to fail without any data loss. The new server also gives us remote access to our holdings, allowing us to run maintenance processes and gain access to our data from anywhere in the world.
The entire process of finding, installing, and maintaining a new server would not have been possible without bringing in some outside help. CSNTM was fortunate to be connected with The Core Technology Group, an IT consultancy based in the Dallas area. They quickly understood our organization’s mission and data storage needs, helping us find a server setup that would be cost-effective, reliable, and scalable in the long-term. They have also answered dozens of emails and phone calls from us along the way.
Finally, CSNTM has found new ways to archive our images in additional locations and formats. Aside from our new server, we also purchased an LTO tape drive and several LTO tape cartridges. In the past, CSNTM used gold-plated DVDs for the purpose of additional backups, but with the rapid expansion of the size of our images, this backup method has become unwieldy and inefficient. It would take thousands of DVDs to store the massive number of images we plan to take during the next few years. LTO tapes, on the other hand, each hold more than 2 TB of data, only cost about $30 each, and last up to 30 years. We are creating multiple LTO tape backups for every image in our collection, in both RAW and TIFF formats, in order to conform with archival best practices.
Preservation begins with capturing beautiful images, but it doesn’t end there. With our new backup protocols, we will be processing over 60,000 images during the next few months. Each image will be converted and stored in three different formats, on two different media, and in three different locations. Though we aren’t in the field digitizing, the work continues as we safely archive our data for generations to come.