Audio-Digitization Starting Out Right Pamela Vadakan California Preservation

Audio-Digitization: Starting Out Right Pamela Vadakan, California Preservation Program Maureen Russell, University of California, Los Angeles Howard Besser, New York University June 28, 2015 Sponsored by: PARS Co-Sponsor: Video Round Table #alcts. AC 15

Audiovisual Digitization: Starting Out Right Pamela Vadakan California Preservation Program

Planning & Implementing Audiovisual Digitization Projects: A Manager’s view from the California Audiovisual Preservation Project Access available handouts at ala. 15. ala. org/sessions/handouts.

californialightandsound. org Access available handouts at ala. 15. ala. org/sessions/handouts.

• How the physical characteristics of a collection impact the ability to reformat • Standards and best practices for audiovisual digitization for both preservation and access • Guidelines for outsourcing • Long-term preservation and access strategies Access available handouts at ala. 15. ala. org/sessions/handouts.

Physical characteristics of a collection can impact the ability to reformat • • • Collection size Arrangement Description Condition Availability of playback equipment *Triage is often necessary* Access available handouts at ala. 15. ala. org/sessions/handouts.

Planning a Digitization Project What do you have? Preservation Needs Assessment Depending on resources available • Collection level • Box level • Item level Access available handouts at ala. 15. ala. org/sessions/handouts.

Preservation needs assessment Collection-level (random sample) For each sample gather: • Collection • Date (range) • Format • Condition (visual inspection) – note red flags • Generation – note if archival or unique • Storage conditions and history (if known) Free collection assessment tools California Preservation Program’s CALIPR (audiovisual and paper-based collections) http: //lib. berkeley. edu/preservation/CALIPR/index. html AVPreserve and Indiana University’s Media. SCORE & Media. RIVERS (audiovisual collections) https: //avpreserve. com/tools/mediascore-mediarivers Access available handouts at ala. 15. ala. org/sessions/handouts.

Preservation needs assessment Box-level (collection guides) Make a basic count to gather: • Collection • Number (by format) • Date (range) • Generation (duplicate vs. original) • Condition (visual inspection) – note red flags • Content Summary (keywords) – paper clues like accession records • Production history (if known) • Storage conditions and history (if known) Free collection guide creation tool Archivists toolkit http: //archiviststoolkit. org/faq#t 20 n 44 Access available handouts at ala. 15. ala. org/sessions/handouts.

Preservation needs assessment Item-level (inventory)* *Do NOT attempt playback for identification! *Enhance description after digitization Minimal fields • Title • Date • Format • Generation (duplicate vs. original) • Maximum size/length/duration (include total number reels/tapes) • Creator(s) • Paperwork/other clues as to content • Copyright status • Condition (visual inspection) – note red flags Free cataloging tools Independent Media Arts Preservation (IMAP) Cataloging Tutorial http: //www. imappreserve. org/cat_proj/tutorial. html AVCC Cataloging Toolkit http: //keepingcollections. org/avcc-cataloging-toolkit Access available handouts at ala. 15. ala. org/sessions/handouts.

Preservation needs assessment Item-level (inventory) Ideal fields • Identifier (call number or unique number) • Location • Silent/Sound; Color/Black and White • Running Speed • Base Type (Nitrate/Acetate/Polyester) • Contributor(s) • Description or Content Summary • Subject Topics and Entities • Production history (if known) • Storage conditions and history (if known) Free cataloging tools Independent Media Arts Preservation (IMAP) Cataloging Tutorial http: //www. imappreserve. org/cat_proj/tutorial. html AVCC Cataloging Toolkit http: //keepingcollections. org/avcc-cataloging-toolkit Access available handouts at ala. 15. ala. org/sessions/handouts.

Identification Be alert for existing description Access available handouts at ala. 15. ala. org/sessions/handouts.

Inspection and condition assessment Visual inspection of magnetic materials can tell you: • If there’s a label look for clues: Generation (“master” vs. dub”) Commercial vs. non-commercial release Field tape vs. complete program • Format is also a clue: Approximate date Professional vs. consumer/institutional • Condition http: //specsbros. com/whitepaper. html Take the tape out of the container: • Check physical container for damage • Check for mold* *If detected stop inspection! • Smell the tape for dirty socks or vinegar… • Check for white powder, crystalline residue and black/brown flakes of oxide indicating breakdown • If open reel, allow outer wraps to hang loose– check for physical distortion and discoloration indicating breakdown Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization Migration Choices – all affect cost: • Selection of best sources • Selection of destination format(s) • Selection of vendor if outsourcing • Selection of equipment if digitizing in-house • Level of restoration desired • Kind of access copies? Consider potential use and users Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization Migration • Analog Source to Digital Files • Pros: • Avoid equipment obsolescence issues • Move from arcane realm of video preservation to much larger world of IT and digital collection management • Broader visibility for the collection: potential online access/research Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization Migration • Analog Source to Digital Files • Cons: • Infrastructure is needed to support large data. 1 TB = 13 hours of SD video • • Long-term data storage costs More choices to be made (storage: servers/HDD/LTO? ) Managing digital files presents difficulties (obsolescence) Need for strategic plan of maintenance (migration and QC) Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization Outsourcing • Good vendors bring: • expertise, equipment • established procedures • guidance like file naming conventions, destination formats, access formats, etc. • Add to out-of-pocket cost, include shipping expenses • Plan to QC new files • Less “real time” staff cost • Estimate digitization costs based upon format and max running time Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization Outsourcing • Clear communication is key to a good vendor relationship! • Guide to Developing a RFP for the Digitization of Audio* https: //avpreserve. com/papers-and-presentations/guide-to-developinga-request-for-proposal-for-the-digitization-of-audio • • Vendor questionnaire http: //eai. org/resourceguide/preservation/singlechannel/pdf/video_qu estionnaire_final. pdf Digitizing Video for Long-Term Preservation: RFP Guide and Template http: //library. nyu. edu/preservation/VARRFP. pdf Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization Expectations for the vendor Once the recording arrives at the vendor: • Vendor photographs recording and container • Vendor inspects, preps and transfers recordings according to your specifications • Treatment only if necessary – vendor should check with you before proceeding • Vendor compiles technical metadata about the transfer with metadata supplied according to your preferred schema It helps to work with a lab that can handle all audiovisual formats Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization File standards • The audio preservation field is more consistent than the video preservation field, settling on the broadcast wave format, 24 bit depth, 96 k. Hz sampling rate for preservation masters. • There is no universally accepted or recommended digital video file format for preservation yet. Some argue there shouldn’t be. Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization File standards There are numerous choices to be made: Software | Hardware | File formats | Metadata There IS one way that is the best for your institution with careful planning and consultation with all stakeholders: Archivists, IT, web programmers, IP consultants, end users *Document every step of the process! Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization File standards General guidelines for preservation formats: • Same or better quality than original • Seek the highest sampling and least compression • Widely supported, open source format • Choose reputable technologies and machineries • Proven track record of use • Consider purpose of reformatting/potential use (for broadcast, digital asset management migration, etc. ) • Consider total digital storage capacity Access available handouts at ala. 15. ala. org/sessions/handouts.

Standards and best practices for audiovisual digitization File standards Digital object “instantiations” • Preservation file – Highest resolution, lossless/uncompressed master file derived from the original source. For long-term preservation. Store offline on hard disk drive or on LTO tape. • Mezzanine file – Compressed file derived from the preservation file. For production use. Store on hard disk drive or locally on servers. • Access file – Most compressed file derived from the preservation file. Can come in several states as a CD/DVD file or as an online/streamfriendly file. For daily use. Store locally on servers or online. Access available handouts at ala. 15. ala. org/sessions/handouts.

File standards Destination formats (create at least two sets for redundant storage) Preservation file • Audio – Broadcast. wav – 24 bit depth, 96 k. Hz sampling rate (approx. size: 1. 3 GB for 40 min. recording) • Video –. mov, 10 bit uncompressed, Quick. Time wrapper. PCM audio track, 24 bit depth, 48 k. Hz (approx size: 102 GB for 60 min. recording) Access available handouts at ala. 15. ala. org/sessions/handouts.

File standards Destination formats (create at least two sets for redundant storage) Mezzanine file • Audio – Broadcast. wav – 16 bit depth, 44. 1 k. Hz sampling rate (approx size: 400 MB for 40 min. recording) • Video –. dv 50, 8 bit, Quick. Time wrapper. PCM audio track, 16 bit depth, 48 k. Hz (approx size: 27 GB for 60 min. recording) Access available handouts at ala. 15. ala. org/sessions/handouts.

File standards Destination formats (create at least two sets for redundant storage) Access file (for online use) • Audio –. mp 3 – 320 Kbps sampling rate (approx. size: 47 MB for a 40 min. recording) • Video –. mp 4 h. 264, Quick. Time wrapper, 1. 5 Mbps sampling rate. Mpeg 2/4 audio track, 16 bit depth, 44. 1 k. Hz (approx size: 102 GB for 60 min. recording) Access available handouts at ala. 15. ala. org/sessions/handouts.

File standards Considerations (all sources): • File name (unique persistent identifier) • Codec/File format • Wrapper • Bit depth and bit rate • File size • Open source vs. proprietary • Required player (software) For moving image sources: • Maintain original aspect ratio • Frame size • Standard (NTSC vs. PAL) • Color space (YUV vs. RGB) IASA/AMIA Survey of Wrappers and Codecs http: //page 2 pixel. org/amia-iasa 2010 Access available handouts at ala. 15. ala. org/sessions/handouts.

Metadata standards Recommended schema: PBCore (Dublin Core adapted by the public broadcasting community- now maintained by WGBH and the Library of Congress. ) http: //pbcore. org *Includes descriptive, rights, administrative and technical metadata about the original source and about the digital file(s). Ideally, also includes notes about the transfer process and QC of files. Access available handouts at ala. 15. ala. org/sessions/handouts.

Quality Control • • • Run checksums (a digital thumbprint of the file) and ensure that the files you receive are exactly what the vendor created (*free tool: md 5 tool) Check technical specifications of files including file format, file naming and directory structure (*free tool: Media Inspector) Play file – check image and sound quality, ideally in its entirety (if staff time permits). Beginning, middle and end is suitable. In your best estimation is the file true to the original source? Is the quality adequate for intended use? (*free tool: VLC open source media player) Confirm that content is what you expected Confirm that metadata is accurate and complete Don’t be afraid to ask the vendor questions. You’re building a relationship! Access available handouts at ala. 15. ala. org/sessions/handouts.

Quality Control Free tools A/V Artifact Atlas http: //preservation. bavc. org/artifactatlas/index. php/A/V_Artifact_Atlas California Audiovisual Preservation Project Quality Assurance steps http: //calpreservation. org/wp-content/uploads/2014/08/CAVPP-Quality -Assurance-web-2014 aug 20. pdf Various AVPreserve tools https: //www. avpreserve. com/avpsresources/tools Access available handouts at ala. 15. ala. org/sessions/handouts.

Long-term preservation and access strategies Digital Repository: Maintaining a comprehensive preservation environment • Infrastructure: Digital object management system and file support/storage • Redundant storage • File naming convention • Descriptive, rights, administrative and technical metadata model(s) • File validation • File integrity (checksums) • Means of providing user access • Quality control systems • Refreshing and migration Access available handouts at ala. 15. ala. org/sessions/handouts.

Long-term preservation and access strategies Storage options • Optical media (CDs, DVDs, Blu-Ray) • External hard drives (HDDs) • RAID storage (off-line and online/servers) • Data tape (LTO) • Cloud Storage estimates Average storage needed for 12 sound recordings: 21. 48 GB Three file states. Assumes maximum running time (40 minutes) Average storage needed for 12 moving image recordings: 1. 23 TB Three file states. Assumes maximum running time (60 minutes) *Assume all sound recordings, max rt, fit 69 titles on each 1. 5 TB LTO 5 tape. Assume all moving images, max rt, standard definition, fit 14 titles on each 1. 5 TB LTO 5 tape. Even more on LTO 6… Access available handouts at ala. 15. ala. org/sessions/handouts.

Long-term preservation and access strategies Storage costs $250 per hard drive (holds up to 2 TB) $35 per LTO-5 Tape (holds up to 1. 5 TB) $100 per LTO-6 Tape (holds up to 2. 5 TB) $400 per RAID (holds up to 6 TB) minimum $0. 01/month for cloud storage (per 1 GB) Additional Resources ARSC Guide to Audio Preservation http: //www. clir. org/pubs/reports/pub 164. pdf California Audiovisual Preservation Project Resources List http: //calpreservation. org/wp-content/uploads/2014/12/CAVPPResources-List-for-Workshops-2014. 11. 19. pdf Access available handouts at ala. 15. ala. org/sessions/handouts.

Thank you! Pamela Vadakan pamelajean@berkeley. edu californialightandsound. org calpreservation. org/projects/audiovisual-preservation Access available handouts at ala. 15. ala. org/sessions/handouts.