====== MAVIS-SAMMA Integration ====== **by ʻUluʻulu : The Henry Kuʻualoha Giugni Moving Image Archive of Hawaiʻi** ʻUluʻulu uses SAMMA machinery and software for physical inspection of video tapes, and the cleaning and migration of 3/4" U-matic video tapes. We use xml files and csv files for integrating metadata created by SAMMA processes into MAVIS. What follows is a summary of our workflow as it pertains to MAVIS-SAMMA integration. [see [[#overall_workflow|appendix]] for workflow diagram] ===== Accessioning ===== The first thing we do is accession our video tapes, and add basic metadata for each tape. This is done on a MAVIS Workstation, with metadata entered manually through MAVIS Client. As each carrier is created, we paste the carrier item id number into our label printing software, print out an item id label, and put this on the video tape container. When all of the tapes have been accessioned, we export the item list out of MAVIS Client. [see appendix for [[#accessioning1|flowchart]], [[#export_for_samma_prep_stylesheet|export stylesheet]]] ===== Importing .csv file into SAMMA Prep ===== At this point, we have a comma-separated values file with the following fields to be imported into the SAMMA Prep software: tapeid^video_standard^tape_format^stocklength^tapeid_sequence^location^description^author^comments^output_filename ==== Change encoding of .csv file ==== We need to change the encoding of the .csv file to UTF-8, so that it will be read correctly by the SAMMA software. ==== Log into SAMMA Prep ==== Since we use a Virtual Private Network (VPN) connection to access our MAVIS database, we need to log out of MAVIS Client and disconnect from the VPN before starting the SAMMA Prep software. This is because the Prep software needs to connect to the main SAMMA database, which is on a computer on our local network, which becomes unreachable when our workstation is logged into the VPN. After disconnecting from the VPN, we open the SAMMA Prep software log in. ==== Import file ==== SAMMA Prep has a CSV Import tool. We open this tool, load the .csv file, and import the data into the SAMMA database. ==== Input metadata into online spreadsheet ==== At this stage, we also input metadata into an online spreadsheet that is accessible to all staff that needs to manipulate its data. These spreadsheets will be imported into MAVIS further down in the workflow. We currently use Google Docs. Which spreadsheet we use depends on the format of the video tape and how it is destined to be migrated. The main options are the SAMMA post-migration spreadsheet, the Migrating with Final Cut Pro spreadsheet, and the Outside Vendor spreadsheet. For video tapes to be migrated with the SAMMA Solo, we open the .csv file as a spreadsheet and paste the tape ids and file names into our sammaPostMigrationMetadata online spreadsheet. This gives our media specialist a list of everything to be migrated with SAMMA, and a place to add metadata that is not automatically created by SAMMA during the migration process. ==== Mark .csv file as Imported ==== We then move the .csv file into a folder that lets us know that it has already been imported into SAMMA Prep. ===== Physical Inspection ===== At this point, the video tapes we accessioned should exist in the SAMMA database. The next step is going through the SAMMA Prep process, which entails physically inspecting the item and printing out SAMMA barcodes. This is currently done on our SAMMA Laptop Workstation, using SAMMA Prep software. After physically inspecting the video tape, committing the record, and printing/attaching the barcodes, we need to create an xml file. This is not created automatically during the Prep process. We need to re-open the recently committed record, go to the Finish tab, and Save As an .xml file. The correct file name needs to be entered by the operator. ===== Importing Prep XML into MAVIS ===== After a tape has been physically inspected at the Physical Inspection Station, an xml record should exist on the SAMMA Laptop Workstation. These xml files should be copied onto a MAVIS Workstation on a regular weekly basis and uploaded into MAVIS. ==== Retrieve XML files ==== We currently transfer the xml files through our local network, using the Windows file browser on the MAVIS Workstation (although the visibility and accessibility of networked computers is a bit inconsistent). So the file transfer process is basically a drag and drop procedure. After a file has been transferred to the MAVIS Workstation, the file on the SAMMA Laptop Workstation is moved into a special folder marking it as having been moved to the MAVIS Workstation. ==== Import XML files ==== We then open MAVIS Client on the MAVIS Workstation, and import the xml file using a special stylesheet. This creates a treatment record and condition notes for each video tape carrier, using the physical inspection categories and values from SAMMA Prep. [see [[#import_from_samma_prep_stylesheet|appendix]] for stylesheet] **END OF SAMMA Integration Procedures for NON-U-matic VIDEO TAPES** ===== SAMMA Clean ===== We have a 3/4" U-matic SAMMA Clean machine. For these tapes, the next step in the process is cleaning. The cleaning process is done on the SAMMA Laptop Workstation, using the SAMMA Clean software. A clean.xml file is automatically created during the cleaning process. We then import this file into MAVIS. ===== Importing Clean XML into MAVIS ===== This process is similar to the Prep process, however, it is slightly different because the Clean process also creates a clean.pdf file for each tape. So first, we copy the clean XML files onto the MAVIS Workstation computer. Next, we back up the clean.pdf files on our servers. After that, we import the XML file, using MAVIS Client. This creates a clean treatment record and a reference material (pointing to the location of the clean pdf) for each video tape carrier. [see [[#import_from_samma_clean_stylesheet|appendix]] for stylesheet] ===== SAMMA Migration ===== The next step for 3/4" tapes is their digitization. This occurs at our SAMMA Solo Workstation, using SAMMA Solo software. The relevant part of this workflow is the SAMMA operator entering their MAVIS name ID number in the SAMMA record's "Migrated By" section. Each SAMMA video tape record already has the MAVIS information that was imported into the SAMMA database during the SAMMA Prep stage. An XML file is automatically created during the migration (along with the digital video files and a PDF file). ===== Post-Migration Procedures ===== ==== Entering metadata in online spreadsheet ==== There is certain information that we want to attach to the created files that is not included in the Solo XML file. For this we use our sammaPostMigrationMetadata online spreadsheet. We have fields for color type and audio information, examiner notes, examiner user name, and checksum. This is all input during our QC process. ==== Backing up files on Servers ==== After running and recording checksum information, we back up our files on our servers. ==== Importing Solo XML files into MAVIS ==== After backing up the media files, we are ready to import the Solo XML files. === Retrieve XML files === We retrieve the XML files created during the SAMMA Solo migration using Filezilla, since, in addition to housing the SAMMA database, the Solo computer also has an FTP Server. On the MAVIS Workstation, we connect to the ftp server (must be disconnected from the VPN first) and download the Solo XML files. === Import files === This is done on the MAVIS Workstation, using MAVIS Client. This import creates Digital Components and copied from info for each file created with the SAMMA Solo. [see [[#import_from_samma_solo_stylesheet|appendix]] for stylesheet] ==== Importing post-Migration Metadata into MAVIS ==== === Add MAVIS item numbers to the online Spreadsheet === Before importing the metadata that was added to the sammaPostMigrationMetadata spreadsheet, we need to add MAVIS item numbers, so that the data gets attached to the correct records. We need to retrieve the component and carrier item ids for the items that were just created during the Solo XML import. We get these numbers through a MAVIS Client export from an items list window. [see [[#export_for_online_spreadsheet_stylesheet|appendix]] for stylesheet] We then open up the exported file and copy and paste numbers into their proper locations in the online spreadsheet. === Proofread Data === Then we proofread the data. === Import Data === == Convert to .csv file == First we download the online spreadsheet as an OpenOffice spreadsheet. Then we open the file and clean it up, getting rid of extraneous rows. Then we save it as a .csv file with the proper delimiters. == Import into MAVIS == This is done on the MAVIS Workstation, using MAVIS Client. We choose the .csv file and use the convert delimited file option. This import adds checksum info and the examiner notes to the recently created items. [see [[#import_from_online_spreadsheet_stylesheet|appendix]] for stylesheet] ===== Other Things ===== Something to keep in mind is that every time we make a change in SAMMA configuration, we need to change MAVIS codes and import stylesheets. And procedures get slightly more complicated when multiple SAMMA profiles (PRJ's and SXT's) are being used. ===== Conclusion ===== Anyway, that is the end of our MAVIS-SAMMA integration procedures. We start in MAVIS, export data to be imported into SAMMA Prep. In MAVIS, we import XML files from SAMMA Prep, Clean, and Solo. We export data from MAVIS to be input into an online spreadsheet. We then import this spreadsheet as a CSV file back into MAVIS. And that is that. More detailed workflows can be provided upon request. ===== Appendix A: Diagrams ===== ==== Overall Workflow ==== {{:playground:samma_mavis_workflow.png?direct&|}} ==== Accessioning ==== {{:uluulu:accessioningvideotapesworkflow.png?direct&|}} ===== Appendix B: Stylesheets ===== ==== Export for SAMMA Prep Stylesheet ==== tapeid^video_standard^tape_format^stocklength^tapeid_sequence^location^description^author^comments^output_filename ==== Import from SAMMA Prep Stylesheet ==== /VideoTapeCarrier/key/ PE I OD 0 3 2 MO 0 3 2 DS 3 0 TT 3 0 DI 0 3 2 1 CI 2 0 LW 0 3 2 ED 0 2 3 1 TW 0 3 2 PR 0 2 3 1 LS 0 2 3 1 SP 0 2 3 1 // - :: PM12:: PM12:: AM:: AM // - :: PM12:: PM12:: AM:: AM ==== Import from SAMMA Clean Stylesheet ==== /VideoTapeCarrier/key/ CR I // - :: PM12:: PM12:: AM:: AM SC Y URL ==== Import from SAMMA Solo Stylesheet ==== /VideoTapeCarrier/key/ V PC AC IC SAMMA // - :: PM12:: PM12:: AM:: AM VQ VM4 VQ PCI FPD // - :: PM12:: PM12:: AM:: AM WXP SS431 UN8 422 TV I YCbCr 480 720 LP ST 48 2 29.97 H264 TV D YCbCr 480 640 AAC ST 48 2 29.97 DVCP25 411 TV I YCbCr 480 720 LP ST 48 2 29.97 /VideoTapeCarrier/key/ /VideoTapeComponent/key/ SHA-DIGEST preservation03/ access/_acc_1.mp4 intermediate/ URL SS Y metadata/ URL /Work/key/ Y Y ==== Export for Online Spreadsheet Stylesheet ==== Item Description^Component Item Number^Carrier Item Number^Copied From^File Name Int^File Name Acc^Title Number c__int_1.movc__acc_1.mp4 ==== Import from Online Spreadsheet Stylesheet ==== /DigitalComponent/key/ /Work/key/ Y /DigitalCarrier/key/ HKG Examiner Notes: Audio: /DigitalComponent/key/ /Work/key/ Y /DigitalCarrier/key/ HKG Examiner Notes: Audio: /DigitalComponent/key/ /Work/key/ Y /DigitalCarrier/key/ HKG Examiner Notes: Audio: HKGCS HKGCS HKGCS