EMEN 2 Steve Ludtke NCMI Baylor College of

EMEN 2 Steve Ludtke NCMI Baylor College of Medicine NCRR

Current EMEN Database • Diverse data requirements: purification data collection reconstruction • OODB (Zope based) • Direct equipment interface • 290 users from dozens of labs • ~7 Tb image data • 2 -3 Tb/year of image data • 385, 000 total records

LIMS • Centralized databases (PDB, EBI, etc. ) vs. in-house archives with detailed information • Scientific Database vs. Electronic Notebook

Film/Frame Id Defocus Mag Total Dose Exp. Time ccd_2004031 4193633 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4193634 #2 2. 5 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4194642 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4194643 #2 2. 5 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4194917 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4194918 #2 2. 5 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4195514 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_2004031 4195515 #2 2. 5 um 138. 44 k 20. 0 1. 0 s ccd_search 1 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_search 2 #2 2. 5 um 138. 44 k 20. 0 1. 0 s ccd_search 3 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_search 4 #2 2. 5 um 138. 44 k 20. 0 1. 0 s ccd_search 5 #1 0. 8 um 138. 44 k 20. 0 1. 0 s ccd_search 6 #2 2. 5 um 138. 44 k 20. 0 1. 0 s • Excellent mineability • Limited flexibility • Good for centralized databases (standards)

Excellent flexibility Rich information content Limited mineability

Goals • BOTH flexibility and mineability • KISS • Database should think like the scientist, not the other way around • Archive detailed experimental protocols • Association of databases

New Concepts • Object Oriented Databases (OODB) • Web Ontologies (semantic web) • Evolving collaborative environment (wikipedia) • XML • Peer to Peer Networking • Blogging

Experimental Protocol Record Descriptive Text Experimental Parameter Experimental Parameter

Parameter Ontology Temperature Specimen_Temperature Ambient_Conditions Ambient_Temperature Ambient_Relative Humidity Grid_Temperature_ Previtrification Ambient_Pressure Grid_Temperature_ Imaging

Protocol Ontology TEM_specimen grid vitrified_grid manually_vitrified_grid_ flash_photolysis negative_stain_grid vitrobot_vitrified_grid holey_carbon_grid quantifoil_grid

manually_vitrified_grid A preprepared #grid was placed in a pair of forceps and loaded into the plunger. $cryogen was preprepared below the plunger (ethane and other cryogens which may become solid are reliquefied using a room temperature copper rod immediately prior to plunging). $grid_volume of specimen was deposited on the front of the grid using a pipette. The grid was then blotted on $grid_blot_side using $filter_paper_type and the plunger was triggered after a $grid_plunge_delay to rapidly submerse the grid in the cryogen. The forceps were then released from the guillotine and the grid was placed in $grid_storage_id in $grid_storage_slot. The grid storage button was then placed in $cryofreezer_id for storage until imaging.

EMEN 2 • • Ease of use (new protocols without DBA) Protocol archival Protocol and Parameter Ontologies (P 2 P) ‘Blogging’ Traceability Workflow Dissemination (mirroring) Data Mining

plot bfactor vs truedefocus where truedefocus is between 0. 1 and 5. 0 and bfactor is between 1 and 1000

Project Microscope Microscopy Session Micrograph CCD Frame Scan Particles

plot bfactor vs truedefocus where truedefocus is between 0. 1 and 5. 0 and bfactor is between 1 and 1000 split by protocol

plot bfactor vs truedefocus where truedefocus is between 0. 1 and 5. 0 and bfactor is between 1 and 1000 split by creator

plot bfactor vs truedefocus where truedefocus is between 0. 1 and 5. 0 and bfactor is between 1 and 1000 split by microscope

Project Microscope Microscopy Session Micrograph CCD Frame Scan Particles

EMEN 2 Status • Core library functional – Berkeley. DB + Python • All EMEN data EMEN 2 • Begun work on Web-based front-end – Apache + Cheetah • P 2 P incorporated into design, but implementation incomplete • Formal XML interfaces (OWL for exchange incomplete)

Acknowledgements • Haili Tu • Runsun Pan Thanks to: National Center for Research Resources Agouron Institute

Relational vs. OODB Tables Tabular Storage Fixed Records No Table-mixing Classes Hierarchical Storage Flexible Records Mixed Class Reports

EMEN Goals • • Project Management Data Archival Data Mining Automation Flexibility Communication with Collaborators Dissemination Portability

plot intendeddefocus vs truedefocus where truedefocus>0 and intendeddefocus>0

Group Address Affiliations City Contact Email Contact Name Fax # Group Name Institution Phone # State Support Sources Website URL Zip Code Microscope CCD Camera Type CCD Model # CCD Serial # CCD Size X CCD Size Y CC CS Microscope Serial # Pole Piece Sensitivity Title Voltage Project Axis Codes Bio-hazard Codes Biomedical Properties Biophysical Properties Biochemical Properties Genetic Characterization Goals of Project Height (of specimen) Keywords for Project Length (of specimen) Mass (of specimen) Particle Diameter Project Description Project Title Sequence Specimen Storage Location (Data) Symmetry (of specimen) Purification Aliquot Freezing Session Buffer Concentration Description Purification Meth. Spec. Stability Storage Condn. Buffer Concentration Date Received Identifiers Received By Storage Loc. Volume Aliquots Used Apparatus Blotting Concentration Frozen By Grid Batch Grids Used Grid Type Hole Size Mesh Size Number of Grids Post Freezing Pre-Treatment Storage Loc. Substrate Prep. Freezing Tech. Vitrobot Parm. Apertures Camera Length Camera Units Condenser CS Develop Time Film Type Freezing Session Magnification Microscope Room Humidity Room Temp. Specimen Temp. Spot Size Voltage Reference Comments User Address Degrees Department Email First Name Institution Last Name Login Name Phone Micrograph Microscopy Sess. Labnotebook Links (web) Notebook Text Structure Factor From Whom Processed Source CCD Anti Blooming Astigmatism Parm. Beam Diameter B Factor Binning Camera Length Camera Units Dose Drift Parm. Energy Filter Exposure # Exposure Time Frame ID Ice Comments Ice Thickness Identifier Intended Defocus Lens Current Magnification Peak S/N Ratio Screen Current Screen Magnification Tilt Angle True Defocus True Magnification (X, Y) Coordinates Zoom Factor Amplitude Cont. Astig. Parm. Beam Diameter B Factor Camera Length Camera Units Contamination Dose Drift Parm. Energy Filter Exposure # Exposure Time Film ID Ice Comments Ice Thickness Illum. Angle Intended Defocus Lens Current Magnification Maximum Res. Micrograph Qual. Peak S/N Ratio Screen Current Screen Mag. Tilt Angle True Defocus True Mag. (X, Y) Coordinates Scan Averaging Fac. Brightness Contrast Exposure Time Parameters Scanned By Scanner Used Scan Step