Integration and development within the CCPN framework Expanded

Integration and development within the CCPN framework Expanded from presentation, Paris Oct 2006

• Overview • ARIA 2 integration • DASH integration • CLOUDS integration

Integration options ■ Integration requires lossless exchange of data with the CCPN data standard : ● Direct API access. Application interacts directly with CCPN data structures. Easiest for new applications. ● In-memory conversion. Application loads CCPN data and converts them to its own data structures in memory. ● Format conversion. Application input/output files are converted to data standard via file conversion. Less well integrated than the alternatives ■ Interoperability, or combining applications into a pipeline, requires further coordination.

Integration options 2 The integration options in this presentation will work for applications in any language that has a CCPN implementation. The CCPN applications are written mainly in Python, as are the utility function libraries and GUI widgets presented here. Only Python applications can be developed ‘inside the CCPN framework’ in the manner presented here. Applications in other languages will have only the generated APIs and I/O libraries available.

CCPN Interface Schemes Via Format. Converter Application Proprietary Memory Formatted File Format Converter CCPN XML/SQL In-memory conversion Application Proprietary Model Custom conversion CCPN Data Model Direct API access Application CCPN Data Model Ccp. Nmr Functions CCPN XML/SQL

Development inside the CCPN framework ■ Ccp. Nmr Macros ● Small home-use Python functions ■ Additions to function library ● Functions incorporated in software release ● Community sharing ■ Embedded options ● Extension to Ccp. Nmr application ■ Stand-alone applications ● Built on CCPN libraries and API

Ccp. Nmr Analysis Function Library ■ Assignment ■ Constraints ■ Data Analysis ■ Experiments & Spectra ■ Peaks ■ Structure ■ Spectrum Windows assign. Res. To. Dim(peak. Dim, resonance) Assign a resonance to peak dimension Checks: • Any atoms are of a valid molecule • Isotopes match dimension • Shift is within tolerances • Whether the aliasing is changed Creates: • Covalent links between resonances • Peak annotation label • Updated chemical shift value • Data Model objects

Ccp Stand-Alone App. Template ■ Menu System ■ Project handling ● ● New Load Save Backup ■ Popup template ● Widgets ● Geometry ● Plumbing

Ccp. Nmr Graphical Widgets ■ A library for any developer to use Color. List Pulldown. Menu Scrolled. Matrix Label. Frame Check. Button Label Entry Button. List

Ccp. Nmr Mega Widgets ■ Build them into your own code! ● Scrolled. Matrix ● Scrolled. Graph ● Structure. Frame

Example Applications ■ ARIA 2 ● Structure calculation by iterative assignment of ambiguous restraints ● In memory data model to data model conversion: Python API code ● Constraints, Violations, Structures, Molecules ■ DASH ● ● ● Predict dihedral angles from chemical shifts GUI based on application template Write input and read output of C program Collate reference chemical shifts with structure: Format. Converter Molecules, Dihedral constraints ■ CLOUDS ● ● ● Assignment free structure determinations Python and C modules: Python-C interface Interpret proprietary files

• Overview • ARIA 2 integration • DASH integration • CLOUDS integration

ARIA 2 -CCPN interface Python-Python in-memory conversion Application ARIA XML ARIA Data Model Custom conversion ■ ARIA imports ● Peak Lists ● Constraints ■ ■ ■ Distance Dihedral RDC ● Sequences ● Chemical shifts CCPN Data Model ■ ARIA exports ● ● Peak Assignments Constraints Violations Structures CCPN XML/SQL

ARIA 2 -CCPN interface Python ARIA XML ARIA Data Model ARIA functions & CCPN functions CCPN Data Model CCPN XML/SQL

CCPN object selection

CCPN conversion in ARIA 2 CCPN Nmr. Shift ARIA Chemical. Shift def get. Aria. Chemical. Shift(self, ccpn_object): from Datum import Chemical. Shift if ccpn_object is None: val = None err = None else: val = ccpn_object. value if hasattr(ccpn_object, 'value. Error'): err = ccpn_object. value. Error else: err = ccpn_object. error return Chemical. Shift(val, err)

ARIA 2 functions in CCPN ■ Bundled with CCPN distribution ■ This side of the interface maintained by CCPN ■ Gives Ccp. Nmr Applications access to ARIA projects import. Aria 2 Run. Data(dir. Name, constraint. Set=None, project=None, load. Structures=False) get. Aria 2 Objects. From. Pickle(file. Name) get. Chain. From. Aria 2(restraints, project) get. Structures. From. Aria 2 Dir(dir. Name, chain) get. Constraints. From. Aria 2(aria. Restriaints, chain, constraint. Set) get. Constraint. From. Aria 2(aria. Restraint, constraint. List) get. Fixed. Resonance. From. Aria 2 Atom(aria. Atom, chain, constraint. Set) get. Aria 2 Atom. Set. Pairs(aria. Restraint)

ARIA 2 conversion in CCPN ARIA Violation CCPN Nmr. Constraints. Violation def get. Violation. From. Aria 2(aria. Violation, constraint, violation. List): violation = None if aria. Violation. is. Violated(): calc. Dist = aria. Violation. get. Average. Distance(). get. Value() calc. Error = aria. Violation. get. Average. Distance(). get. Error() frac. Viols = max(0. 0, min(aria. Violation. get. Degree. Of. Violation(), 1. 0)) viol. Value = aria. Violation. get. Upper. Bound. Violation(). get. Value() violation = violation. List. new. Violation(violation=viol. Value, calc. Value=calc. Dist, calc. Value. Error=calc. Error, constraint=constraint, fraction. Violated=frac. Viols) return violation

ARIA 2 calling CCPN functions import ccpnmr. analysis. Constraint. Basic as CB self. message('CCPN export: Accessing data model. . . ') ccpn_project = self. open_ccpn_data_model(project) export = 1 chain = CB. get. Chainfrom. Aria 2(restraints, ccpn_project) if not chain: self. warning(Standard. Error, 'CCPN export: No molecular system found. ') return structures = CB. get. Structures. From. Aria 2 Dir(path, chain) if not structures: self. warning('CCPN export: Unable to load any structures from iteration directory %s' % path) else: self. message('CCPN export: PDB files exported. ') struct_gen = CB. make. Structure. Generation(structures, constraint. Set)

• Overview • ARIA 2 integration • DASH integration • CLOUDS integration

DASH Overview ■ Predicts peptide dihedral angles from chemical shifts ● Similar to TALOS ■ Uses CCPN Python API directly in code ● No real CCPN conversion needed ■ Scientific C code separate from CCPN DASH database CCPN XML DASH Python Tkinter GUI Input File DASH C Output File

DASH Overview ■ Imports ● ● ■ Exports Chemical shifts Reference chemical shifts Reference structures Molecular sequence ● Dihedral angle constraints ■ Values ■ Errors Chemical shifts Molecular sequence BMRB chemical shifts NMR Star format PDB protein structures PDB format Ccp. Nmr Format. Converter & Analysis functions Dihedral angle constraints DASH Python DASH C

CCPN API use in DASH phi. Resonances = (fixed. Resonances[0], fixed. Resonances[1], fixed. Resonances[2], fixed. Resonances[3]) phi. Constraint = constraint. List. new. Dihedral. Constraint(resonances=phi. Resonances) psi. Resonances = (fixed. Resonances[1], fixed. Resonances[2], fixed. Resonances[3], fixed. Resonances[4]) psi. Constraint = constraint. List. new. Dihedral. Constraint(resonances=psi. Resonances) # make items DASHres = self. DASHmol. get. Residue(i + first_res. Num) phi = DASHres. pred_phi[DASHres. index] psi = DASHres. pred_psi[DASHres. index] phi_limit = DASHres. sd_phi[DASHres. index] * 2 psi_limit = DASHres. sd_psi[DASHres. index] * 2 phi. Constraint. new. Dihedral. Constraint. Item(target. Value=phi, upper. Limit =phi+phi_limit, lower. Limit =phi-phi_limit, error=phi_limit) psi. Constraint. new. Dihedral. Constraint. Item(target. Value=psi, upper. Limit =psi+psi_limit, lower. Limit =psi-psi_limit, error=psi_limit) F, Y angle prediction CCPN Dihedral. Constraint

DASH – Python GUI ■ Uses the Ccp. Nmr application template ■ Uses the Ccp. Nmr Python Tkinter graphics objects (widgets) ● App. Template provides menu ● Scrolled. Matrix provides tables ● Scrolled. Canvas used custom graphics

DASH – Python GUI ■ DASH Ramachandran plot used as basis for Ccp. Nmr equivalent ● Development goes both ways!

• Overview • ARIA 2 integration • DASH integration • CLOUDS integration

The CLOUDS Protocol ■ Automated assignment & structure determination ● Miguel Llinas, Alex Grishaev, et al. ● Spatial distribution of anonymous resonances generated with NOEs H H ■ Integrated within CCPN ● ● An Analysis module Data Model glues modules Functional platform Distribution network H H A network of distance constraints between anonymous atoms is sufficient to generate a low resolution protein structure.

The CLOUDS Protocol A family of Clouds

Ccp. Nmr Clouds Spectra ■ Implementation of existing software ● ● ● Interface existing code Re-written code modules Embed in Ccp. Nmr Link. Peak. Lists Pick Peaks, Link Shifts & Combine Spin Systems MIDGE BACUS NOE matrix Relaxation Matrix Optimisation Distance Constraints HClouds. MD Hydrogen Atom Molecular Dynamics Proton Clouds Cloud. Threader Chain Fitting & Molecular Replacement Chain Assignment ARIA 2. 1 Full Structure Calculation Protein Structure

Ccp. Nmr Clouds: BACUS ■ ■ Preserve C code Generate input files Interpret output files Really simple GUI! Resonances Shift. List NOE Peak. Lists Assigned Peak. Lists Input Files BACUS Output Files

Ccp. Nmr Clouds: HClouds. MD ■ FORTRAN -> C ■ Python wrapper ● Control distance constraints ● Cooling scheme parameters ● Dynamics engine Distance Constraints Parameters Resonances HClouds. MD Family of Clouds

CCPN Python using CLOUDS objects def run. Dynamics. Protocol(atom. Coord. List, noe. Constr. List, dynamics. Engine, noe. Force. Field, cooling. Scheme, rp_force_const=1): for (i, temp_f, ncooling, nsteps, tau, rp_scale) in cooling. Scheme: tref = temp_i rpf = rp_force_const * rp_scale if (ncooling > 1): dtemp = (temp_f - temp_i) / float(ncooling - 1) else: dtemp = 0 dynamics. Engine. rp_force_const dynamics. Engine. tref dynamics. Engine. tau dynamics. Engine. nsteps = = rpf tref tau nsteps for j in range(ncooling): dynamics. Engine. run(atom. Coord. List, noe. Constr. List, noe. Force. Field) dynamics. Engine. tref = dynamics. Engine. tref + dtemp return atom. Coord. List

Python-C interface #include "python_util. h" static Py. Object *init_Py_Dynamics(Py. Object *self, Py. Object *args, Py. Object *keywds) { float rp_force_const = 1, beta = 10, rmin = 2. 25, drzap = 2; float tref = 1000, tau = 0. 001, elapsed_time = 0; int nsteps = 1000, nprint = 3000; static char *kwlist[] = { "rp_force_const", "beta", "rmin", "drzap", "tref", "tau", "elapsed_time", "nsteps", "nprint", NULL }; if (!Py. Arg_Parse. Tuple. And. Keywords(args, keywds, "|fffffffii", kwlist, &rp_force_const, &beta, &rmin, &drzap, &tref, &tau, &elapsed_time, &nsteps, &nprint)) RETURN_OBJ_ERROR("need arguments: [ rp_force_const, beta, rmin, drzap, tref, tau, elapsed_time, nsteps, nprint ]"); return (Py. Object *) new_py_dynamics(rp_force_const, beta, rmin, drzap, tref, tau, elapsed_time, nsteps, nprint); }

Python callable functions in C static Py. Object *run(Py. Object *self, Py. Object *args) { Py_Dynamics py_dynamics = (Py_Dynamics) self; Py. Object *atom_coord_list_obj, *noe_force_obj; Py_Atom_coord_list py_atom_coord_list; Py_Dist_constraint_list py_noe_list; Py_Dist_force py_noe_force; Atom_coord_list atom_coord_list; Dist_constraint_list noe_list; Line error_msg; Ccpn. Status status; if (!Py. Arg_Parse. Tuple(args, "OOO", &atom_coord_list_obj, &noe_force_obj)) RETURN_OBJ_ERROR("must have three arguments: atom. Coord. List, noe. Force"); … … py_atom_coord_list = (Py_Atom_coord_list) atom_coord_list_obj; py_noe_list = (Py_Dist_constraint_list) noe_list_obj; py_noe_force = (Py_Dist_force) noe_force_obj; atom_coord_list = get_atom_coord_list(py_atom_coord_list); noe_list = get_dist_constraint_list(py_noe_list); status = run_dynamics(py_dynamics->dynamics, atom_coord_list, noe_list, py_noe_force->dist_force, error_msg);

Popup Constructors and Notifiers ■ Init ● Setup local variables ● Subclass popup window Initialisation Widgets ■ Body ● Arrange Graphical elements ● Set up Data Model notifiers ● Set initial state ■ Update ● Process updated values ● Redraw widgets based on status ■ Widget callback ● From entry, buttons etc ● User functions ● Data Model change User Influence Body Notifiers Update Filter Update External Influence Data Model