Скачать презентацию Downlink Segment Test Status S 13 0 Russ Скачать презентацию Downlink Segment Test Status S 13 0 Russ

d7d75c94c51433fcd7428fe705310266.ppt

  • Количество слайдов: 26

Downlink Segment Test Status (S 13. 0) Russ Laher August 29, 2005 S 13. Downlink Segment Test Status (S 13. 0) Russ Laher August 29, 2005 S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (1)

Summary of S 13 D/L work performed l l l l l D/L-group software Summary of S 13 D/L work performed l l l l l D/L-group software changes implemented for S 13. 0 ¤ 99 pipeline-related CRs/AIs (105 for S 12. 0) ¤ 38 database-related CRs/AIs (67 for S 12. 0) ¤ 22 ARs (31 for S 12. 0) D/L segment-testing to check specific CR/AR/AI changes ¤ Small cluster of 16 pipeline-drones (mini “ops-like” environment) ¤ SODB (fresh S 12 database via Ed. Jackson) ¤ Product archiver run on pipeline results for end-to-end verification ISTs doing science verification of pipeline products and testing of SPICE QA team verification of QA sub-system upgrades Test plan documents written by Cog. Es to help I&T Code walk-throughs and training sessions conducted on a weekly basis over the last four months to improve software quality and help I&T Weekly downlink meetings with developers and IST Cog. E. s Many “Advanced Data Products” development team meetings with developers and customers Weekly “Archive Working Group” meetings with developers, IRSA group, SSC deputy manager (L. Storrie-Lombardi), and archive scientist (M. Lacy) S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (2)

What’s New for S 13. 0 l IRS New IRS pipeline to produce background-subtracted What’s New for S 13. 0 l IRS New IRS pipeline to produce background-subtracted images and spectra ((pl. Script. Id 2023) ¤ New IRS pipelines to generate mosaics and point-source extractions for peakup images (pl. Script. Ids 2340 and 2348 for FIF and MOPEX pipelines) ¤ Upgrades to SPICE (various improvements; new spectral rectification IRAC ¤ New IRAC imask product (SOSDL-SIS-PD-3030) ¤ IRAC delay-time calculation extended across AOR boundary ¤ Superboresight creation, archival and _usage_ in pointing transfer to ensure all IRAC channels get best refined pointing in future campaign reprocessings (based on optimal reprocessing of channel 1 and 2 data) ¤ New database table (QA_ptg_refine), schema, store procedures to store IRAC pointing-refinement information on a BCD basis MIPS: Si ¤ All P-masked pixels will be explicitly Na. N'd in final MIPS-24 products ¤ All legitimate 0 -valued pixels in the difference plane of MIPS-24 SUR-mode DCEs will be set to Na. N and propagated to the final BCD ¤ For all MIPS channels/modes: use calibrated CSM_PRED versus Y_spot position to correct for pointing due to unpredictable scan-mirror wobble ¤ All MIPS BCD products will have background model estimate keywords: ZODY_EST, ISM_EST, CIB_EST ¤ Significant enhancements to MIPS-24 offline pipelines. e. g. , user-friendly tutorials and the inclusion of many utility scripts ¤ For the very first time, a document outlining Frequently Asked Questions on MIPS-24 processing. To be used by ISTs, I&T, Cog. Es and the PO. See: http: //spider. ipac. caltech. edu/staff/fmasci/home/sirtfwork/MIPS 24_FAQs. pdf ¤ l l S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (3)

What’s New for S 13. 0 – cont. l MIPS: Ge ¤ Upgrades to What’s New for S 13. 0 – cont. l MIPS: Ge ¤ Upgrades to Ge. RT (offline Germanium Reprocessing Tool) ¤ Fixed source bug preventing clean compile on Linux and Solaris with GCC ¤ Modification to treat Infinity as Na. N in calibration files ¤ Modification so that Na. N's get written to calibration files, not Infinity ¤ Modification to allow namelist to interchange order of Column and Hipass filters ¤ WARMFLAG included in as filtering criterion in caltrans calls ¤ Separate WARMFLAG namelist blocks added l Infrastructure Changes for “Warm” Spitzer Observation Campaigns ¤ New warm. Flag field in campaigns, Mips. Ge Fallback/Metadatabase tables and supporting software ¤ Upgrade to C-API function and database-stored function to read warm. Flag from the database ¤ Modification to job. c to write WARMFLAG to fos 16. fits FITS header Post-BCD Software l ¤ New offline script to create MIPS SED mosaics Bandmerge upgrade ¤ Porting of MOPEX and bandmerge modules to Linux and Mac Os. X ¤ MOPEX GUI development for Solaris ¤ Improved user documentation for mosaicker ¤ Various post-BCD-pipeline bug fixes (e. g. , increased CRVAL precision by one digit) Generic Software ¤ Robust scheme in pointing-transfer to guard against outliers in input Boresight Pointing History File (BPHF; all instruments) ¤ New adaptive-smoothing tool ¤ l S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (4)

What’s New for S 13. 0 – cont. l l l On-the-fly reprocessing (automated What’s New for S 13. 0 – cont. l l l On-the-fly reprocessing (automated pipeline running and product archival) ¤ Database schema, stored procedures, triggers, and perl API functions ¤ Webform for SSC-internal job submission ¤ “Lights out” perl script IRSA Connectivity ¤ Populating new database tables for storing IRSA metadata for BCD and post-BCD products ¤ Creation of JPEG image previews for all primary BCD and post-BCD products and compressed/lossy FITS files for mosaics (all imaging instruments), and GIF spectralplot previews for IRS, in order to support the user quick-look interface in Leopard QA sub-system upgrades ¤ SDAT upgraded to run under proxy apache server ¤ New show. Qa. Ptg. Xfer. cgi CGI script to query QA_PTG_XFER database table and display results ¤ Populating new QA_IRS_REFFRM and QA_IRS_RHFIND database tables ¤ New fields being populated in QA_PTG_XFER database table ¤ Improvements in SDAT image display ¤ Miscellaneous minor improvements S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (5)

Tests Under S 13. 0 l IRS All Sci (BCD&BQD), Dark. Cal, Dark. Ens, Tests Under S 13. 0 l IRS All Sci (BCD&BQD), Dark. Cal, Dark. Ens, Flatcal and peak-up-imaging pipelines; used fallback cal; all data run through archiver ¤ Details in IRS S 13 Test Plan Document MIPS-Ge ¤ Sci (scanmap, photometry, sed, tpm), darks, flats, warm AOR, fine photometry; used fallback cal; all data run through archiver ¤ Details in MIPS-Ge S 13 Test Plan Document MIPS‑ 24 ¤ SUR-mode sci scanmap (slow, med, fast cases); SUR- and RAW-mode photometry; SUR- and RAW-mode TPM; flat-field cal w/ pointing transfer, non-linearity cal w/ pointing transfer; SUR & RAW dark cal w/ pointing transfer ; used fallback cal; all data run through archiver ¤ Details in MIPS-24 S 13 Test Plan Document IRAC ¤ sub-, full-array science and cal pipelines for all exposure modes, frame times, and combinations of internal and external repeats; product records and file system contents checked for data run thru each pipeline; used metadata and fallback cal; all data run through archiver ¤ Details in IRAC S 13 Test Plan Document Post-BCD ¤ FIFcreation; pointing refinement; single_frame_pnt_src_ext; multi-frame pnt-src_ext; mosaic pntsrc_ext; overlap correction; image_interp, outlier; image_reinterp; mosaic_coadd for IRAC, MIPS 24, 70 &160; all data run through archiver ¤ Details in Post. BCD S 13 Test Plan Document QA, SDAT ¤ Loader run for all tested pipelines, SDAT tool run, qa pages and statistics pages brought up ¤ Details in QA/Infrastructure S 13 Test Plan Document ¤ l l l S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (6)

Tests Under S 13. 0 l l Testing of the “lights out” script for Tests Under S 13. 0 l l Testing of the “lights out” script for on-the-fly reprocessing for many req. Keys Testing of SDM’s new channelized product archiving many req. Keys S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (7)

S 13. 0 Functions/Test Data Flows S 13. 0 Test Readiness Review, August 29, S 13. 0 Functions/Test Data Flows S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (8)

S 13. 0 Capabilities Sub-system Capability Ingest Supports DCEs, Pntng, ECSV, Rpt files, Mem S 13. 0 Capabilities Sub-system Capability Ingest Supports DCEs, Pntng, ECSV, Rpt files, Mem Dmps. Sped up, Sig. Events, FEIdelete, Re. Run, Archiver naming, HK compression Pipeline Exec real-time data reduction; reprocessing Instrument pipelines All three instruments science and calibration Calibration Transfer Fallback rules (incl NIT) for all three instruments, three nearest in time rules for all three instruments, blessed status flag, asymmetric discriminant selection, interpolation rule Instrument Performance Monitoring Create/Populate HK tables in the HK DB; web-based viewing and data retrieval; updated menus Quality Assessment Creation and populating of QA tables; SDAT web-based QA inspection; analysis and display functions; support archive release CDF Populating of version controlled CDF files on the pipeline drones; retrieve CDF in pipelines, version tracking in the SODB Imaging post-BCD pipeline Point source extraction, pointing refinement (w/ & w/out 2 MASS, USNO, UCAC, FIRST), tiling, overlap correction, outlier identification and mosaic coadding, bandmerging S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (9)

S 13. 0 Success Criteria l l l l Run, re-run requests that failed S 13. 0 Success Criteria l l l l Run, re-run requests that failed on OPS Interact with the SODB via perl & c-APIs Deposit raw data into the archive file system Create pipeline manifests & ensembles Run manifests through APES Run multiple jobbers on 16 pipeline drones Use CDF to get configuration files Use Cal. Transfer to get calibration data (NIT and/or fallbacks) Execute post-BCD pipelines including pointing refinemnet Put QA statistics into QA DB tables Put processing results in sandbox and set file pointers in DB Use SDAT to access and view results on segment test Get “better” BCDs & post-BCD prods IST sci verification of seg test data & buy off (incremental from IOC) Pass the running of the (channelized) archiver S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (10)

Applicable SISs Calibration l SOSDL-SIS-CL-3000 : darkcal output FITS file, dark image l SOSDL-SIS-CL-3001 Applicable SISs Calibration l SOSDL-SIS-CL-3000 : darkcal output FITS file, dark image l SOSDL-SIS-CL-3001 : lincal output FITS file, linearity model cube l SOSDL-SIS-CL-3002 : lincal output table file, pixel cal data l SOSDL-SIS-CL-3003 : lincal output cmask image l SOSDL-SIS-CL-3004 : darkcal output cmask image l SOSDL-SIS-CL-3005 : pmask definitions l SOSDL-SIS-CL-3006 : Electronic Nonlinearity Calibration File l SOSDL-SIS-CL-3007 : MIPS-Ge stim_latent l SOSDL-SIS-CL-3008 : function loadcalkeywords l SOSDL-SIS-CL-3009 : IRAC-loadcalkeywords version-1 SODB Table l SOSDL-SIS-CL-3010 : darkest cmask l SOSDL-SIS-CL-3011 : Cal. Trans Rule List l SOSDL-SIS-CL-3012 : IRS Umask l SOSDL-SIS-CL-3013 : IRS Lmask l SOSDL-SIS-CL-3014 : IRAC flatcal data cube l SOSDL-SIS-CL-3015 : IRAC flatcal cmask l SOSDL-SIS-CL-3016 : IRS flatcal cmask l SOSDL-SIS-CL-3017 : IRS Omask l SOSDL-SIS-CL-3018 : IRS Flux Conversion Table l SOSDL-SIS-CL-3019 : IRS FOV PSF Table l SOSDL-SIS-CL-3020 : IRS COADD 2 D AMASK l SOSDL-SIS-CL-3021 : IRS COADD 2 D FMASK Updated l SOSDL-SIS-CL-3022 : IRS WAVSAMP TABLE l SOSDL-SIS-CL-3023 : MIPS-Ge cmask Data Management l SOSDL-SIS-DM-3000 : Data Tools l SOSDL-SIS-DT-3000 : l SOSDL-SIS-DT-3004 : l SOSDL-SIS-DT-3005 : l SOSDL-SIS-DT-3006 : l SOSDL-SIS-DT-3007 : Updated profile. tbl (IRS BQD) ridge. tbl (IRS BQD) extract. tbl (IRS BQD) irs_tune. tbl (IRS BQD) New Executive l SOSDL-SIS-EX-3000 : plinexyz. cdf l SOSDL-SIS-EX-3001 : run time environment variables Ingest l SOSDL-SIS-IN-3000 : Instrument Performance Monitoring l SOSDL-SIS-IP-3000 : S 13. 0 Test Readiness Review, August 29, 2005 New Products l SOSDL-SIS-PD-3000 : cvti 2 r 4 output FITS file, 32 -bit real data l SOSDL-SIS-PD-3001 : cvti 2 r 4 output FITS file, dmask l SOSDL-SIS-PD-3002 : muxbleedcorr output DCE FITS file l SOSDL-SIS-PD-3003 : imfliprot output FITS file l SOSDL-SIS-PD-3004 : MIPS Ge bmask l SOSDL-SIS-PD-3005 : imagest output bmask l SOSDL-SIS-PD-3006 : tranhead keyword-translate-only list l SOSDL-SIS-PD-3007 : Tile/BCD association files l SOSDL-SIS-PD-3008 : Mosaic-geometry interface l SOSDL-SIS-PD-3009 : BCD delta file l SOSDL-SIS-PD-3010 : PREDICTSAT s-mask l SOSDL-SIS-PD-3011 : dldeltafile library l SOSDL-SIS-PD-3012 : PRF/PSF map l SOSDL-SIS-PD-3013 : Point source extraction table l SOSDL-SIS-PD-3014 : Detection/Tile/BCD Association File l SOSDL-SIS-PD-3015 : imagest output slope image l SOSDL-SIS-PD-3016 : latimflag l-mask l SOSDL-SIS-PD-3017 : fiducial frame definition parameters l SOSDL-SIS-PD-3018 : mosaic_interpolation table l SOSDL-SIS-PD-3019 : matched-point-source refined position table l SOSDL-SIS-PD-3020 : fiducial frame input parameters l SOSDL-SIS-PD-3021 : bandmerged point source output file l SOSDL-SIS-PD-3022 : bandmerger pointer dump file l SOSDL-SIS-PD-3023 : band-pair registration uncertainties l SOSDL-SIS-PD-3024 : bandmerge statistics l SOSDL-SIS-PD-3025 : MIPS 24 bmask l SOSDL-SIS-PD-3026 : FPGen Command Line Input l SOSDL-SIS-PD-3027 : FPGen SODB Entries file (input) l SOSDL-SIS-PD-3028 : FPGen Template Input file l SOSDL-SIS-PD-3029 : Bandmerge (QA) spectral combination counts l SOSDL-SIS-PD-3030 : IRAC imask Pointing Transfer l SOSDL-SIS-PT-3000 : pixel-scale table file l SOSDL-SIS-PT-3001 : Ptg Refinement Absolute Source List Input Fmt l SOSDL-SIS-PT-3002 : MIPS Scan Mirror Parameters Table l SOSDL-SIS-PT-3003 : Instrument FOV Table l SOSDL-SIS-PT-3004 : Super Boresight Pointing History File l SOSDL-SIS-PT-3005 : Refined DCE Position Table File l SOSDL-SIS-PT-3006 : Aperture-ID-to-name translation CDF table Quality Assessment l SOSDL-SIS-QA-3000 : QA file (*. qa) format Russ Laher (11)

Pipeline-Related CRs/AIs (99) 6576 6578 6845 6844 6545 6595 6565 6608 6254 6246 6604 Pipeline-Related CRs/AIs (99) 6576 6578 6845 6844 6545 6595 6565 6608 6254 6246 6604 6697 6403 6625 6655 6533 6698 6529 6715 6839 6416 6675 6594 6848 6674 6822 6693 6692 6805 downlink downlink-analysis downlink-irac downlink-irs downlink-irs downlink-mips-24 downlink-mips-ge downlink-pl-cal Hesselroth_Ted Laher_Russ Hesselroth_Ted Fowler_John Hesselroth_Ted Pesenson_Meyer Lee_Wen-Piao Khan_Iffat Narron_Bob Makovoz_David Narron_Bob Narron_Bob Masci_Frank Makovoz_David Masci_Frank Henderson_David Laher_Russ Wheelock_Sherry Henderson_David Lee_Wen-Piao In libtblio build, skip soft link step. . . In libmathc 90 build, skip soft link step. . . Parchiver. For. Dnl bug fix Update to D/L segment-test build script Tblio null filepointer bug Have cvti 2 r 4 return an exit code of "1" for MIPS Ge stims Move shared library suffix definitions to platform-dependent makefiles Release adaptive smoothing tool Creation of imask, final product mask file Skydark Delaytimes within the Header Change needed in CRVAL precision Update BCD extraction wrapper: w_irs_bqd_process 2. pl New IRS Post-BCD Peakup Imaging pipeline Coadd 2 d to pass bits from BMASK to AMASK Do not perform a simple average of RA_SLT, DEC_SLT and PA_SLT IRS background subtracted images and spectra Update w_irs_bqd_process. pl to include call to "Load. Irsa. Ptg. Meta. Data". Add header card for cal_set ID Ensure that all p-masked pixels are Na. N'd in final mips 24 BCD products. Do source extraction for MIPS-24 on mosaic, not individual BCDs Make 0 -valued pixels in raw difference frames Na. N's for bcd pipeline Add warmflag keyword to caltrans calls Allow changing the order of MIPS-70 filtering Slopecal hipass filter does not always produce reliable results Write new WARMFLAG FITS-keyword to fos 16. fits Upgrade MIPS calibration-observation database Add "WARMFLAG" to calkeywords input-keyword-list for MIPS: Ge Mod. to calkeywords library for new "warm. Flag" field Caltrans. h modification S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (12)

Pipeline-Related CRs/AIs (99) – cont. 6571 6572 6765 6642 6789 6680 6682 6723 6725 Pipeline-Related CRs/AIs (99) – cont. 6571 6572 6765 6642 6789 6680 6682 6723 6725 6726 6727 6756 6764 6599 6741 6670 6777 6640 2597 1891 6621 6525 6632 6549 6550 6553 6630 6555 downlink-pl-cal downlink-pl-exec downlink-pl-exec downlink-pl-exec downlink-pl-ing downlink-pl-prod downlink-pl-prod downlink-pl-prod downlink-pl-prod Fang_Fan Laher_Russ Makovoz_David Makovoz_David Laher_Russ Masci_Frank Fowler_John Henderson_David Lee_Wen-Piao Makovoz_David Khan_Iffat Masci_Frank Hesselroth_Ted Khan_Iffat Hesselroth_Ted Masci_Frank Wavsamp module output signed offset files Wavsamp module pixel sizes need to be changeable APES static-file changes for new IRS "reffr" and "mopex" threads Rename fif. tbl to mosaic_fif. tbl in mosaic. pl, mosaic_mask. pl, . . . APES static-table changes for new IRS pl. Script. Id=2023 Req. Mode=111 not recognized by create. Ensembles. pl New offline script to create MIPS SED mosaics Update w_bqd_run_overlap. pl not to create empty offsets tables Modify w_bqd_run_fiducial_frame. pl to accomodate IRS peak-up imaging Modify create. Fif. Table. pl to accomodate IRS peak-up imaging Modify get. Fif. Table. Filename. And. Status in Wrapper. Utils. pl to include IRS "Lights out" script for on-the-fly reprocessing APES static-file changes for new Onthefly job-manifests Add "refined. Pointing" file. Type to subroutine "get. File. Type. Abbreviation" Code walk-through fixes/updates to "Load. Irsa. Ptg. Meta. Data" perl subroutine. New product. Types table entries for preview images for IRSA connectivity Update w_bqd_run_mopex. pl for IRSA previews and metadata-generation Improve imagest data-reduction logic and chi-square computation Add photometric nonlinearity module to MIPS: Ge pls Module needed to flag "banding" effect in IRAC channels 3 & 4 Add call to new subroutine "add. QA_ptg_refine" in w_bqd_pointrefine. pl New module to replace data in a FITS file by a given value Add subroutines "Load. Irsa. Ptg. Meta. Data. Table()", . . . String literal treated as character array in nl_and_params. c New module to combine coverate weighted images Multiply defined global variables in irs modules New libraries/modules to create new downlink products to support IRSA Register some pointing-refinement (IRAC) keywords into the DB. . . S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (13)

Pipeline-Related CRs/AIs (99) – cont. 6620 6584 6840 6593 6681 6384 6679 6427 6694 Pipeline-Related CRs/AIs (99) – cont. 6620 6584 6840 6593 6681 6384 6679 6427 6694 6728 6612 6438 6733 6468 6734 6735 6737 6740 6818 6607 6437 6554 6710 6600 6100 6517 6752 6781 5706 downlink-pl-prod downlink-pl-prod downlink-pl-prod downlink-pl-prod downlink-pl-prod downlink-pl-ptg downlink-pl-ptg downlink-pl-qa Laher_Russ Narron_Bob Khan_Iffat Masci_Frank Khan_Iffat Lee_Wen-Piao Masci_Frank Hesselroth_Ted Makovoz_David Hesselroth_Ted Khan_Iffat Masci_Frank Fowler_John Khan_Iffat Masci_Frank Masci_Frank Hesselroth_Ted Jacobson_Jeff Laher_Russ Please add new subroutine "add. QA_ptg_refine" to Wrapper. Utils. pl Libim should optionally not to convert all non-finite numbers to Na. Ns New mode in mosaic_outlier to create minimum value mosaic Write background model estimate keywords: ZODY_EST, ISM_EST, CIB_EST New module to compute FIF for MIPS 70 SED mosaic Remaining code walk-through action items for latimflag module. . . Update w_pointinghistory. pl to add calls to "Load. Irsa. Ptg. Meta. Data" Tblio/dlimage duplicate symbol errors Update w_bqd_run_mopex. pl to include call to Load. Irsa. Ptg. Meta. Data. Port of mopex to OS X New module to compute the covariance matrix in a set of images Use calibrated CSM_PRED to correct for MIPS pointing Update cvt. R 4 i 2 module to report MIN, MAX pixel values to stdout. Upgrade/bug fixes to bandmerge New library and modules to create of lossy FITS images for U/L. New module to robustly compute image noise. . . Linux and OS X ports of bandmerge Bug in tblio related to delete_rows processing Update pointing transfer thread to write ORIGINAL pointing keywords Update get. PH_online for "refined. Pointing" file. Type (SBPHF) queries. Come up with scheme to archive and register superboresight PH files. . Implement robust scheme in pointing transfer to guard against outliers Update Gen. SBPHFs. For. Campaign. pl following comments from code walk-thru. . Update Gen. SBPHFs. For. Campaign. pl to optionally ingest SBPH files Update pointing transfer to optionally use new SBPH files Allow "Frame" window to be able to close in SDAT Improvements for SDAT image display Add qa_irs_rhfind table for qaloader Write SDAT. log to $SOS_LOGS/cgiout directory S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (14)

Pipeline-Related CRs/AIs (99) – cont. 6564 6774 6769 6799 6751 6783 6808 6736 6479 Pipeline-Related CRs/AIs (99) – cont. 6564 6774 6769 6799 6751 6783 6808 6736 6479 6775 6129 1577 1732 downlink-pl-qa downlink-pl-qa downlink-pl-qa sos sys-engring-i+t Laher_Russ Narron_Bob Jacobson_Jeff Masci_Frank Wheelock_Sherry Narron_Bob Laher_Russ Wheelock_Sherry Jacobson_Jeff Laher_Russ Bennett_Lee Yu_Felix Dqa_deploy. csh: check to see if SOS_LOGS is defined Qaloader change for the new QA_irs_reffrm database table New qa_irs_reffrm database table Make association of refined ptg tables with BPHFs more robust Allow SDAT to be displayed in the proxy server environemnt Qaloader change for the new QA_irs_rhfind database table New SODB_Utils. pl function SODB_Get_Qa. Ptg. Xfer QA scripts to dynamically read PERL_PATH from environment Display some fields in table qa_ptg_xfer New database stored procedure add. QA_irs_reffrm Change QA primary proxy URL Super-Boresight Test Plan and operational instructions for S 13. 0. MIPS-24 S 13. 0 Test Plan. . S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (15)

Database-Related CRs/AIs (38) 6660 6701 6339 6763 6696 6677 6782 6784 6785 6787 6795 Database-Related CRs/AIs (38) 6660 6701 6339 6763 6696 6677 6782 6784 6785 6787 6795 6798 6402 6800 6434 6802 6807 6809 6813 6816 6678 6631 6743 6615 6712 6747 6749 6753 archive-database sdm-archive sdm-database sdm-database sdm-database sdm-database sdm-database sdm-database sdm-database Jacobson_Jeff Jacobson_Jeff Jacobson_Jeff Laher_Russ Jacobson_Jeff Jacobson_Jeff Jacobson_Jeff Update SPs to support the archive by channel Add software. Version column in processing. Version table Be able to archive on an channel basis New pl. Scripts DB records for IRS "reffr" and "mopex" threads Add warm. Flag to C-API-function sdm_get. Dce. Rec. For. DP (sdm_sodb. ec) New "warm. Flag" boolean field in DB campaigns table New database stored procedure add. QA_irs_rhfind Mod. to database procedure delete. Non. Archived. Products New IRS background-subtraction pipeline (pl. Script. Id=2023) Database SP to compute new cross-AOR delaytimes from r_sclk Perl API function for create. Pv. Ids. For. On. The. Fly. Reprocessing New database stored function get. Qa. Info. For. Request New DB stored function for calculating delay time Perl API for compute. Irac. Delay. Time. Between. Successive. Requests New perl API-function compute. Irac. Delay. Time New and modified stored procedures to support OTF request archiving New DB stored function get. Qa. Ptg. Xfer. Data. For. Request() New perl API function get. Qa. Ptg. Xfer. Data. For. Request() Need schema and perl API update for new columns in QA_ptg_xfer table. Need stored function to query QA_ptg_xfer table for input dce. Id. New "warm. Flag" boolean field in MIPS-Ge cal. DB tables Update campaign stored procedures for new "warm" MIPS campaign. Perl APIs for on-the-fly reprocessing New "QA_ptg_refine" table requested by IRAC IST. . New records to be deleted in DB SP delete. Non. Best. Products New database SP create. Pv. Ids. For. On. The. Fly. Reprocessing() New database tables for on-the-fly reprocessing New database fields for on-the-fly reprocessing S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (16)

Database-Related CRs/AIs (38) – cont. 6754 6695 6627 1719 1720 1695 1716 1667 1664 Database-Related CRs/AIs (38) – cont. 6754 6695 6627 1719 1720 1695 1716 1667 1664 1727 sdm-database downlink sdm sdm sos Jacobson_Jeff Laher_Russ Bennett_Lee Jackson_Ed Bennett_Lee Laher_Russ New database stored procedures for on-the-fly reprocessing Return warm. Flag in DB stored function get. Dce. Profile Schemas, sql loading stored procedures and perl-API for IRSA metadata New product. Types entries for RSA-related products Change exposure type for several MIPS fine scale flat observations New ensemble-creation rules for new IRS "reffr" and "mopex" threads New IRS ensemble-creation rule. Id=8 for pl. Script. Id=2023 Data-content change in Product. Types database table New data content for Product. Types database table Update release level in product types table for 7157 S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (17)

ARs (22) 6575 6583 6577 6773 6748 6833 6713 6758 6122 6431 6705 6503 ARs (22) 6575 6583 6577 6773 6748 6833 6713 6758 6122 6431 6705 6503 6704 6501 6820 6827 6582 6563 6838 6750 6770 6755 downlink-irac downlink-irs downlink-mips-ge downlink-mips-ge downlink-mips-ge downlink-pl-prod downlink-pl-qa sdm-database sdm-pr-archiver Hesselroth_Ted Fowler_John Hesselroth_Ted Lee_Wen-Piao Narron_Bob Henderson_David Henderson_David Pesenson_Meyer Henderson_David Fowler_John Makovoz_David Hesselroth_Ted Jacobson_Jeff In libim build, skip soft link step. . . Pick does not display Infinity values. . . In libintegrate build, skip soft link step (conditional) Darkdrift module change INSBPOSDOM writes no processing history to header RIDGE not reading FOV file EXTRACT module: improper dependency of table output on input header Slopecal mistreats calibration files with Na. N values Slopecal module generates fitsio errors under certain conditions Ge_STIMEXTRAP bmask bit Typo in exptype to cdf block variant lookup for exptype="dfl" Hardcode interp module to use geb_STIMEXTRAP Mipsflat needs command line option for coverage map file name Mips_ge_mask. h MIPS-160 exptype=FINE doesn't exist Downlink/include/nanvalue. h does not produce linkable code. . . Na. N's in a data file turn into "Infinity" when read in by fitsio The pick utility has a display error for unsigned integer. . . Apex_1 frame. pl can not handle images > 10000 pixels in one dim SDAT can not display MIPS channel 3 DCE More robust get. Cal. Products. . . database stored procedures Get. Cal. Products. For. Request need to return int 8 type for file size S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (18)

Major issues to be tackled in S 14 l l l l Recent bug Major issues to be tackled in S 14 l l l l Recent bug found in post-BCD position-refinement pipeline: the QA_IRAC_REFINE database table is not being populated! Bandmerge pipeline wrapper upgrade so it can be executed in routine operations Upgrades to MOPEX GUI Porting MOPEX GUI to Linux and Mac Os. X Bandmerge GUI development Porting bandmerge GUI to Linux and Mac Os. X More user-friendly e-mail notification from on-the-fly reprocessing Improved user documentation for point-source extractor More work needs to be done to make before/after pointing-refinement performance data more accessible (need to store it in the database) Least-square stim interpolation for MIPS: Ge TPM MIPS: Ge pipeline that uses self-generated darks Multi-threaded Ge. RT port to Mac Os. X S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (19)

Sample of new on-the-fly reprocessing webform S 13. 0 Test Readiness Review, August 29, Sample of new on-the-fly reprocessing webform S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (20)

Sample output from new on-the-fly reprocessing script sscpipe 14% $SIRTF_SCRIPTS/otf. Reprocessing. pl -o 20050812 Sample output from new on-the-fly reprocessing script sscpipe 14% $SIRTF_SCRIPTS/otf. Reprocessing. pl -o 20050812 T 1825. out otf. Reprocessing. pl, Version 1. 0 By Russ Laher (laher@ipac. caltech. edu), Ron Beck (beck@ipac. caltech. edu), and Jeff Jacobson (jdj@ipac. caltech. edu). Copyright (C) 2005 Spitzer-Space-Telescope Science Center (SSC), California Institute of Technology, Pasadena, CA Script for "lights out" on-the-fly reprocessing. Database, role = sodb_dnl 5, pipelineswrole Date/time = Fri Aug 12 18: 22: 31 2005 Open database connection. . . List of unique req. Keys: Checking (req. Key, chan. Num) pairs for current status. . . Campaign. Id 662 708 708 Instrument MIPS IRSX Req. Key 7120384 4980736 Chan. Num 1 1 2 3 Reprocess. Flag 1 0 0 1 List of pending on. The. Fly. Reprocessing records: Proc. Id User. Id 54 54 54 55 6 6 E-mail russlaher@msn. com S 13. 0 Test Readiness Review, August 29, 2005 Instrument IRSX MIPS Req. Key 4980736 7120384 Chan. Num 1 2 3 1 Russ Laher (21)

Sample output from new on-the-fly reprocessing script – cont. Deleting records from WIP tables. Sample output from new on-the-fly reprocessing script – cont. Deleting records from WIP tables. . . Loading WIP tables. . . Making new pv. Ids. . . Aug 12 18: 22: 32: Querying WIP tables. . . Campaign. Id 662 708 708 Proc. Id Instrument MIPS IRSX User. Id 54 54 54 55 6 6 Req. Key 7120384 4980736 Chan. Num E-mail russlaher@msn. com 1 1 2 3 Reprocess. Flag 1 0 0 1 pv. Id 1055 0 0 1056 Instrument IRSX MIPS Req. Key 4980736 7120384 Chan. Num 1 2 3 1 Aug 12 18: 22: 32: setup manifests. . . Aug 12 18: 22: 32: create campaign_reproc thread entries. . . dnl. Access -> set. Pl. Hopper 54 dnl. Access -> set. Pl. Hopper 55 Aug 12 18: 22: 34: monitor pipeline processing. . . checking progress of reqkeys. . . Onthefly-IRSX-3 4980736 has not started yet. . . Onthefly-MIPS-1 7120384 has not started yet. . . Aug 12 18: 22: 34: sleeping 300. . . checking progress of reqkeys. . . start: Aug 12 18: 27: 13 Onthefly-IRSX-3 4980736 2017 stop: Aug 12 18: 27: 14 Onthefly-MIPS-1 7120384 1030 Aug 12 18: 27: 35: sleeping 300. . . dnl. Access -> set. Pl. Done 54 dnl. Access -> set. Pl. Done 55 Aug 12 18: 32: 35: End pipeline processing. . . S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (22)

Sample output from new on-the-fly reprocessing script – cont. Aug 12 18: 32: 35: Sample output from new on-the-fly reprocessing script – cont. Aug 12 18: 32: 35: Begin product archiving. . . Setting archive. Campaign flag for pv. Id = 1055 Setting archive. Campaign flag for pv. Id = 1056 Aug 12 18: 32: 35: Polling to see if product archiving has completed. . . Certified flag for pv. Id = 1055 is equal to 0 Certified flag for pv. Id = 1056 is equal to 0 Certified flag for pv. Id = 1055 is equal to 1 Certified flag for pv. Id = 1056 is equal to 1 Aug 12 18: 35: Terminating OTF-reprocessing episode. . . Aug 12 18: 35: The following e-mail notifications were sent: russlaher@msn. com -> New Spitzer products are available for req. Keys: 4980736 7120384 Aug 12 18: 35: Close database connection. . . Date: Fri, 12 Aug 2005 18: 35 -0700 (PDT) From: Rick Ebert To: laher@ipac. caltech. edu Subject: SIG EVENT #305740, 050812181451 F 1 INFORMATION Received 2005 -08 -12 T 18: 35 from terminate. On. The. Fly. Reprocessing. Episode( PID N/A) on host DB Procedure, message # 1, New Spitzer products are available for req. Keys: 4980736, 7120384 S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (23)

Sample of new ptg xfer Q/A statistics S 13. 0 Test Readiness Review, August Sample of new ptg xfer Q/A statistics S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (24)

Sample of new ptg xfer Q/A statistics – cont. S 13. 0 Test Readiness Sample of new ptg xfer Q/A statistics – cont. S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (25)

Sample of new ptg xfer Q/A statistics – cont. S 13. 0 Test Readiness Sample of new ptg xfer Q/A statistics – cont. S 13. 0 Test Readiness Review, August 29, 2005 Russ Laher (26)