Introduction APS Engineering Support Division X_Ray

Introduction • APS Engineering Support Division • X_Ray Sciences Division –Beamline Controls and Data Acquisition • Pete Jemian (Group Leader) • Kenneth Evans, Jr. (Scientific Software Section Leader) • Brian Tieman (Software Engineer) –Information Technology • Ken Sidorowicz (Group Leader) • Roger Sersted (Engineer) • Gabrielle Long (Division Director) –Chemisty, Environmental and Polymer Science • Peter Chupas (Beamline Scientist) –Materials Characterization • Ulrich Lienert (Beamline Scientist) • Jon Tischler (ORNL Resident Scientist) –Time Resolved Research • Michael Sprung (Beamline Scientist) • Alec Sandy (Beamline Scientist) –X-Ray Microcopy and Research • Francesco De. Carlo (Beamline Scientist) • Wah Keat Lee (Beamline Scientist) A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Current Operational Workflow A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Preferred Operational Workflow A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Local HPC Resources--What • Tomo – 32 processor cluster – 12 TB disk – Sector 2 dedicated to tomography • Blacklab – 16 processor cluster – 2 TB disk – Development • Orthros – 58 processor cluster – 30 TB disk – On demand data reduction A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Local HPC Resources--Why • On Demand Processing – Data reduction immediate part of workflow – May waste CPU cycles-bad for larger clusters • Alignment (10 s of minutes) • Between Samples (<5 minutes) • Between Shifts (? ? ) • High Throughput – Dozens of samples per day • Automated sample changers • Sometimes Unattended – Multiple beamlines • Semi-Long Term Storage – 3 to 6 months • Low Processor Demand per Application • Low Latency—True Real-Time Processing A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Remote HPC Resources--Why • Modeling applications are generally more demanding – Trickier algorithms—more advanced math – Scale to many more processors – Need to be run many times • More need for remote collaboration – Interpret results – Compare with theory/other results • Beyond APS Resources – It’s all about efficient use of money • • Manpower Hardware Space Etc… A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

A Survey of Use Cases • Tomography –Tomo. MPI –Paraview/MCS solution –DEJ_Texture. Analysis –Local Tomography –Laminography • 3 D X-Ray Diffraction Microscopy –xdmmpi –Near Field Peak Finder –Image. D 11/Fable –Grainspotter/Fable –Box Scans • X-Ray Photon Correlation Spectroscopy –xpcsmpi • X-Ray Micro-Diffraction –Reconstruct –Euler –Rindex A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Common Features of Our Use Cases • Need for HPC resources – On demand clusters – Large scale clusters • Most have need for large data volumes – Archival – Transport • Most still need algorithm development – Parallelism – Optimization – Robustness/Portability • Many used by many unrelated scientific disciplines – Open access – Intuitive interfaces – Tailored interfaces A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Where MCS Can Help… • HPC resources – Help target appropriate systems • How to find them • How to develop for them • How to generate proposals for them – Help understand management of HPC resources • How to use HPC for On Demand computation • How to tuning for performance • What to buy A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Where MCS Can Help… • Dealing with the Data – Archival solutions • Central repository • Nearline/Offline storage – Fast/Reliable data transfer • HPC resources • End users – Ethernet – Sneakernet A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Where MCS Can Help… • Algorithm Development – Help enable scientists to parallelize code • More training sessions • Assistance with initial parallelization – Help with code optimization • Maybe codes exist • Maybe new routines need development – Robustness/Portability • Libraries we should be using • Languages we should be using • Operating Systems we should target A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Ways MCS Can Help… • User access – Service Oriented Architecture • • Hide complexity Intuitive interfaces Remote access Collaboratory Experience – Help users set up HPC software on their systems A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.

Ways APS Can Help… • Look for Joint MCS/APS LDRDs • Explore possibility of APS providing operational funding for MCS – New Hardware – R&D Effort • Station APS FTE in MCS – Information exchange – Provide MCS effort on projects of direct benefit to APS • Conduit to end users – Collect new use cases – Explore potential new funding opportunities – Scheduling • Meetings • Training A U. S. Department of Energy laboratory managed by UChicago Argonne, LLC.