Project Summary

Federated Cloud Model Goals

• Implement a scalable and sustainable multi-institutional cyberinfrastructure cloud federation model that provides data analysis building blocks in support of multiple research disciplines requiring flexible workflows and analysis tools for large-scale data sets. Federation sites are Cornell University, University at Buffalo, and University of California, Santa Barbara.
• Support seven strategic science use cases from intentionally diverse disciplines (earth and atmospheric science, finance, chemistry, astronomy, civil engineering, microbiome, and agriculture) to demonstrate the potential of a federated cloud as a campus bridging paradigm. Explore data analysis techniques and their applicability to different disciplines. Document tools, workflows, challenges, and best practices for each use case.
• Encourage and reward data analysis resource sharing with a new allocations and accounting model that provides a fair exchange mechanism for resource access between and across multiple institutions.
• Develop and build a new tool for cloud metrics into Open XDMoD.
• Develop DrAFTS (Durability Agreement From Time Series) statistics to make online forecasts of cost and performance for Amazon Web Services and Aristotle Clouds.
• Investigate three container technologies (Docker, Singularity, X-Containers) and identify pain points experienced by users when selecting and implementing these technologies and Kubernetes orchestration for scientific software. Share lessons learned and best practices in a technical report and present the findings at a tutorial that works through the issues of building and running non-trivial containers.
• Containerize application kernels and test their performance on Google Cloud versus Aristotle Clouds, Comet, Bridges, Stampede2, Bridges-2, and Expanse. Publish the performance comparisons for the cyberinfrastructure and research community.