Pavan Balaji of Argonne National Laboratory and Wu Feng of the computer science department at Virginia Tech led an international team of researchers that received the International Supercomputing Conference 2008 Distinguished Paper Award.
The paper, “Distributed I/O with ParaMEDIC: Experiences with a Worldwide Supercomputer,” takes a new and non-traditional approach to what has been considered an elusive holy grail in high performance computing. I/O stands for ”Input/Output” in a computing system.
Despite enhancements by the information technology field in network hardware as well as network and I/O software stacks, achieving scalable high performance remains a challenge. In Balaji and Feng’s paper, a worldwide team took a completely new and non-traditional approach to distributed I/O that Balaji and Feng dubbed “ParaMEDIC” or Parallel Metadata Environment for Distributed I/O and Computing.” ParaMEDIC leverages application-specific transformation of data to create orders-of-magnitude smaller meta-data for significantly more efficient I/O processing.
Specifically, the paper details the team’s experiences in deploying a large-scale system to facilitate the discovery of missing genes in the 567 completed microbial genomes, as of October 2007, from the National Center for Biotechnology Information’s Entrez Repository (http://www.ncbi.nlm.nih.gov/genomes/lproks.cgi). This repository has since grown to 699 microbial genomes. Given that the repository is growing at such a rapid rate, the task of discovering missing genes has become computationally more difficult; solving such a problem oftentimes exceeds the computational and storage resources of any one supercomputing site.
With respect to discovering missing genes, João Setubal, associate professor and deputy director of the Virginia Bioinformatics Institute (VBI) at Virginia Tech, notes that most of the genomes completed to date have had their genes detected by gene-finder programs, which may miss real genes. “One way to discover these missed genes is by similarity computations,” said Setubal. “If enough computer power is available, every possible location along a genome can be checked for the presence of genes. That is exactly what the ParaMEDIC team has done by leveraging the mpiBLAST sequence-search program.”
Balaji and Feng’s project involved nine different computational sites spread across the United States and generated a petabyte of data that was “teleported” to a single large-scale facility in Tokyo for storage. A petabyte is equal to one quadrillion bytes of storage, which is roughly equivalent to the content of fifty Libraries of Congress or a stack of 212,766 DVDs that measures about 840 feet in height.
Major contributions to the paper were also made by Heshan Lin, a Ph.D. student from North Carolina State University who is interning with Feng this summer and Jeremy Archuleta, a Ph.D. student co-advised by Feng and who is an Institute for Critical Technology and Applied Science (ICTAS) Fellow at Virginia Tech.
The ISC event has evolved over the years into the Annual International Supercomputing Conference featuring a prestigious high-performance computing exhibition and conference, drawing expert speakers, exhibitors, researchers, IT managers, and students from all over the world – thus providing a unique platform for connecting the high-performance computing community on a global scale.
This conference also marks the first of two releases of the TOP500 List every year. The TOP500 is a list of the most powerful supercomputers in the world, ranked by their performance on the Linpack benchmark. Virginia Tech’s System X supercomputer, now four years old, still resides on this list.
Pavan Balaji of Argonne National Laboratory and Wu Feng of the computer science department at Virginia Tech led an international team of researchers that received the International Supercomputing Conference 2008 Distinguished Paper Award.
The paper, “Distributed I/O with ParaMEDIC: Experiences with a Worldwide Supercomputer,” takes a new and non-traditional approach to what has been considered an elusive holy grail in high performance computing. I/O stands for ”Input/Output” in a computing system.
Despite enhancements by the information technology field in network hardware as well as network and I/O software stacks, achieving scalable high performance remains a challenge. In Balaji and Feng’s paper, a worldwide team took a completely new and non-traditional approach to distributed I/O that Balaji and Feng dubbed “ParaMEDIC” or Parallel Metadata Environment for Distributed I/O and Computing.” ParaMEDIC leverages application-specific transformation of data to create orders-of-magnitude smaller meta-data for significantly more efficient I/O processing.
Specifically, the paper details the team’s experiences in deploying a large-scale system to facilitate the discovery of missing genes in the 567 completed microbial genomes, as of October 2007, from the National Center for Biotechnology Information’s Entrez Repository (http://www.ncbi.nlm.nih.gov/genomes/lproks.cgi). This repository has since grown to 699 microbial genomes. Given that the repository is growing at such a rapid rate, the task of discovering missing genes has become computationally more difficult; solving such a problem oftentimes exceeds the computational and storage resources of any one supercomputing site.
With respect to discovering missing genes, João Setubal, associate professor and deputy director of the Virginia Bioinformatics Institute (VBI) at Virginia Tech, notes that most of the genomes completed to date have had their genes detected by gene-finder programs, which may miss real genes. “One way to discover these missed genes is by similarity computations,” said Setubal. “If enough computer power is available, every possible location along a genome can be checked for the presence of genes. That is exactly what the ParaMEDIC team has done by leveraging the mpiBLAST sequence-search program.”
Balaji and Feng’s project involved nine different computational sites spread across the United States and generated a petabyte of data that was “teleported” to a single large-scale facility in Tokyo for storage. A petabyte is equal to one quadrillion bytes of storage, which is roughly equivalent to the content of fifty Libraries of Congress or a stack of 212,766 DVDs that measures about 840 feet in height.
Major contributions to the paper were also made by Heshan Lin, a Ph.D. student from North Carolina State University who is interning with Feng this summer and Jeremy Archuleta, a Ph.D. student co-advised by Feng and who is an Institute for Critical Technology and Applied Science (ICTAS) Fellow at Virginia Tech.
The ISC event has evolved over the years into the Annual International Supercomputing Conference featuring a prestigious high-performance computing exhibition and conference, drawing expert speakers, exhibitors, researchers, IT managers, and students from all over the world – thus providing a unique platform for connecting the high-performance computing community on a global scale.
This conference also marks the first of two releases of the TOP500 List every year. The TOP500 is a list of the most powerful supercomputers in the world, ranked by their performance on the Linpack benchmark. Virginia Tech’s System X supercomputer, now four years old, still resides on this list.