This is a question that is often asked when I demo the GenomeQuest platform to potential customers. I always answer that question in three phases.

The first phase goes like this: “It’s really fast, it’s certainly not any slower than BWA/Bowtie or anything else out there.”. Next question is always: “Well, do you have any benchmarks?”.

Which nicely transitions me into the second phase of the answer. This phase is much more rigorous and usually starts with: “Well, it depends. Let me try to explain.”:

• Any good computer scientist can write a mapping algorithm that is really fast. However, that doesn’t mean anything unless it produces the kind of results that are needed. The NGS application you’re working with matters here. For example, finding genetic variation in human disease requires very accurate mapping with mismatches and indels. In contrast, with digital gene expression in maize you can cut a lot of corners. You just need to have a rough idea of the number of alignments on a transcript.
• Then there is the matter of the data and the technology that produced it. Do you have long reads (more than 120 bp)? Will your mapper handle them? Will it also increase the number of mismatches / indels it can use to align a read? Will it significantly slow down execution, or eat up all your memory when reads get longer? Does this mapper also support local alignments when you need them? Will it align in colorspace? In paired end mode? I could go on.
• Next there is the matter of connecting results to the following step in your pipeline. How long does it take to de-duplicate a  1TB alignment file in SAM/BAM format? Or to find those alignments who’s position overlaps with your exome capture experiment, or all dbSNP entries? At GenomeQuest we have a very efficient way of storing and handling alignments (including sequences/annotation). This saves real time, especially when compared to the alignment step itself (it saves a lot of disk space as well by the way).

By the time we get to the third phase of the answer I’m usually much more confident: “Well, how fast do you need our mapping algorithm to be?”.

• Does it really matter how fast the read mapper is, as long as it’s comparable in performance to other algorithms for most common use cases? Does it matter if you have the alignments in 2.5 hours instead of 3? Maybe if you analyze thousands of samples per week it matters, but then other things like reliability and professional software support should matter as well.
• Do these other algorithms scale with the hardware you throw at them? How easy is it to run a read mapping on 64 compute nodes, with 2 CPUs, 8 cores per CPU per node? What about if you double the amount of hardware? Will you go twice as fast? With the GQ- Engine  you will. Want to run on 1024 nodes? That’s possible.
• Are you asking about speed for a single run, or the throughput for a bunch of runs? Last weekend I ran 2000 NGS read databases though our read mapping workflow (low coverage genome sequencing, about 80M reads per database). I started them on Friday afternoon, went for drinks with my friends that evening, had a nice family dinner on Saturday afternoon, and watched a movie with my kid afterwards. The runs were finished before I woke up on Sunday. No hiccups, no failed runs, no logs to monitor, and – best of all – no “one million-file” directories to organize. There were a lot of other customers on the system that weekend, doing their NGS analysis as well.

If we ever meet for a demo, please ask me this question. I love to talk about it.

Henk Heus, Ph.D.
VP Product Management & Services
GenomeQuest Inc.

At GenomeQuest we use an extended version of the GASSST read mapping algorithm (among others). Read about it here in Bioinformatics here: http://bioinformatics.oxfordjournals.org/content/26/20/2534.abstract

According to Bob, it was the 2nd largest crowd ever for his seminars and the largest ever for a software topic — an indicator that reseachers are indeed planning for NGS and eager for answers to their “information bottleneck”.

Over 80 principal investigators, Post Docs, and MDs attended from Harvard hospitals including Beth Israel Deaconess, Children’s Hospital, Dana Farber, Brigham and Womens, Mass General, as well as Harvard Medical School.

Based on the questions, most researchers were interested in the RNA-Seq and Variant Detection applications. Richard Resnick stressed GQ’s cloud-based storage/analysis of results, the GQ Browser for whole-genome analysis, as well as cloud-sharing of results.

Harvard bioinformaticians and computational biologists are welcome to a follow-up seminar or training on the GQ API.

Moving data to the cloud remains the biggest obstacle to cloud adoption. The article makes the case for moving the computation to the data instead of vice versa. Presupposing however that people will be willing to move their data at least 1 time. Otherwise, “moving the computation to the data” is an argument for building and maintaining a local compute cluster, nearby the sequencing instrument.

At GQ we realize there is no “one size fits all”. We support a hosted, cloud based solution for customers with limited IT expertise or inclination. Or, for lab operations running multiple sequencing instruments, we can install the cloud locally so you benefit from the bandwidth of the Local Area Network connecting the instrument to computing. Either way, data moves only 1 time and GQ solves for scalability with its algorithms and data model.

Pretty simple, right?

Well, the main challenge is that personalized medicine (PM) is all about comparing you to the history of human results. That’s right, molecular biologists know much about the concepts of DNA/RNA/proteins but they are now in deep learning mode about what causes what at a molecular level. And they are learning mostly by observation — that is, what happens to a person of type X when we do Y.

So the world of health care is collectively building a PM knowledge base and we wish for doctors to act upon it.

And while today’s knowledge base is small, if people are willing to contribute, it could swell to something very meaningful in but a few years. It’s a massively exciting time for medicine (what with the cost of genome sequencing crashing by 5X every year and cloud computing enabling global sharing of all this).

So a major impediment to this nirvana is that folks today are reluctant to share: “but what if something worse is exposed”. With worries that “existing conditions” (EC) will result in a lifelong insurance ban, it’s a reasonable objection.

Well, with a nation-wide heath care plan with EC worries removed, people will be far more open to sharing their data — which will materially accelerate our inexorable move to PM and its considerable social and economic rewards.

I offer this perspective as a (unintended?) consequence of our national health care program and progress that we all can applaud.

A recent commentary in GenomeWeb “Considering a Cloud? Cost isn’t everything…” citing the paper “The Real Cost of a CPU Hour” illustrates the confusion. The paper benchmarks HPC on a dedicated cluster versus bare metal resources on Amazon EC2. The conclusion is foregone since commodity EC2 network can’t keep up with a tuned applications running on a compute cluster with a high-speed network.

A more informative title for the blogger and the article would be: “Considering Infrastructure-As-A-Service? Beware if your application requires Message Passing Interface and therefore High-Speed Network”.

Fortunately for large scale Sequence Data Management the predominant application mode is “embarrassingly parallel” and therefore HPC are not needed, except maybe on the Web Server where response time is critical. The statistics in the article show that  Embarrassingly Parallel applications run acceptably well on Amazon EC2, adding less than 5% overhead to the computations.

So what?

GenomeQuest is now for bioinformatics and computational biologists (we call them developers for short). These are people who prefer to write code in Unix, and prefer awk, perl, and sed to Firefox, Internet Explorer, Safari, or Chrome.

So why is that important?

With no up-front investment, developers can use the GQ API to write large-scale sequence comparison applications for the cloud without regard for the details of the computing and the reference data. And, they can publish their applications to the GenomeQuest Web Application, so research biologist customers can use and reuse them.

Its helpful to think of the cloud as a layered architecture. The VC Bessemer provides a nice definition of this layering.

• Infrastructure-as-a-Service: Web-services applications framework for provisioning computers, networks, and storage resources without regard to the actual hardware topology. Examples of IaaS are Amazon EC2 and SGI

• Platform-as-a-Service:  Web-services application framework for provisioning data, algorithm, and analysis services and developing new applications. Leveraging all the benefits of the IaaS layer, the GenomeQuest Developer API Framework provides services for managing, comparing, and mining sequence databases without regard to the underlying computers and storage.

• Software-as-a-Service: A finished Web-appliction end-to-end solution for a given end-user application. Examples of this in the Life Science space include the Ingenuity IPA product and the GenomeQuest 6.3 SDM platform.

To my knowledge, we have the industry’s only cloud-based “PaaS”. With a few commands from a remote desktop, developers can upload data, apply algorithms on hundreds of cores while accessing terabytes of well managed sequence data. In the future, we intend to leverage the extraordinary economies of scale offered by IaaS providers such as Amazon.

If you’re curious about the API’s, take a look here.

Over at Depth-First there is a blog post about an application in the cheminformatics field: PubCouch: Streams aren’t just for Pipeline Pilot. The author illustrates how a well abstracted Web service avoids the costly database Extract-Transform-Load operations so familiar to most life science development. In the example, the author streams the entire contents of the PubChem FTP server to PubCouch, a web-service based on the NoSQL style document-oriented database CouchDB. CouchDB doesn’t rely on a database, instead it computes the PubChem relationships “on-the-fly” using an approach based on  MapReduce.

So what you say?

The vision is this: Since modern Web-based programming (aka RESTful architecture) hides the details of massive data and computing resources, programmers can focus on “what to do” and not “how to do it” and that increases productivity.

GenomeQuest’s developers have thought deeply about what a scalable computational biology engine should look like in the cloud-based, MapReduce paradigm. If you want to read a primer on the GQ Engine, feel free to check it out.

Soon, we’ll publish the full-blown URL API so that large-scale biological data and computation can be assembled from any Internet connected desktop, using the language of the Web. A command line interface to our Web-services can be found here.

A final remark: Deepak Singh from business|bytes|genes|molecules wonders aloud what is the role of Pipeline Pilot in this new programming paradigm? I’m guessing within a domain, the value proposition might be limited, but across domains these tools will continue to be able to solve even bigger problems by leveraging better designed Web-services.

Dave Dooling at PolITiGenomics finds two good reasons for considering cloud options: when organizations have peak demands for compute power and when limitations on space/power/cooling preclude building a system in-house. These are two good reasons, but hardly enough to justify all the cloud computing hype.

So, what’s the argument for cloud computing?

Unlike computing which gets cheaper every year, people cost more every year. So, it makes sense to evaluate the annual software development and maintenance costs, the cost of managing the reference databases; integrating and maintaining new applications, the productivity of the end-users and how to change the ratio of end-user-to-support-programmer from 2-to-1 to 10-to-1 or 20-to-1. Cloud computing defined as “Infrastructure” (computers, networks, and storage) doesn’t alleviate these costs.

Here is a transcript of my notes from the roundtable:

1. Some felt end-users increasingly accept the privacy of their data in the hands of a secure cloud provider. Others remarked it remains uncomfortable for some end users who worry when they “don’t know where their data lives”. The roundtable agreed that more end-user education is needed.
2. Data transfer from the location of data generation to where it is processed remains a bottleneck. However, the problem is more on the upload side since cloud providers tend to have unlimited bandwidth. Corporate and Institution wide networks will have to improve to remedy this bottleneck.
3. Software application providers will have to develop metering metrics for licensing their applications on cloud resources that can be de-commissioned.
4. Cloud resources should allow for moving data between desktop applications and the centralized resources.
5. Commissioning and de-commissioning fixed resources such as databases can be an issue.
6. From a clinical applicability perspective, cloud providers (and those who run applications on cloud resources) will have to consider how to make their solution suitable for regulatory approval and auditing.
7. Finally, if an application provider such as GenomeQuest uses a commercial cloud provider such as Amazon EC2, the participants agreed that the application provider and not Amazon is accountable for the security, privacy, and over all robustness of the IT.

My takeaways? Cloud computing is becoming viable in the minds of the industry. A few solvable roadblocks remain. With infinite computing and infinite data, managing the data and turning it into insight remains the challenge and the opportunity.