This is a question that is often asked when I demo the GenomeQuest platform to potential customers. I always answer that question in three phases.

The first phase goes like this: “It’s really fast, it’s certainly not any slower than BWA/Bowtie or anything else out there.”. Next question is always: “Well, do you have any benchmarks?”.

Which nicely transitions me into the second phase of the answer. This phase is much more rigorous and usually starts with: “Well, it depends. Let me try to explain.”:

• Any good computer scientist can write a mapping algorithm that is really fast. However, that doesn’t mean anything unless it produces the kind of results that are needed. The NGS application you’re working with matters here. For example, finding genetic variation in human disease requires very accurate mapping with mismatches and indels. In contrast, with digital gene expression in maize you can cut a lot of corners. You just need to have a rough idea of the number of alignments on a transcript.
• Then there is the matter of the data and the technology that produced it. Do you have long reads (more than 120 bp)? Will your mapper handle them? Will it also increase the number of mismatches / indels it can use to align a read? Will it significantly slow down execution, or eat up all your memory when reads get longer? Does this mapper also support local alignments when you need them? Will it align in colorspace? In paired end mode? I could go on.
• Next there is the matter of connecting results to the following step in your pipeline. How long does it take to de-duplicate a  1TB alignment file in SAM/BAM format? Or to find those alignments who’s position overlaps with your exome capture experiment, or all dbSNP entries? At GenomeQuest we have a very efficient way of storing and handling alignments (including sequences/annotation). This saves real time, especially when compared to the alignment step itself (it saves a lot of disk space as well by the way).

By the time we get to the third phase of the answer I’m usually much more confident: “Well, how fast do you need our mapping algorithm to be?”.

• Does it really matter how fast the read mapper is, as long as it’s comparable in performance to other algorithms for most common use cases? Does it matter if you have the alignments in 2.5 hours instead of 3? Maybe if you analyze thousands of samples per week it matters, but then other things like reliability and professional software support should matter as well.
• Do these other algorithms scale with the hardware you throw at them? How easy is it to run a read mapping on 64 compute nodes, with 2 CPUs, 8 cores per CPU per node? What about if you double the amount of hardware? Will you go twice as fast? With the GQ- Engine  you will. Want to run on 1024 nodes? That’s possible.
• Are you asking about speed for a single run, or the throughput for a bunch of runs? Last weekend I ran 2000 NGS read databases though our read mapping workflow (low coverage genome sequencing, about 80M reads per database). I started them on Friday afternoon, went for drinks with my friends that evening, had a nice family dinner on Saturday afternoon, and watched a movie with my kid afterwards. The runs were finished before I woke up on Sunday. No hiccups, no failed runs, no logs to monitor, and – best of all – no “one million-file” directories to organize. There were a lot of other customers on the system that weekend, doing their NGS analysis as well.

If we ever meet for a demo, please ask me this question. I love to talk about it.

Henk Heus, Ph.D.
VP Product Management & Services
GenomeQuest Inc.

At GenomeQuest we use an extended version of the GASSST read mapping algorithm (among others). Read about it here in Bioinformatics here: http://bioinformatics.oxfordjournals.org/content/26/20/2534.abstract

Dave Dooling at PolITiGenomics finds two good reasons for considering cloud options: when organizations have peak demands for compute power and when limitations on space/power/cooling preclude building a system in-house. These are two good reasons, but hardly enough to justify all the cloud computing hype.

So, what’s the argument for cloud computing?

Unlike computing which gets cheaper every year, people cost more every year. So, it makes sense to evaluate the annual software development and maintenance costs, the cost of managing the reference databases; integrating and maintaining new applications, the productivity of the end-users and how to change the ratio of end-user-to-support-programmer from 2-to-1 to 10-to-1 or 20-to-1. Cloud computing defined as “Infrastructure” (computers, networks, and storage) doesn’t alleviate these costs.

Here is a transcript of my notes from the roundtable:

1. Some felt end-users increasingly accept the privacy of their data in the hands of a secure cloud provider. Others remarked it remains uncomfortable for some end users who worry when they “don’t know where their data lives”. The roundtable agreed that more end-user education is needed.
2. Data transfer from the location of data generation to where it is processed remains a bottleneck. However, the problem is more on the upload side since cloud providers tend to have unlimited bandwidth. Corporate and Institution wide networks will have to improve to remedy this bottleneck.
3. Software application providers will have to develop metering metrics for licensing their applications on cloud resources that can be de-commissioned.
4. Cloud resources should allow for moving data between desktop applications and the centralized resources.
5. Commissioning and de-commissioning fixed resources such as databases can be an issue.
6. From a clinical applicability perspective, cloud providers (and those who run applications on cloud resources) will have to consider how to make their solution suitable for regulatory approval and auditing.
7. Finally, if an application provider such as GenomeQuest uses a commercial cloud provider such as Amazon EC2, the participants agreed that the application provider and not Amazon is accountable for the security, privacy, and over all robustness of the IT.

My takeaways? Cloud computing is becoming viable in the minds of the industry. A few solvable roadblocks remain. With infinite computing and infinite data, managing the data and turning it into insight remains the challenge and the opportunity.