This is a question that is often asked when I demo the GenomeQuest platform to potential customers. I always answer that question in three phases.

The first phase goes like this: “It’s really fast, it’s certainly not any slower than BWA/Bowtie or anything else out there.”. Next question is always: “Well, do you have any benchmarks?”.

Which nicely transitions me into the second phase of the answer. This phase is much more rigorous and usually starts with: “Well, it depends. Let me try to explain.”:

• Any good computer scientist can write a mapping algorithm that is really fast. However, that doesn’t mean anything unless it produces the kind of results that are needed. The NGS application you’re working with matters here. For example, finding genetic variation in human disease requires very accurate mapping with mismatches and indels. In contrast, with digital gene expression in maize you can cut a lot of corners. You just need to have a rough idea of the number of alignments on a transcript.
• Then there is the matter of the data and the technology that produced it. Do you have long reads (more than 120 bp)? Will your mapper handle them? Will it also increase the number of mismatches / indels it can use to align a read? Will it significantly slow down execution, or eat up all your memory when reads get longer? Does this mapper also support local alignments when you need them? Will it align in colorspace? In paired end mode? I could go on.
• Next there is the matter of connecting results to the following step in your pipeline. How long does it take to de-duplicate a  1TB alignment file in SAM/BAM format? Or to find those alignments who’s position overlaps with your exome capture experiment, or all dbSNP entries? At GenomeQuest we have a very efficient way of storing and handling alignments (including sequences/annotation). This saves real time, especially when compared to the alignment step itself (it saves a lot of disk space as well by the way).

By the time we get to the third phase of the answer I’m usually much more confident: “Well, how fast do you need our mapping algorithm to be?”.

• Does it really matter how fast the read mapper is, as long as it’s comparable in performance to other algorithms for most common use cases? Does it matter if you have the alignments in 2.5 hours instead of 3? Maybe if you analyze thousands of samples per week it matters, but then other things like reliability and professional software support should matter as well.
• Do these other algorithms scale with the hardware you throw at them? How easy is it to run a read mapping on 64 compute nodes, with 2 CPUs, 8 cores per CPU per node? What about if you double the amount of hardware? Will you go twice as fast? With the GQ- Engine  you will. Want to run on 1024 nodes? That’s possible.
• Are you asking about speed for a single run, or the throughput for a bunch of runs? Last weekend I ran 2000 NGS read databases though our read mapping workflow (low coverage genome sequencing, about 80M reads per database). I started them on Friday afternoon, went for drinks with my friends that evening, had a nice family dinner on Saturday afternoon, and watched a movie with my kid afterwards. The runs were finished before I woke up on Sunday. No hiccups, no failed runs, no logs to monitor, and – best of all – no “one million-file” directories to organize. There were a lot of other customers on the system that weekend, doing their NGS analysis as well.

If we ever meet for a demo, please ask me this question. I love to talk about it.

Henk Heus, Ph.D.
VP Product Management & Services
GenomeQuest Inc.

At GenomeQuest we use an extended version of the GASSST read mapping algorithm (among others). Read about it here in Bioinformatics here: http://bioinformatics.oxfordjournals.org/content/26/20/2534.abstract

Fast local alignments of NGS reads

NGS read mappers typically align reads by trying to fit the entire read into the reference sequence. This is referred to as a global alignment, or best fit, strategy. While this works great for short genomic reads, it is not always the best possible solution for longer reads. When a read gets longer the chance of it matching the reference sequence over its entire length decreases.

The shortcomings of global alignment algorithms become readily apparent in RNA-seq studies where a single read can span multiple exons. These exons can be right next to each other in the mRNA, but separated by megabases of intronic sequence on the genome. The only way to align such reads is to use a local alignment strategy that can map different parts of a read to different positions on the reference sequence.

We have added local alignment capabilities to our GASSST read mapper while keeping the existing speed and scaling. This allows us to analyze NGS-sized data sets regardless of the read length or sample source and gets us ready for PacBio and Ion Torrent. It also supports our new RNA-seq workflow that maps the transcriptome directly to the genomic reference sequence.

Improved support for Paired End (PE) read handling

We have added exciting new possibilities to work with PE reads to the GQ-Engine. By examining all possible alignment combinations for a read pair we can keep the most likely alignment pairs for further analysis. This strategy to find “happy pairs” can be parameterized on the command line and takes into account the expected distance between the reads, the orientation of the alignment (fwd/rev strands), and the number of mismatches and indels that are needed to align the reads at those positions. All of this happens in memory while computing the alignments, and is much more efficient and exhaustive than the post-alignment processing strategies typically implemented by other read mappers.

Because the PE mapping strategy is fully integrated into the GQ-Engine, we have complete flexibility working with the results. We can, for example, decide to also keep the single end reads that are mapped with high confidence (we will). As well, we can dump all non-happy pairs into a separate alignment database to look for interesting things like, copy number variation or structural variations.

For our web interface users, using the PE read mapping strategy will be completely transparent. When you map a PE read database, we will ask confirmation of the expected insert size and read orientation. That’s all.

Interval Indexing and Positional Based Annotation

Interval indexing within the GQ-Engine adds the ability to very quickly find the overlap between different sets of intervals. Examples of use cases are: “find all alignments overlapping with exons of known genes”, or “find all SNPs in my data set that are already known in dbSNP”. This technology will support many use cases in the GenomeQuest platform. To start, it will speed up the existing variant annotation workflow and drive the RNA-seq workflow. More applications will follow soon.

The GenomeQuest 7.1 release is planned for Friday the 8th of July 2011. I hope to see you there.

Henk Heus, Ph.D.
VP Product Management & Services
GenomeQuest Inc

So now researchers can go from their ChIP-Seq NGS runs directly to gene-based annotation of the peaks found by their biology. Select regions of interest, or genes of interest, or peaks of a certain class, and drill down to see the actual evidence that backs up the call.

We’re giving away free ChIP-Seq runs to the first 100 people to sign up.

As always, feel free to leave a comment – we read every one.

Richard J. Resnick
VP Software and Services

Here is a transcript of my notes from the roundtable:

1. Some felt end-users increasingly accept the privacy of their data in the hands of a secure cloud provider. Others remarked it remains uncomfortable for some end users who worry when they “don’t know where their data lives”. The roundtable agreed that more end-user education is needed.
2. Data transfer from the location of data generation to where it is processed remains a bottleneck. However, the problem is more on the upload side since cloud providers tend to have unlimited bandwidth. Corporate and Institution wide networks will have to improve to remedy this bottleneck.
3. Software application providers will have to develop metering metrics for licensing their applications on cloud resources that can be de-commissioned.
4. Cloud resources should allow for moving data between desktop applications and the centralized resources.
5. Commissioning and de-commissioning fixed resources such as databases can be an issue.
6. From a clinical applicability perspective, cloud providers (and those who run applications on cloud resources) will have to consider how to make their solution suitable for regulatory approval and auditing.
7. Finally, if an application provider such as GenomeQuest uses a commercial cloud provider such as Amazon EC2, the participants agreed that the application provider and not Amazon is accountable for the security, privacy, and over all robustness of the IT.

My takeaways? Cloud computing is becoming viable in the minds of the industry. A few solvable roadblocks remain. With infinite computing and infinite data, managing the data and turning it into insight remains the challenge and the opportunity.

Of course we have our variant calling workflow, just as we integrate with other variant calling workflows. All of these produce GenomeQuest-native browsable, mineable, and queryable databases. And because of the GQ Engine, we can easily combine sets of 10s or 100s or even 1,000s of these variant databases into a single queryable entity with “web-speed query performance.”

Nevertheless, while our customers get the benefit of the combined data, they often ask for more. So today I jumped in to the APIs of GenomeQuest and tried to address the simple problem of building a table of SNPs that span a series of experiments. Each SNP should have the specific allele called for each experiment in which it was found. A simple little table designed to be the input into any of a number of linkage disequalibrium mapping packages. I made a GQ Plug-in: 5 lines of code to make it accessible in the user interface, and another 100 lines of code (I’m wordy) on the back-end to build the table and present it. And so, the multi-experiment haplotype table is born. I might even convince the development team to include it in our next live push.

If you want to hear more or check out the code, drop me a line.

Richard J. Resnick
VP Software and Services

Faced with the dual forces of budget pressures and need to invest or die, pharma is giving itself “permission” to consider economic alternatives to in-house management of bioinformatics data and applications. This could signal the time is right for GenomeQuest and its competitors to show pharma that we have the goods to do it “better, cheaper, and faster” and with more innovation over time than they can do internally.

Of some concern is the idea that public-private funding models where pharma collaborates with publicly funded bioinformatics agencies to solve problems common. I hope these monies go into projects that create opportunities for, and do not compete with private funded enterprises. A vibrant eco-system of commercial informatics tools and service providers will over time innovate more and lower-costs of discovery and pharma industry execs should make sure they have surveyed the commercial landscape before committing their budgets to publicly funded agencies.