The Assemblathon

Slides from an Assemblathon 2 talk

Thu, 25 Apr 2013 14:14:59 -0400

Here are some slides that I made for a talk that I gave to a general audience at UC Davis. These slides were also used by Prof. Ian Korf for his presentation at the Genome 10K workshop (5/25/2013). Notes are included for each slide.

The first part of this talk presents a simple overview of genome assembly and introduces many of the terms that are used. The next part of the talk discusses results from Assemblathon 2.

Assemblathon 2 talk from Keith Bradnam

Assemblathon has moved to a new home

Sun, 03 Mar 2013 01:10:00 -0500

Goodbye Posterous.

Hello tumblr.

Please let me know if you spot any errors or broken links. If you had any bookmarks to pages on the Assemblathon site, these most likely will need to be updated.

Feedback and analysis of the Assemblathon 2 pre-print

Mon, 28 Jan 2013 16:55:00 -0500

There has already been some discussion of the pre-print of the Assemblathon 2 manuscript. Although a pre-print is not the same thing as a peer-reviewed, accepted paper — I don’t want us to get too ahead of ourselves! — I thought it useful to start collecting together some of the online commentaries:

Homologus blog post 1: highlights a few conclusions from the paper
Homologus blog post 2: delves into the results, and attempts to estimate some of the costs of genome assembly. Assemblathon co-author Sébastien Boisvert adds some useful comments.
Haldane’s Sieve post: an invited blog post by lead author Keith Bradnam, that summarizes what the Assemblathons are all about by way of a pizza-themed analogy
Reevaluating Assembly Evaluations with Feature Response Curves: GAGE and Assemblathons: this is not a blog post, but a recently published paper that evaluates some of the Assemblathon 2 data
Thoughts on the Assemblathon 2 paper: by C. Titus Brown (a reviewer of the manuscript)
Homologus blog post 3: reactions to the previous post by C. Titus Brown
Assemblathon 2 review, round 1, parts thereof: a concise version of C. Titus Brown’s formal manuscript review (minus the specific suggestions)
On assembly uncertainty (inspired by the Assemblathon 2 debate): blog post by Lex Nederbragt in response to post by C. Titus Brown

Remember, the Assemblathon twitter account has also tweeted about these pieces, as well as many other articles relating to genome assembly.

The Assemblathon 2 paper has been submitted!

Thu, 24 Jan 2013 12:58:00 -0500

The manuscript has been submitted to the GigaScience journal and a pre-print of the paper is now available on arXiv.org. Additional data files that support the paper are currently available here (these are also part of the manuscript submission).

Please note that the pre-print has not undergone peer review and we do not assume that the manuscript will be automatically accepted by the journal.

If all goes well, the submitted Assemblathon 2 assemblies, along with sets of CEGMA gene predictions for each assembly, will be available from GigaScience’s GigaDB database. These are also currently available from the Korflab website.

Image taken...

Fri, 20 Jan 2012 16:32:00 -0500

Image taken from http://www.flickr.com/photos/incrediblehow/5577200173/

Why we need the Assemblathon

This short essay was originally written in Fall 2011 for a Genome 10K newsletter that never came to pass so I’m posting it here. The views in this essay are my own and should not necessarily be taken to represent official views of the Assemblathon.

Keith Bradnam, Jan 20th 2012

Note: article updated 2/21/12 to include mention of Meraculous among the best performing assemblers in Assemblathon 1

There was a time when scientists were grateful for any sort of genome sequence no matter its level of quality or completeness (let alone its N50 length). In the late 1990s there were still only a handful of completed eukaryotic genome sequences in existence and many of these had been painstakingly completed using the ‘clone-by-clone’ sequencing approach. If you had a detailed genetic map for your species of interest – which is what the clone-by-clone approach ideally required – then you also had a good chance of knowing how complete your genome sequence might be. End users of these genome sequences were mostly grateful for what they had been given. Even a ‘working draft’ version of a genome which might contain lots of errors was better than no sequence at all.

This era of ‘genomic innocence’ did not last long. As sequencing projects moved away from the clone-by-clone approach and instead started to employ the whole genome shotgun (WGS) strategy, it became clear that there are many different ways to put a genome sequence together. Sometimes you could produce a drastically different assembly from the same sequence data just by using a different version of the same assembly program (e.g. the N50 length of scaffolds from the dog genome was seen to double just by using a later version of the ARACHNE assembler).

It also became clear that just because you can assemble a genome in a way that increases its N50 size (or whatever metric du jour you care to use), it may not necessarily be a better genome. For example, the v1.95 assembly of the sea squirt Ciona intestinalis had a N50 length of 234 Kbp. A later v2.0 assembly produced over a tenfold increase in this metric, but in the process the newer assembly removed several highly conserved genes that were present in the earlier assembly. One step forward, two steps back.

These challenges were happening in an era when a genome sequence was something that represented a species rather than an individual (let alone a tissue type); an era when sequence reads were typically longer than your email signature; and when a genome sequencing project was expected to take years not days. In short, these challenges took place before the dark times, before the Empire before next generation sequencing (NGS) technologies took off.

How times change. Whereas the human genome project took about ten years before it was published – longer before all chromosomes were actually finished – we can now generate enough raw sequence data to produce a similar genome sequence in just ten days. And that’s only considering the output from a single NGS machine. Of course that’s assuming that you can put it all together in the correct order.

The question of ‘how do I assemble a genome from NGS data?’ is not always an easy question to answer. It can be made easier if you have some expectation of what the final genome should look like. However, it is one thing to try assembling a genome sequence with NGS data when you already have a completed sequence from a closely-related species to compare it to. It’s another thing altogether when there is no such ‘reference’ genome. Imagine trying to put a jigsaw puzzle together when there is no picture on the box to guide you, no edge pieces, and only a ‘best guess’ as to how many pieces there are. Oh, and there may be up to 100 copies of every piece.

The biggest headache about dealing with NGS data is that things are probably going to get worse before they get better. It is true that read lengths are increasing year-on-year, and newer sequencing chemistries continue to claim ever greater accuracy. These improvements should make genome assembly a much easier problem in the very near future. However, we still have to deal with the explosion of sequence data that is happening now. Everybody has genomes that need to be sequenced, and without genome assembly the raw output from a genome project is pretty much a meaningless set of very large files taking up space on a disk somewhere.

Projects such as Genome 10K are aiming to sequence the genomes from – wait for it – 10,000 vertebrate species (the clue was in the name) and all those genomes will need assembling. It was thanks to Genome 10K that the Assemblathon project was born. The Assemblathon is more than just a made-up word…it is an ongoing project that is attempting to help drive improvements in the field of genome assembly. It will try to do this by bringing together all of the major players in the genome assembly field together for regular ‘contests’. To date there has been one such contest that has been completed which we oh-so-cleverly named ‘Assemblathon 1’. We are currently in the middle of the inventively named ‘Assemblathon 2’. We plan – subject to our Assemblathon grant application being successful – to organize more Assemblathons in future. So how exactly can the Assemblathon help to assemble genomes?

In many ways, a more pertinent question to ask about genome assembly is not ‘how do I assemble a genome?’ but rather ‘how will I know if my assembly is any good?’. This is a very easy question to ask, but not such an easy question to answer. Genome assembly programs – like most bioinformatics software – will always spit out some output…the challenge is to make sense of that output. There are a range of fairly simple statistical metrics that are commonly used to describe genome assemblies. In most cases we like to believe that higher values for those metrics indicates a better genome assembly, but as the earlier example with C. intestinalis showed, this is not always the case.

Hopefully though, there are always some intelligent questions that we can ask of any genome assembly, and many of these will involve leveraging any other sequence information that may exist for that species (finished BACs, optical maps, ESTs etc.). For example, we should expect that a good genome assembly will contain most of the genes that are present in the genome. We often do not know what those genes will look like, but we know that a large handful of important genes remain essentially unchanged across millions of years of evolution and so we should be able to find those at least. This approach has previously been used by our group to assess the ‘gene space’ in a variety of new genome sequences, and we are currently employing this technique as just one of many strategies in the evaluation of Assemblathon 2 entries.

In the world of genome assembly, beauty is very much in the eye of the beholder. Some people consider a genome assembly that is full of genes to be a very beautiful thing. However, a genome assembly can theoretically contain 100% of the genes, but still only represent ~5–10% of the genome. Some researchers want genome assemblies that capture the maximum amount of the underlying genome. Other researchers don’t care for getting all of the genome – or even all of the genes – just as long as what they do get is highly accurate. If there is a lot of heterozygosity in your genome of interest, then you may prefer an assembly that attempts to best resolve that heterozygosity and produce two haplotype sequences. Or maybe your research concerns repetitive elements and you just want to assemble a genome in a way that still captures a lot of the hard-to-assemble repetitive regions.

If you want the best genome assembly possible, you may have to accept some trade-offs. The assemblers that may perform well in one area may not perform as well in other areas. Everybody wants to be told ‘genome assembler X will give you the best assembly’ but at the moment it doesn’t seem fair to make such bold assertions. What we did find out from Assemblathon 1 was that a number of genome assemblers performed admirably across many, but not all, of the different metrics. For example, the assembler that did the best job at increasing coverage (the amount of the known genome present in the assembly), ranked 9th when considering the number of substitution errors present in the assembly. Conversely, the assembler that did the best job at minimizing substitution errors ranked 8th in terms of coverage. You pays your money and you takes your choice. We should mention, however, that these two assemblers (SOAPdenovo and SGA), along with ALLPATHS and Meraculous were consistently ranked highly for many of the metrics that we used and were the four best overall assemblers in Assemblathon 1.

One of the unusual features of Assemblathon 1 was that it involved synthetic genome data. We actually made a small artificial genome, let it evolve for millions of years, and then produced synthetic reads from the final genome. We did this so that we would know what the final answer should look like (a luxury in the world of genome assembly). However, many people who are developing the latest and greatest algorithms for putting genomes together do not necessarily want to devote too much time to such ‘toy’ problems. They would much rather get their hands dirty with real world data, so their efforts can be of actual use to many researchers around the world. Ask, and you shall receive. In Assemblathon 2 we provided real data from three (count ‘em!) vertebrate genomes: a snake, a fish, and a bird. For the latter species (a parrot), we actually provided sequencing data from three different technologies (Illumina, 454, and PacBio) to see if groups would try a ‘mix & match’ strategy.

Assemblathon 2 is now in the ‘evaluation phase’ where the 43 genome assemblies that we received are being prodded, poked, and subjected to expert scrutiny. We hope that we can produce new metrics to complement those that are currently used to assess genome assemblies. We also hope that we can provide accurate assessments of how good these assemblies are with respect to what different users might want from a genome sequence. But most of all, we hope that we can continue to be useful by drawing the genome assembly community together – both physically and virtually – in order to share ideas and and to start occasional arguments promote vigorous discussion. The Assemblathon will be maximally useful if we help improve the state of genome assembly to the point where we no longer need to organize Assemblathons.

Announcing the arrival of a beautiful, baby…mailing...

Tue, 15 Nov 2011 19:12:00 -0500

Announcing the arrival of a beautiful, baby…mailing list

Image copied from http://www.flickr.com/photos/mknott/4652064320/sizes/m/in/photostream/

At the recent CSHL Genome Assembly Workshop, one of the issues that was raised was for the need to develop a new file format that can capture and accurately represent all of the information inherent in a genome assembly.

For example, you might know that within a contig in your genome assembly, there is a particular subsequence that contains a run of 5–10 adenosine nucleotides. If you generate a FASTA file to represent this, you cannot specify such variation and have to choose a fixed number of As in your sequence. More fundamentally, a genome assembly is – or should be – attempting to represent sequences from two different haplotypes. In an ideal world, the file format should be able to capture this variation.

There has already been a lot of discussion over what such a file format might look like, and the Assemblathon would like to help move this process forward. We don’t want to dictate any standards ourselves, but rather just provide a suitable forum for intelligent discussion that might help drive things along. If the genome assembly community can agree on a new file format, then we will of course be happy to recognize that format and support it in future Assemblathons.

If you want to be part of the conversation, head on over to our mailing list page and sign up for our new mailing list: assemblathon-file-format. This will hopefully co-exist peacefully alongside our main mailing list and become a focused forum for this important issue.

Agenda for CSHL breakout meeting on Genome Assembly

Thu, 03 Nov 2011 14:01:00 -0400

Saturday November 5th – 1:00–6:00

Agenda

1:00 - 1:10 General introduction

1:10 - 1:30 Groups and individuals

1:40 - 1:50 Evaluation update

1:50 - 2:50 Soapbox session

2:50 - 3:00 Group formation

3:00 - 4:30 Group discussion breakout

4:30 - 6:00 Group presentations

Details

Groups and individuals: We are not a huge community, and while some of us know each other, it’s not true of everyone. One of the goals of the Assemblathon is to foster a friendly, yet competitive, community that maximally advances the state of the art in the field of sequence assembly. We would like to hear a brief introduction from assemblers, data providers, and other interested parties.

Evaluation update: The evaluation team will describe the Assemblathon 2 analyses that are currently underway and those that will begin in the very near future. Please don’t expect any announcements of winners at this time. It’s too early for that!

Soapbox session: We want to give 10–12 people a short (~5 min) slot to present an assembly issue that is very important to them. This could be assembly file format, choices of genomes, new algorithm ideas, community resources, etc. This session will inform the next one.

Group work: We will break up into groups to discuss various assembly issues. Such discussions may focus on, but are not limited to, the following topics:

A: Assembly file formats

B: Future Assemblathon targets (transcriptomes, metagenomes, diploid assembly etc.)

C: Assemblathon 2 evaluation

D: New evaluation procedures

E: Improving and merging Assemblathon 2 genome assemblies

At the end of the meeting we will have groups present the results of their discussions. We suggest that these presentations focus on the following key points:

1. Concise description of the problem.

2. What resources are required to solve the problem?

3. What are the achievable intermediate goals?

4. What is the timeline for achieving those goals?

Assemblathon 2 basic assembly metrics

Thu, 13 Oct 2011 19:03:00 -0400

I’ve calculated a set of basic assembly metrics for each Assemblathon 2 submission using our assemblathon_stats.pl script (requires FAlite.pm). Results for individual assemblies are available below but you can also download this information as one combined text file or one CSV file.

Most of these basic measures are hopefully self-explanatory, but here are some further notes:

All sequences in submitted assemblies are initially treated as ‘scaffolds’.
Each scaffold was split on runs of 25 or more ‘N’ characters to form ‘contigs’. Any contigs that could be split in this way were regarded as ‘scaffolded contigs’
Scaffolds that didn’t have runs of 25 or more ‘N’ characters were also counted as ‘unscaffolded contigs’.
N50 scaffold/contig length is calculated by summing lengths of scaffolds/contigs from the longest to the shortest and determining at what point you reach 50% of the total assembly size. The length of the scaffold/contig at that point is the N50 length.
The L50 measure is the number of scaffolds/contigs that are greater than, or equal to, the N50 length.
The NG50 and LG50 measures are the same as the N50 and L50 measures except that rather than compare against the total assembly size, we compare against the estimated genome sizes of the three species. These estimates are crude approximations averaged from a combination of data in the published literature and calculations from Assemblathon 2 participants. The NG50/LG50 measures permit fairer comparisons between assemblies of different sizes. Though note that some assemblies were not big enough to be able to calculate these metrics.

In addition to calculating NG50, you can also calculate NG60, NG70, NG80 etc. E.g. ‘NG25’ would refer to the length of the scaffold at which you have seen a total of 25% of the estimated genome size (assuming you sum all scaffold lengths starting from the longest). Plotting all such ‘N(X)’ values from NG1 through to NG100 allows you easily compare all assemblies against each other. This is shown in the graphs below. Note that the Y-axis is on a log scale.

A parrot, a fish, and a snake walk into a bar...

Mon, 03 Oct 2011 14:42:00 -0400

Image from http://www.flickr.com/photos/enchufado/435293692/

At midnight on Sunday we reached the deadline for Assemblathon 2 submissions. As scientists like to laugh in the face of deadlines, there was somewhat of a last minute rush to submit assemblies. The following graph shows a timeline of the entries received since the submission period started. Nearly a third of all submissions arrived in the last 6 hours of Saturday night!

In total we received 43 assemblies which is fantastic. They are very evenly split between the three species:

Although we received 16 fish assemblies, 5 of these are for evaluation only; such entries are not eligible for any prizes, and we may not be able to assess them as comprehensively as the main entries (though we will try). Similarly, 3 of the 15 parrot assemblies are for evaluation, but all of the 12 snake assemblies are for the main competition.

For the parrot assemblies, there were three available datasets (Illumina, 454, and PacBio), which participants were allowed to use in any combination that they desired. A few weeks ago, I asked people which combinations they think they would use. The actual usage in the submitted assemblies is a little bit different from what we were expecting:

It will be interesting to have a discussion (perhaps in the comments below), as to why fewer teams combined sequences from the different datasets than what was expected.

Prior to the competition, I also asked how many people intended to take part and 21 groups said ‘yes’ and that is exactly how many ended up taking part. Countries represented include: Canada, China, France, Italy, Portugal, Russia, the UK, and the USA. The full list of participating teams is shown below, team name(s) are included in parenthenses:

Wellcome Trust Sanger Institute (Phusion + sga)
Human Genome Sequencing Center, Baylor College of Medicine (BCS-HGSC)
Université Laval (Ray)
National Research University of Information Technologies, Mechanics, and Optics (Computer Technologies Department)
European Bioinformatics Institute (Curtain)
I.G.A Applied Genomics Institute (GAM)
UC Berkeley (MacManes)
DoE Joint Genome Institute (meraculous)
454 Life Sciences (Newbler+454)
Universidade de Lisboa (CoBiG² - Computational Biology & Population Genomics Group)
University of Maryland (CBCB)
IRISA -Institut de recherche en informatique et systèmes aléatoires (Symbiose)
The Broad Institute (Allpaths)
Canada’s Michael Smith Genome Sciences Centre (ABySS)
CRACS - Center for Research in Advanced Computing Systems (CRACS)
BGI - SOAPdenovo
Cold Spring Harbor Laboratory (CSHL)
University of Georgia (IOBUGA)
UCSF
Wayne State University (AB2L)

I will soon be making all of the assemblies from these groups available for any participant to inspect (and for the evaluators to…well, evaluate). In the interests of ensuring fairness, the identity of each assembly will be kept secret and will be known only to me until we finish evaluating the assemblies. You will of course, be able to work out which assemblies are yours by looking at the file sizes/md5 values/base count etc.

Thank you to everyone who is participating. I hope the results of our analysis will become a useful resource to anyone interested in producing better genome assemblies (insert your own definition of ‘better’ here).

Hot off the presses: Assemblathon 1 results published! Image...

Mon, 26 Sep 2011 20:11:00 -0400

Hot off the presses: Assemblathon 1 results published!

Image reproduced from http://www.flickr.com/photos/nitsrejk/126982680/sizes/z/in/photostream/

A preliminary version of the Assemblathon 1 paper is now available on Genome Research’s website. Congratulations to all involved!

Assemblathon 2 overview

Tue, 06 Sep 2011 11:14:00 -0400

With one month to go to the submission deadline, I recently asked the genome assembly field as to whether they would be participating in Assemblathon 2. Twenty-one people completed my short survey (you can still complete it too if you haven’t done so).

The first question was simply to ask whether people were going to take part:

The ‘maybe’ category suggests that some groups don’t yet know whether they will have enough time to complete their genome assemblies. By way of comparison, Assemblathon 1 had 17 entrants, so we are in the same ballpark as last time and may still exceed that figure. Question 2 asked which of the three possible species, groups would be submitting assemblies for:

At the moment only four groups are submitting genome assemblies for all three species and eight groups are only submitting assemblies for a single species (though these numbers will probably change a little bit). Remember that each species is effectively a separate Assemblathon contest and groups are not required to submit assemblies for all species.

Parrot is currently proving most popular species, possibly because of the varied sequencing technologies that have been used to generate genome sequence data for this species. This leads into the last question, which tried to assess which combinations of sequencing technologies people would use to complete the parrot genome.

Six of the seven possible combinations will be represented with five groups looking to use a combination of Illumina, 454, and PacBio data. This diversity in how the input data are being used will make it interesting to see how the the final assemblies differ (assuming that they differ).

Update: 9:56 am

Since writing the blog post an hour or so ago, a few more people have completed the feedback form. There are now 18 confirmed participants (and still 5 maybes).

Assemblathon breakout meeting to follow CSHL Genome Informatics...

Mon, 15 Aug 2011 13:06:00 -0400

Assemblathon breakout meeting to follow CSHL Genome Informatics meeting

Image taken from http://www.flickr.com/photos/idiolector/100071458/sizes/m/in/photostream/

The CSHL Genome Informatics meeting ends at lunchtime on Saturday 5th November. We are planning to hold an Assemblathon breakout meeting to take place on this afternoon (after lunch). This meeting will discuss issues relating to Assemblathon 2, and potentially cover other issues such as directions for future Assemblathons. We envisage that the meeting may last until 6:00 pm and this breakout session will be open to participants of the Genome Informatics meeting.

More information will be posted on the Assemblathon timetable page as details emerge. Also, an announcement will hopefully be made a the start of the Genome Informatics meeting.

Parrot genome in Assemblathon 2 will now be represented by a...

Mon, 27 Jun 2011 17:47:37 -0400

Parrot genome in Assemblathon 2 will now be represented by a third sequencing technology

Image reproduced from http://www.flickr.com/photos/kiyoshi_be/2436967848/sizes/m/in/photostream/

Thanks to the generosity of Pacific Biosciences, we will soon be releasing a third set of sequences that assemblers can optionally use when assembling the parrot genome. Reads from a PacBio RS machine will be giving us 10X coverage from two libraries:

7Kb insert library: average read sizes = ~1.2kb.
10Kb insert library: average read sizes = ~3Kb, with 5% of the reads over 6–8Kb

This dataset adds to the existing 219X coverage of the genome by Illumina HiSeq data, and the 16X of coverage by Roche 454 data. Assemblers are free to ‘mix and match’ from any of these three datasets when making their parrot genome assembly.

Full Assemblathon 1 analysis by UCSC team now available

Fri, 03 Jun 2011 19:19:00 -0400

The Assemblathon group from UC Santa Cruz have now released their full analysis of the Assemblathon 1 genome assembly assement. Click on the following link to access documents, data and code:

Assemblathon 1 UCSC Materials

Assemblathon 2 begins today! Today is the start date for...

Thu, 02 Jun 2011 00:08:00 -0400

Assemblathon 2 begins today!

Today is the start date for Assemblathon 2. There are next-generation datasets available for three species: a fish (a cichlid), a bird (budgerigar), and a reptile (snake). The cichlid and snake datasets are all based on Illumina data (mate pair + paired-read) and are available to download now. The parrot data has a mixture of 454 and Illumina data (enough of each to make two separate assemblies if you so desired), and will hopefully be available on our website within a day or two.

We are expecting up to 25 participants, and although people are free to assemble any or all of these datasets, we hope that participants will attempt all three if possible. To encourage this, we will hopefully have a prize for the team that produces the highest quality assemblies for all three species.

Initially, it might take some time (possibly up to a week) to download these datasets. We hope to be able to announce some mirror download sites in the coming days which might help things. The rules are simple. At the start of September we will require that each participating team would have uploaded one assembly for each dataset that they are assembling. I.e. no more than three assemblies per team.

We are extremely grateful to Illumina for generously providing the snake data, for the Broad Institute for providing the cichlid data, and for Erich Jarvis at Duke University & the BGI for providing the parrot data.

Good luck and happy assembling!

Assemblathon 1 results

Wed, 01 Jun 2011 14:28:00 -0400

The majority of results were initially made available during the Genome Assembly Workshop, though some were not fully available until after this meeting. The final publication describing the official set of Assemblathon 1 results was published in Genome Research, in late 2011.

Collectively, the Assemblathon evaluation groups at UC Santa Cruz and UC Davis generated over 100 different genome assembly metrics, many of which make use of the fact that we had a definitive picture of what the genome of species A actually looks like.

This page links to various results, submitted assemblies, and Assemblathon-related talks. We advise against focusing on a single metric in order to claim that any one assembly is ‘better’ than another. As we will hopefully make clear, there are many complex ways of describing assemblies and sometimes assemblies perform very well when using one metric, but very badly when using a different – and equally valid – metric.

Note that most of the files described below are contained in one directory on the Korf Lab website. You might find it easier accessing them from this directory.

Assemblathon Talks

March 2011 - Genome Assembly Workshop: UC Davis Assemblathon talk - 1.8 MB PDF with notes
March 2011 - Genome Assembly Workshop: UC Santa Cruz Assemblathon talk - 33 MB PDF
May 2011 - CSHL Biology of Genomes: Assemblathon talk - 2.6 MB PDF with notes

The Assemblies

Each submitted assembly is available here as a gzipped file of scaffolds:

A1
B1 B2
C1 C2
D1 D2 D3 D4 D5
E1 E2 E3
F1 F2 F3 F4 F5
G1
H1 H2 H3 H4 H5
I1 I2
J1
K1 K2 K3
L1
M1 M2 M3 M4 M5
N1 N2 N3
O1
P1
Q1

Results and reports

Basic Assemblathon metrics

The Perl script that generated these results is available here (requires FAlite.pm). Most of these basic measures are hopefully self-explanatory, but here are some further notes:

All sequences in submitted assemblies are initially treated as ‘scaffolds’.
Each scaffold was split on runs of 25 or more ‘N’ characters to form ‘contigs’. Any contigs that could be split in this way were regarded as ‘scaffolded contigs’
Scaffolds that didn’t have runs of 25 or more ‘N’ characters were also counted as ‘unscaffolded contigs’.
N50 scaffold/contig length is calculated by summing lengths of scaffolds/contigs from the longest to the shortest and determining at what point you reach 50% of the total assembly size. The length of the scaffold/contig at that point is the N50 length.
The L50 measure is the number of scaffolds/contigs that are greater than, or equal to, the N50 length.
The NG50 and LG50 measures are the same as the N50 and L50 measures except that rather than compare against the total assembly size, we compare against the known genome size of species A (using the average size of haplotypes 1 and 2). These measures permit fairer comparisons between assemblies of different sizes.

Preliminary UC Davis report - version 0.6 (4/28/11)

This PDF contains provisional findings from some of the analysis performed by the Assemblathon group at the UC Davis Genome Center. It will be updated in subsequent days and some of this information will end up in the main Assemblathon paper that is being written.

Mauve analysis

This PDF report was prepared by Aaron Darling and uses the Mauve multiple genome alignment software to map genome assemblies to the known genome of species A. As with the previous report, some of these results will be included in the main Assemblathon paper.

UC Santa Cruz Assemblathon analysis - A summary of all of their results, 16 MB PDF. N.B. still being updated!
UC Santa Cruz Assemblathon 1 materials - a comprehensive set of documents, data, and code

The published Assemblathon 1 paper

Assemblathon 1 rules

Wed, 01 Jun 2011 14:04:00 -0400

Assemblathon ‘1’ – 2010

A total of three two genome sequences will be made available to participants.
One genome sequence will be a real set of Illumina reads from an unspecified organism. Update: real Illumina data will now be available as part of Assemblathon 2
The other two genomes will be derived from a pair of related ‘virtual’ species whose genomes have been artificially evolved using the EVOLVER program (Edgar, R.C., Asimenos, G., Batzoglou, S, and Sidow, A.). The estimated divergence time between the two virtual species is 100 million years.
One synthetic genome will already be assembled and participants will be able to (optionally) use information from this ‘sister’ species’ genome to guide the assembly of the other unassembled genome (which will exist as a set of synthetic Illumina reads).
Reads from the virtual assembly will be made to simulate the dynamics of ‘real’ Illumina reads as much as possible and will be derived from a mixture of Paired-reads and Mate Pairs from a mixture of insert sizes.
Genome data will be available to download from December 1st, 2010.
Participants have until February 6th 2011 to submit their assemblies for the synthetic genome.
Assemblies will be assessed using a variety of metrics: one of the goals of the Assemblathon is to devise new ways of quantifying and qualifying genome assemblies.
Results from the Assemblathon will be discussed by invited participants at the Genome Assembly Workshop, in March 2011

Assemblathon 1 participants

Wed, 01 Jun 2011 13:58:00 -0400

Here are the details of the groups participating in the Assemblathon. Please let us know if any details are incorrect, need amending, or whether you would like links directed to specific web pages of labs/groups. Also make a note of your ‘team ID’ as we may use these in subsequent discussions about the Assemblathon.

Team ID	Team name/details	Affiliation(s)	Principle contact	Number of entries	Software tools used for assembly
A	GIS_CMB1	Agency for Science, Technology and Research, Singapore	Pramila Ariyaratne	1	PE-Assembler
B	Phusion	Wellcome Trust Sanger Institute, UK	Zemin Ning	2	Phusion2, phrap
C	Ensembl Genomes’ Curtain	European Bioinformatics Institute, UK	Matthias Haimel	2	SGA, BWA, Curtain, Velvet
D	sanger-sga	Wellcome Trust Sanger Insitute, UK	Jared Simpson	4	SGA
E	Borgs	CRACS (Center for Research in Advanced Computing Systems), Portugal	Nuno Fonseca	3	ABySS
F	ABySS	BC Cancer Genome Sciences Centre, Canada	Shaun Jackman	5	ABySS, Anchor
G	Plant Genome Assembly Group	DOE Joint Genome Insititute, USA	Jarrod Chapman	1	Meraculous
H	Team Symbiose	L’IRISA (Institut de recherche en informatique et systèmes aléatoires), France	Rayan Chikhi	5	Monument
I	Terrapins	CSHL (Cold Spring Harbor Laboratory), USA	Michael Schatz	2	Quake, Celera, Bambus2
J		Department of Computer Science, Iowa State University	Xiaoqiu Huang	1	PCAP
K	Super Dawgs	Computational Systems Biology Laboratory, University of Georgia, USA	Wen-Chi Chou	3	Seqclean, SOAPdenovo
L	PRICE @ deRisi Lab	UC San Francicso, USA	Graham Ruby	1	PRICE
M	Softberry	Royal Holloway, University of London, UK	Victor Solovyev	5	OligoZip
N	TGAC/TSL/Oxford	The Genome Analysis Centre, Sainsbury Laboratory, and Wellcome Trust Centre for Human Genetics, UK	Mario Caccamo	3	Cortex_con_rp
O	KAS	Department of Computer Science, University of Chicago, USA	Fangfang Xia	1	Kiki
P	BGI-Shenzhen	BGI, China	Zhenyu Li	1	SOAPdenovo
Q	ALLPATHS Assembly Team	Broad Institute	David Jaffe	1	ALLPATHS-LG

Assemblathon 1 data

Wed, 01 Jun 2011 13:53:00 -0400

Currently there are 13 files available to download. The first 8 of these files contain synthetic Illumina reads for species ‘A’. The next 3 files contain: the complete (synthetic) genome of the related species ‘B’, a reference version of that genome, and annotations for the reference. Finally, a README file provides more detailed information about the contents of these files and a checksums file contains the MD5 checksum value for each file. See the Rules page for more information about the Assemblathon and please contact us if you have any problems in downloading these data.

Please visit the download website or use the direct links below for HTTP access to individual files.

README.txt - HTTP
speciesA_200i_40x.1.fastq.bz2 - 1.8 GB - HTTP
speciesA_200i_40x.2.fastq.bz2 - 1.9 GB - HTTP
speciesA_300i_40x.1.fastq.bz2 - 1.8 GB - HTTP
speciesA_300i_40x.2.fastq.bz2 - 1.9 GB - HTTP
speciesA_3000i_20x_r3.1.fastq.bz2 - 931 MB - HTTP
speciesA_3000i_20x_r3.2.fastq.bz2 - 946 MB - HTTP
speciesA_10000i_20x_r3.1.fastq.bz2 - 930 MB - HTTP
speciesA_10000i_20x_r3.2.fastq.bz2 - 945 MB - HTTP
speciesA.diploid.fa.bz2 - 62 MB - HTTP
speciesB.diploid.fa.bz2 - 62 MB - HTTP
speciesB.reference.fa.bz2 - 31 MB - HTTP
speciesB.annotations.gff.bz2 - 11MB - HTTP
checksums.md5 - HTTP

Important update - January 5th 2011

The original set of mate-pair files contained errors and have been replaced with new files. Please use the mate-pair files listed above as thesecorrect some problems in the orientation of the mate-pair reads (see below for more details). The paired-end libraries are unaffected by this issue. We apologize for any inconvenience this may cause you.

Explanation of problem

In the previous dataset both reads were placed on the same strand by mistake. A quick-fix would be to simply reverse complement all of the first reads and reverse the quality strings which would lead to the reads being in the proper orientation. However, fixing the reads in this way would result in the underlying error applied to the reads being done so in reverse. For example an observed error rate typical of the beginning of an Illumina read would show up at the end of the first reads. We have modified the sequencing simulator code so that it produces mate-pair reads in the proper orientation, and then used this modified code to generate two new replacement mate-pair libraries.

After we fixed this initial problem (and made available release ‘r2’ of the data), we discovered a subsequent problem. The non-biotin pull-down reads within the mate-pair library should have been orientated towards each other but were mistakenly orientated in the same way as the rest of the mate-pair library. This was subsequent fixed in a third release of the data (‘r3’) available above. The files containing errors are listed below but we would strongly urge people to re-download the r3 files from above.

Older files containing errors

speciesA_3000i_20x.1.fastq.bz2 - 931 MB - HTTP - DO NOT USE! (see above)
speciesA_3000i_20x.2.fastq.bz2 - 946 MB - HTTP
speciesA_10000i_20x.1.fastq.bz2 - 930 MB - HTTP
speciesA_10000i_20x.2.fastq.bz2 - 945 MB - HTTP
speciesA_3000i_20x_r2.1.fastq.bz2 - 931 MB - HTTP
speciesA_3000i_20x_r2.2.fastq.bz2 - 946 MB - HTTP
speciesA_10000i_20x_r2.1.fastq.bz2 - 930 MB - HTTP
speciesA_10000i_20x_r2.2.fastq.bz2 - 945 MB - HTTP

Setting out the stall for Assemblathon 2 Image taken...

Wed, 30 Mar 2011 14:37:00 -0400

Setting out the stall for Assemblathon 2

Image taken from http://www.flickr.com/photos/lwr/101661613/

As we finish up the analysis of Assemblathon 1, our thoughts turn to Assemblathon 2 and we’d like to ask for some guidance from the genome assembly community. Specifically, we are seeking feedback as to what combination of data sets we should use for the next Assemblathon event (due to start in May). Currently we are planning to use parrot sequence data from the BGI and cichlid data from the Broad Institute. While there is Illumina data for both species the 454 data only exists for parrot at the moment, but this could change.

Therefore we have the option of using only Illumina data or a mixture of Illumina and 454 data (either as a combined genome assembly challenge or as separate challenges for each sequencing technology). Of course, increasing the number of challenges also greatly increases the workload for the you (the participants) and us (the evaluators).

If you entered Assemblathon 1 and/or if you plan to enter Assemblathon 2 please help us by completing the following form:

Assemblathon 2 datasets