A guide to using exonerate in client:server mode

`a generic tool for sequence alignment`

A guide to using exonerate in Client-Server mode

This page provides examples of using exonerate in Client-Server mode exonerate.
It is assumed that you've already had a look at the examples in the beginner's and advanced guides.
Further information can be found on the exonerate-server man page.
If you have questions, or more examples to add to this tutorial, please email me: guy@ebi.ac.uk

Overview

Normally when exonerate is run, the query and target sequences are used as fasta files. The run time of any search at least be proportianal to the size of these input files. However, the exonerate-server allows a set of sequences and an index of words which they contain to be held in memory. The server may be used in place of the target database on the exonerate command line, which allows very fast searches to be conducted (with run time independent of the size of the database). To run the exonerate with exonerate-server, you need to build a database file and an index file, and start the server using these files. These steps are explained below:

Building the database (.esd) file
Building the index (.esi) file
Starting the server
Running the client
Running under LSF

Building the database (.esd) file

Firstly, a .esd (exonerate sequence database) file must be generated from a fasta format input file. This is done using the fasta2esd utility provided with exonerate. eg:
```
fasta2esd genome.fasta genome.esd
```
Alternatively, many fasta files may be used to generate a single .esd file as shown here:
```
fasta2esd --fasta chr1.fa chr2.fa chr3.fa --output genome.esd
```
By default, it is assumed that the input is softmasked. You may turn off this feature (as shown below), so that masked parts of the sequence are included in the resulting indices, although this will slow searches as the indices will be much larger:
```
fasta2esd --softmask no genome.fasta genome.esd
```
The .esd file only links to the fasta files, so will typically be very small. The source fasta files must be left in place for the indexing stage and running the server.
Typical run time for fasta2esd on the human genome: 90 seconds

Building the index (.esi) file

Once the .esd file is ready, the .esi (exonerate sequence index) file can be generated using the esd2esi utility. This contains the index of word positions from the input sequences:
```
esd2esi genome.esd genome.esi
```
If the type of exonerate search to be used is translated (ie. where DNA input generates protein alignments, eg. with the protein2genome model), a translated index must be generated (an index of the words in the translated frames of the input):
```
esd2esi genome.esd genome.trans.esi --translate yes
```
By default, the esd2esi step will use up to 1GB (1024MB) of memory for the indexing step. The more memory is available, the fewer passes of the input data are required, and the faster the indexing is completed: This default may be increased or decreased by the parameter --memorylimit:
```
esd2esi genome.esd genome.esi --memorylimit 2048
```
It is also possible to set other parameters which affect the seed words and word neighbourhood. Parameters which may be set in esd2esi will be ignored when exonerate is run using exonerate-server. For a full list of parameters which may be set by esd2esi may be seen by using esd2esi -h eg:
```
esd2esi genome.esd genome.esi --dnawordlen 11 --wordjump 3 --saturatethreshold 100
```
Typical run time for esd2esi on the human genome: 30 minutes (for a translated index: 85 minutes)

Starting the server

Once an .esi file has been prepared, the server may be started:
```
exonerate-server genome.esi
```
By default, the server will run on port 12886, but an alternative port may be specified on the command line:
```
exonerate-server genome.esi --port 12345
```
The server is multithreaded - once a connection has been accepted, it will be handled in a separate thread. By default, upto four connections are permitted at once, but this may be changed on the command line:
```
exonerate-server genome.esi --maxconnections 10
```
Typical startup time for exonerate-server on the human genome: less than one minute
The memory requirements of the server are minimised as both the sequences and indices are held in memory in a compressed form. Typically, for a softmasked human genome, about 3GB of memory is required, or 4.5GB when using a translated index)

Running the client

Finally, once the server is running, it may be used for fast queries using exonerate. The server is specified on the command-line in place of the target sequences in the form hostname:port :
```
exonerate query.fa localhost:12886
```
The rest of the command line used is the same as if exonerate was running without the server. In example below, the est2genome model is used and the geneseed parameter is set to 250. Parameters which can be used by the server, such as --geneseed in this example, are passed to the server to accelerate the query. For a typical cDNA of 1.5kb, this query is answered in less than a second.
```
exonerate est.fa localhost:12886 --model e2g --geneseed 250
```
Similarly, a protein query may search a server using a translated index. For a typical 400 residue protein, this query takes less than a second
```
exonerate protein.fa localhost:12886 --model p2g --geneseed 250
```
The exonerate-server may also be used for analyses using a large number of queries, such as the example below searching a genome using short (36bp) reads. Typically, about 60 reads can be queried against the genome per second, ie. < 30 minutes for 100000 example queries below (on a single CPU).
```
exonerate solexa.100000.fa localhost:12886 --model a:l --seedrepeat 4 --dnahspthreshold 90 --score 120
```
Memory usage of the client is normally very low, as most of the data is held server-side

Running exonerate-server under LSF

You need to tell both LSF and exonerate-server how many CPUs you wish the server use. eg. to run on 2 CPUs:
```
bsub -n2 -R'span[hosts=1]' exonerate-server --maxconnections 2 genome.esi
```
If you are running exonerate-server under LSF, it is easy for the memory limits to be exceeded, which will result in your job being killed. To avoid this, the server should be started specifying the maximum ammount of memory which will be used. eg. allowing up to 4GB of memory for the server:
```
bsub -M4000000 -R'select[mem>4000] rusage[mem=4000]' exonerate-server genome.esi
```
Once the server is running, the client jobs may be directed at the host running the server job, and submitted.
As soon as the client jobs are completed, the server job may be terminated.
NB. prior to version 2.1, exonerate-server did not work well with LSF.

Please email me if you have any questions

Guy St.C. Slater.