1. Introduction

Compute Find synteny between two sequenced eukaryotic genomes with optional annotation. Draft sequence ordering by synteny, i.e. align a draft genome to a fully sequence (not draft-to-draft). Self-synteny. Query and view For multiple selected synteny pairs, display using dot plot, circular, and side-by-side. Complete queries on annotation, collinear genes, cross-species gene families, etc.

Publications

        C. Soderlund, M. Bomhoff, and W. Nelson (2011)
        SyMAP: A turnkey synteny system with application to plant genomes.
        Nucleic Acids Research 39(10):e68.

        C. Soderlund,  W. Nelson, A. Shoemaker and A. Paterson (2006)
        SyMAP: A System for Discovering and Viewing Syntenic Regions of FPC maps
        Genome Research 16:1159-1168.

Follow the steps below to get started with SyMAP.

2. Requirements

For viewing alignments with viewSymap, CPU and memory needs are typically negligible, unless you are performing queries on more than 4-5 genomes at once.

The largest component of SyMAP execution time is in running MUMmer^1,2. The time and memory for MUMmer all depends on the size of the genomes. For example, to align rice (12 chromosomes, 370Mb) to maize (10 chromosomes, 2Gb) required 1 hour and 3 minutes using 8 CPUs with 2.3Ghz speed.

The memory usage of MUMmer is typically 5G per CPU, however it can be as high as 10G for very long or repetitive chromosomes. If MUMmer fails, it is often due to insufficient memory, see the MUMmer document, which explains how to determine the problem and ways around it. It explains the CPU and Concat options. And it explains how to run MUMmer on a different machine and port the results to the symap_5/data/seq_results directory.

Java: You must have Java version 17.0.11 or later. The released symap.jar file has been compiled with Java 17.0.11, which is upward compatible. If you need a version compile with Java 1.8, email symap@agcol.arizona.edu.

MySQL: If your machine does not have MySQL or MariaDB, download and install it. For example, MySQL can be downloaded from dev.mysql.com. On a personal MacOS, simply download the '.dmg' file and following the instructions. On a work server, the system administrator may need to install it.

Important Note: The default settings of MySQL are poorly suited for large-scale data storage. You will want to adjust the parameters innodb_buffer_pool_size and innodb_flush_log_at_trx_commit as described in Trouble Shoot MySQL.

Installation

     > tar -xf symap_5.tar.gz

   LICENSE   README   data/   ext/   java/
   scripts/  symap    symap.config   viewSymap  xToSymap

Externals: The ext/ directory contains the external programs MUMmer^1,2 (for sequence alignment) and MUSCLE⁷ (for Queries). The directory contains:

	README		mummer/		mummer4/	muscle/

Example symap.config.

  db_name             = symapDemo
  db_server           = localhost
  db_adminuser        = <adminid>
  db_adminpasswd      = <password>
  db_clientuser       =
  db_clientpasswd     =

  >./symap -c symapTmp.config

4. Demo

1. Change into the symap_5 directory.

2. Edit symap.config and enter database and host information (see MySQL).

3. From the command line, type ./symap.

The first time you run SyMAP, it will create the database with information written to the terminal, e.g.

Creating database 'symapDemo' (jdbc:mysql://localhost/symapDemo?characterEncoding=utf8).

It will also check that the provided external programs (e.g. MUMmer) are executable; if it shows any problems, see Executables. For MacOS, you may also need MacOS externals.

The Project Manager window will show the three demo projects provided with the SyMAP tarball. Check Demo-Seq and Demo-Seq2 and they will be displayed on the right panel.
A link Load All Projects will be displayed at the top of the right panel; select it to load the projects, which will take several minutes. Verify the results by selecting the View link. If loading the Demo-Seq takes more than a few minutes, you may need to adjust the MySQL parameters, see TroubleShoot MySQL. When done, the Manager will look like the image shown on the right. In the Available Syntenies table, the cell for Demo_Seq2 and Demo_Seq will automatically be selected. Click the Selected Pair button to start the Alignment&Synteny. The All Pairs button can also be selected, as it will perform Alignment&Synteny on all pairs in the table.
The Alignment&Synteny takes less than 5 minutes on the MacOS 10.5 but could take up to 30 minutes on a slow machine. When done, the table will have a checkbox, signifying that the synteny is available for viewing.
Click Summary to view the summary shown on the right; there may be slight differences in the #Cluster hits because of different numbers of CPUs, MacOS vs Linux, etc (but the #Blocks come out the same). To view the other interfaces, see Demo Results.   Once the alignments are computed, the Parameters can be experimented without having to redo the alignments. Try using the Algorithm 2 (gene-centric) for Cluster Hits; bring up the Parameters window and select Algorithm 2 with its defaults. This gives a summary similar to this.

Load the Demo-Draft project. Under the Demo-Draft listing, you will see the parameter "Order against: demo_seq2". With this setting, the Demo-Draft contigs will be ordered using synteny to Demo-Seq2; this was set in the project's Parameters window.
Run the Alignment&Synteny, where the alignment should take less than 30 minutes with one CPU. When done, open the Summary for the pair, as shown on the right; as mentioned above, there may be slight difference in the number of anchors. See the first dot plot in Demo-draft. It is recommended that the Cluster Hits algorithm 1 be used for ordering sequence.

The ordering algorithm changes the order of the draft contigs in the database, but does not change the sequence files on disk. However, it writes the following files:

1. File of ordered contigs: It writes the order of the contigs along with whether they should be flipped to a file called /data/seq/demo_draft/ordered.csv.

2. FASTA files of ordered sequences: It creates sequence files from the ordered contigs that are flipped when appropriate, which are put into a new project with the suffix _ordered, as shown in the image below. The chromosome names correspond to the order-against project (e.g. demo_seq2), and the third chromosome is 'chr0' which contains all draft sequences that were not placed.

As shown in the image on the left, a new project has been added. Click Demo_Draft-ordered and load it. Running the Alignment&Synteny between the _ordered project and Demo_seq2 will provide a more coherent display (see the second dot plot for Demo-draft). Using the project's Parameters window, the Demo_Draft-ordered name can be shortened.

Self alignment

To perform self-synteny, select the cell for Demo_seq row and column (it turns green) followed by Selected Pair. This computes very few blocks, as shown in Demo self-synteny.

Creating a new project

The following gives an outline of the steps and important details of the directory structure, linking to the appropriate sections in Interface and parameters document, which details all functions and parameters.

Project-name: Each project has a project-name. The project-name should be a short and unique name containing only letters, numbers, and underscores. In the project's Parameters, you will be able to add a more descriptive Display name. The use of the project-name is shown below.

Project directory structure:

Each project has a directory as follows:

  /data/seq/<project-name>

The default location for sequence and annotation files is:

  /data/seq/<project-name>/sequence
  /data/seq/<project-name>/annotation

1. Create these sub-directories under /data/seq and put your files there,
   e.g using project-name=foobar:

     cd data/seq
     mkdir foobar
     cd foobar
     mkdir sequence
     mkdir annotation
   

   Move your FASTA file(s) to data/seq/<project-name>/sequence (e.g. data/seq/foobar/sequence) and your optional GFF files(s) to data/seq/<project-name>/annotation (e.g. data/seq/foobar/annotation)

2. Create these sub-directories under /data/seq and use soft links to point to the file locations,
   e.g using project-name=foobar:

     cd data/seq
     mkdir foobar
     cd foobar
     ln -s <location of directory of sequence files> sequence
     ln -s <location of directory of annotation files> annotation
   

3. Use the Add project button on the symap interface (lower-left corner) to add the project name to the data/seq directory. Use the project's parameter window to enter the location of the sequences and optional annotation files into the Sequence files and Anno files parameters.

All sub-directories in data/seq are shown on the left-panel of the symap interface. Each project is shown with its Display name, which is set in the project's parameters. The project-name is shown on the right hand image as the Directory name, as they are the same.

See pair parameters on setting the pair parameters for the this step. Then select All Pairs or Selected Pair to run the Alignment&Synteny.

Alignment&Synteny files: The result files are in the following directories:

   /data/seq_results/<project-name1>-to-<project-name2>/align
   /data/seq_results/<project-name1>-to-<project-name2>/final

The log files are in the /logs directory, see Running MUMmer for more details.

See Using MUMmer with SyMAP for a discussion on how it works in SyMAP, trouble-shooting, and running MUMmer externally (i.e. if your local machine does not have enough memory or CPUs, you may need to run it on a bigger machine).

1. Load both sequences.
2. For the draft, bring up the project's Parameters window. Beside the Order against row is a drop-down of all loaded projects; select your whole genome project.
3. Run the Alignment&Synteny. At the end, you will see a new project with the suffix _ordered in the left panel. It will contain:
   • A sequence directory with a ".fa" FASTA file. Any scaffolds matching the Order against genome will be assigned the same chromosome name. All scaffolds aligning to an Order against chromosome will be appended together in order with 100 N's between each scaffold. Any extra sequences will be put in ">Chr0".
   • An annotation directory with a ".gff" file that specifies where the gaps are.
4. Load the _ordered project. Then run Alignment&Synteny between the whole genome project and the new _ordered project.

If the draft sequence is in too many sequence contigs, (1) it takes a long time for the MUMmer comparisons, (2) the display is very cluttered, and (3) the blocks display does not work right. Limit the number of sequence contigs by setting Minimum length in the project's Parameter window to only load the largest 150 sequences. To determine the minimal length, use the Lengths button in the xToSymap interface, which will print out all the lengths; set the Minimum length to the 150th length. However, even 150 are a lot of blocks to view so you might want to start with the largest 50, merge them, then repeat.

To perform self-synteny, select the cell for the same project (it turns green) followed by Selected Pair. The All Pairs option does not include self-synteny.

The Alignment&Synteny Parameters window has an option to set Self Args, which is only used when comparing the chromosome sequence file to itself. Make sure that the Cluster Hits Algorithm 1 option is selected.

A better demonstration than the demo is to download Arabidopsis thaliana from NCBI, convert it with the NCBI convert script, and run the self synteny. It took 16 minutes with one processor on a Mac Mini (2018) with 64GB main memory. The dot plot is shown on the right (click on the image for a closeup view).

Parameter Self Args: The "--maxmatch" MUMmer parameter was the SyMAP default, but now it is up to the user as to whether they want to add it to Self Args (it can greatly increase the execution time). Reasoning for using "--maxmatch": MUMmer ordinarily seeds its alignments with unique matches, which eliminates the possibility of off-diagonal seeds in the alignment of a chromosome to itself. To overcome this problem, individual chromosome self-alignments can use the MUMmer parameter -maxmatch, which removes the uniqueness requirement at the cost of greatly increased noise. The extra noise is then filtered to a large extent by the default SyMAP filters, but the diagonal squares of the dot plot will still have more noise visible than the off-diagonal.

The Load and A&S methods have a popup progress window, as shown on the right. There is a Cancel button on the bottom that can be clicked to cancel the execution; it will remove the results from the database and exit. Occasionally, the Cancel will cause it to create an error, writing to the error.log or to the terminal. This is not a problem, though you may need to remove the results yourself. If MUMmer is running when you Cancel, make sure there is "Error: Failed command:" line to terminal for each MUMmer alignment that was running; if there is not, use the "top" linux command to view the running processes and stop any MUMmers still running.

6. General

• If the symap.jar is available from the download site and there are only changes to it, download it and replace the one in symap_5/java/jar.

• Put the new symap_5.tar.gz in a permanent location and untar it.
• Replace the /data and symap.config from your previous SyMAP location to this new location.
• This approach is safest as it acquires all changes (e.g. scripts) except for changes to the demo files.

• Put the new symap_5.tar.gz in a temporary location and untar it.
• Move symap_5/java/jar/symap.jar to the java/jar location of your permanent SyMAP.
• Check to see if there are any /scripts or /ext changes that need to also be copied over.

The following table shows what versions require action by the user.

1. Always get the new java/jar/symap.jar.

2. The Alignment&Synteny will use existing MUMmer files if they have not been removed.

1. It has the FPC demo files.
2. It has BLAT³ in the /ext directory.
3. It has the tar file doc.tar.gz of the documentation.
4. The AGCoL documentation applies to this release. See AGCoL System Guide.
5. This is the last release made from AGCoL, so the documentation will stay consistent.
6. It is also available from Github.

² Marcais, G., A.L. Delcher, A.M. Phillippy, R. Coston, S.L. Salzberg, A. Zimin (2018). MUMmer4: A fast and versatile genome alignment system, PLoS computational biology, 14(1): e1005944.

³ Kent, J. (2002) BLAT--the BLAST-like alignment tool, Genome Research 12:656-64.

⁴ Krzywinski, M., J. Schein, I. Birol, J. Connors, R. Gascoyne, D. Horsman, S. Jones, M. Marra (2009). Circos: An information aesthetic for comparative genomics. Genome Research doi:10.1101/gr.092759.109.

⁵ Soderlund, C., Nelson, W., Shoemaker, A., and Paterson, A.(2006). SyMAP: A system for discovering and viewing syntenic regions of FPC maps. Genome Res. 16:1159-1168.

⁶ Soderlund, C., Bomhoff, M., and Nelson, W. (2011). SyMAP: A turnkey synteny system with application to multiple large duplicated plant sequenced genomes. Nucleic Acids Res V39, issue 10, e68.

⁷ Edgar, R (2004). MUSCLE: a multiple sequence alignment method with reduced time and space complexity. BMC Bioinformatics 113.

⁸ Soderlund, C., S. Humphrey, A. Dunhum, and L. French (2000). Contigs built with fingerprints, markers and FPC V4.7. Genome Research 10:1772-1787.

⁹ Engler, F., J. Hatfield, W. Nelson, and C. Soderlund (2003). Locating sequence on FPC maps and selecting a minimal tiling path. Genome Research 13:2152:2163.