Note, familiarity with singleTCW is essential, as MultiTCW projects are created by merging existing sTCW projects.

Contents

• Overview
  • Software Requirements and Installation
• Running the demo - Highly recommended to do this first
• The four steps
  • Top three rows
  • Step 1. sTCWdbs (single TCW databases)
  • Step 2. Compare sequences
    • Add Pairs from Hits
  • Step 3. Cluster Sets
    • Methods
      • BBH
      • Closure
      • BestHit
      • OrthoMCL
      • User-defined clusters
    • Majority hit per cluster
  • Step 4. Statistics
    • Details on running KaKs_calculator
    • Details on MSA score and MstatX
• Additional Details
  • Timings
  • Trouble shooting
  • View/Query with viewMultiTCW
• References

runMultiTCW

1. Input:
   • Two or more sTCW databases. The sequences, annoDB hits, GOs, TPM and DE p-values are imported to the mTCWdb.
   • For NT-sTCW, both the nucleotide sequences and their translated ORFs will be imported into the mTCWdb.
2. The results are the best if:
   • The sTCW databases are annotated the same.
   • The conditions names are exactly the same (when applicable). For example, if two species both have counts for the tissue type 'leaf', the condition name provided in runSingleTCW must be the same for both (e.g. leaf), though the name is case-insensitive.
   • The DE column names are exactly the same (when applicable), as in the previous point.
3. Computation:
   • Compare the AA sequences using diamond or blast, and the NT sequences with blastn.
   • Compute one or more sets of clusters using the following methods: BBH (bi-directional best hit), Closure, Best hit, orthoMCL¹, and/or user-supplied clusters.
   • For a NT-mTCW database created from only NT-sTCW databases, statistics such as Ka/Ks², synonymous, etc are computed.
   • The clusters are scored.

Software Requirements and Installation

Create the three 'ex' sTCWdbs: Using runSingleTCW, create sTCW_exBar as follows:

Select "exBar" from the Project dropdown. Execute the three steps to load data, instantiate and annotated. For the annotate step, only the two invertebrates and fullSubset are selected by default. Use runDE to add the differential expression p-value for each pair of conditions. Create sTCW_exFoo and sTCW_exFly as above.

Using the Add button beside the sTCWdb table, add the exBar, exFoo and exFly databases.   Select Build Database.   Select Run Search.   Select Add Pairs from Hits.   Using the Add button beside the Cluster Sets table, add the "Closure" method using the defaults. See Adding demo methods to add the other methods if desired.   Execute Add New Clusters. Upon completion, the methods in the table will be italicized.   Select Run Stats. When it completes, the label will change to "No action selected".   Exit runMultiTCW in order to run KaKs_calculator. -Change directory to: projcmp/ex/KaKs. -Execute: sh runKaKs -Start up ./runMultiTCW again. The Statistics label should say "Read KaKs", if it does not, select Settings and select it. Then Run Stats.   Add GOs can be run anytime after the Build Database

Adding demo methods:

B12: Set the Method to "BBH". Use the Select sTCWdbs to select the databases bar and fly, set the prefix to "B12", keep. Do the same for B13 and B23.

OM: Set the Method to "User Defined", select the file orthoMCL.OM-40 (provided in the projcmp/ex directory), set the prefix to "OM", keep. See the note below.

To try mixing NT-sTCWdb and AA-sTCWdb as input, create a mTCWdb with demoTra, demoAsm and demoPro. You can also experiment with only AA sTCWdbs by first making protein sTCWdbs from the ORF files of exBar, exFoo and exFly (use the project's orfFiles/bestORFs.pep.fa as input.)

Top three rows

Remove...: Select one or more options. When you select 'Ok', you will be prompt to verify each removal. Clusters from database: Only remove the clusters from the database. The pairs and any associated annotations will remain. Pairs and clusters from database: Removes the pairs and clusters so you can start over without running Build Database again, e.g. if you want to use a different hit file, create all new clusters, etc. Once the pairs are removed, you can change the settings and then re-add the Pairs. mTCW database: Remove the database but leave the project on disk.

• Pairs and method files from disk: If you remove the pairs and clusters from database, it is a good idea to remove all associated files from disk using this option.
• Hit files from disk: If you recreate the database and there have been changes to the sequences, you definitely want to remove the hit files so that it allows you to re-search. Or, if you want to re-run a search program, you need to first remove the hit files using this option.
• All files from disk: If you no longer are using the project, you can delete the database (above) and the all relevant files here.

A multiTCW database can be created with a mix of NT-sTCWdbs and AA-sTCWdbs, as illustrated in the image on the right.   Click the Add button next to sTCWdbs table. This brings up the sTCW selection panel shown on the right. Click Select sTCW Database which produces a panel of existing sTCW databases. Choose the desired sTCWdb from the list. The 'prefix' is only used in the Method files, so it does not matter what it is as long as its unique. The remark can be anything, and can be added/changed after the database is created. Avoid special characters such as quotes. When you select Keep, it will take you back to the main panel and this database will be shown in the sTCWdb table.

The image on the right is the Settings. The Run Search puts the results in the project /hitResults directory in files named hitsAA.tab and hitsNT.tab; these names cannot be changed. On the Settings page, you can change the AA search program from the default Diamond to Blast. The NT search program is Blastn. Once the search is run, you can no longer change the parameters; if you want to change the parameter and re-run the search, remove the hit files using the main panel Remove....

On the main window, loads the pairs from the hits result file. This step also assigns the shared description, which is the same algorithm as used for the Majority Hit for clusters. The only difference is that if there is no shared hit, the hitID will be "*NoShare" whereas clusters always are assigned an annotation if at least one sequence has a hit.

Methods

The BBH finds the bi-directional best hit based on hit e-value. It uses the hits that were loaded into the database with Add Pairs from Hits. The following explains the parameters: Amino acid or nucleotide (for NT-mTCW only). %Similarity - the hit similarity (Identity). %Coverage - Either or both sequences must have the %coverage. See below for details. The Select sTCWdbs will only be present if there are more than two sTCWdbs loaded into the mTCWdb. The rules are as follows: Select two sTCWdbs for the standard BBH of one pair per cluster. Select N (N>2) sTCWdbs, and clusters of exactly size N will be created, where each pair in the cluster is a BBH pair.

%Coverage:

• This is computed by: Cov= (alignment_length/sequence_length_1)*100; this can give a percent coverage>100 if there are many gaps and the alignment is the same length as the sequence.
• You can choose Either, which requires that either Cov1>%Coverage OR Cov2>%Coverage. If you choose Both, then Cov1>%Coverage AND Cov2>%Coverage. For example, for an alignment:

  		Seq1    -------------------------------
  		Seq2     ----------
  	

  This will pass the filter %Coverage>=80 if Either is selected, but not Both.

Closure has the same %Coverage and %Similarity parameters as the BBH algorithm, and also uses the hits from the database. It creates clusters as follows: All sequences in a cluster must have a hit with all other sequences in the cluster. Each sequence must pass the filters with at least one other sequence in the cluster.

All sequences share the same best HitID or best Description. If the best description is used: If it contains such words as "uncharacterized protein", then the hitID is used. Otherwise, the description is used minus the "{ECO:...}" portion of the UniProt descriptions. Each sequence must have >=%Similarity and >=%Coverage to the protein hit. Each sequence must have >=%Similarity and >=%Coverage to the pair hit with all/any other sequences in the cluster. All: If the All Hit filter is Yes, then each sequence in a cluster must pass rule #3 with all other sequences in the cluster. Any: If the All Hit filter is No, each sequence only needs to pass rule #3 with one other sequence in cluster. If the parameters are too lax, this can create some bad clusters. Rule of thumb: Yes - you can reduce %Similarity and %Coverage. No - you should increase %Similarity and %Coverage significantly.

OrthoMCL requires numerous steps to run, and uses a temporary MySQL database; TCW organizes all these details. DBD::mysql must be installed for the Perl-MySQL interface. OrthoMCL uses the hit file hitsAA.tab. It does not guarantee that all sequences in a cluster have a hit with each other. OrthoMCL occasionally fails -- every time this has happened to me, I rerun and it works.

Create a file specifying the groupings, and the interface simply uploads that file. Hit results are not used. The group file has the following format: .. D26: tra|tra_030 tra|tra_184 tra|tra_094 pro|pro_100 D27: tra|tra_045 tra|tra_209 pro|pro_011 ... Each line starts with "DN", where N is the group number, and then has a space-separated list of the sequences in the group, prefixed by the project prefix that you entered when you set up the mTCW. (The is the same file format used by all methods.)

1. Abbreviated annotations: a list of 'Best Anno' is created where each is modified as follows:
   • The words "probable", "punitive", or "predicted" are removed.
   • The first two words are extracted.
   • The annotation is saved in lower case.
2. The number of times an abbreviated annotation is found for a sequence in the cluster is counted (counted only once per sequence no matter how many times it is found in the sequence hit list).
3. The annotation that has the highest count is assigned to the cluster. If there is a tie, then it is further sorted on (1) E-value, (2) whether is it SwissProt and (3) number of GOs.
4. A sequence can have a 'Best Anno' of "uncharacterized" if it has no good annotation hits. This will not be used unless all sequences in the cluster only have "uncharacterized".
5. If none of the sequences have annotation, the hitID will be "*Novel".

The cluster is also assigned a hitID for the selected description using the best E-value. The %Hit is computed, which is the percentage of sequences with the hit.

These rules are also used for Pairs Annotation. The only difference is that the cluster will always be assigned an annotation if there is at least one sequence with the annotation, whereas a pair must have a shared annotation.

The statistics are broken into four sections: Run on the total hit pairs in the database: The PCC (Pearson Correlation Coefficient) is only relevant if there are shared conditions; it is used to determine how similar the TPM values of the conditions are. It is run on all pairs in the database. NOTE: TCW can compute RPKM instead of TPM. The viewSingleTCW overview, top line, states which was computed. Alignment of hit pairs in clusters. This is "only" relevant for a mTCW database created from at least two NT-sTCWdbs: For each alignment, the following is computed: Synonymous codons, nonsynonymous codons, %match, #gaps, GC content, etc. The summary statistics shown on the Overview for Pairs. Outputs the Ka/Ks files for input into KaKs_calculator2. Ka/Ks: After the Ka/Ks files are written, from the terminal window, change directories to projcmp/<project>/KaKs and execute "sh runKaKs". See further details below. Execute Run Stats again with Read selected. Multiple alignment of clusters. Align all clusters using MAFFT3. Compute consensus length, standard deviation of length, sum-of-pairs score, and Wentropy score; see MSA Score for information about using MstatX4.

• Exit runMultiTCW or use a different terminal window.
• Change directories to projcmp/<project-name>/KaKs. There will be multiple files with the name oTCWn.awt where n starts at '1'. They each have pairs of aligned sequences minus the gaps. There is also a file called runKaKs which has the commands to run the KaKs_calculator on each file, e.g.

  /Users/cari/Workspace/TCW_4/Ext/mac/KaKs_Calculator -i oTCW1.awt -o iTCW1.tsv -m YN &
  /Users/cari/Workspace/TCW_4/Ext/mac/KaKs_Calculator -i oTCW2.awt -o iTCW2.tsv -m YN &
  

  The only reason the method "YN" is used here is because it is fastest; this should probably be changed. The default method is "MA", which will be used if you remove the "-m YN" on all lines. Alternatively, you can edit the file to replace the "YN" with a different method. To view the different methods, run (replace 'mac' with 'linux' if you are on a linux machine):

   ./Ext/mac/KaKs_Calculator -h
  

• From the command line, type: sh runKaKs.
• Read the KaKs files into the mTCWdb:
  • If you have exited runMultiTCW and start it back up, the label beside the pair statistics Setting will say "Read KaKs". Just select Run Stats and the files will be read.
  • If you had not exited runMultiTCW but ran runKaKs from another terminal window, the label will still say "No action to be performed". Select Settings, select Read KaKs files and Keep. Now Run Stats.

Score2 is "Wentropy", which is copied directly from the MstatX⁴ with the exception that the scores are (1-score) so that 1 is the best score.

The MstatX executable is in the /Ext directory and can be used for computing the scores. From the appropriate /Ext sub-directory, run ./mstatx -h to view the scoring methods available. The method used by runMultiTCW can be changed from the command line, as follows:

	./runMultiTCW -M1 <method> -M2 <method>

	./runMultiTCW -M1 trident

Update scores: The scores can be changed after ALL the MSAs are computed, as follows: Check Compute MSA and scores and execute Run Stats. The MSAs from the database will be used for computing the scores, and the database will be updated with the new scores.

For developers: you can add your own method to the MstatX program and use it in runMultiTCW.

NOTE: The Wentropy or one of the other mStatX statistics should be valid to use in a publication. Do not use the Sum-of-Pairs score in publication unless cleared with a statistician. The Sum-of-pairs score is useful on the MSA display in viewMultiTCW when viewing column scores.

Timings

The Add GOs is time-consuming; this task can be done at anytime, so it is recommended to wait until everything else is finalized before adding the GOs.

The search programs (e.g. blast) are run on multiple CPUs (#CPU), but all mTCW tasks only use one CPU.

A file called mTCW.error.log is created if there is an error. If its not clear how to fix the problem, send the file to tcw@agcol.arizona.edu.

The clusters can be viewed by either: → Click the Launch viewMultiTCW button in the runMultiTCW interface. → Execute ./viewMultiTCW and a window of existing mTCW databases will be displayed, where databases can be selected for display. → Execute ./viewMultiTCW <database name>.

1. Li, L., Stoeckert, C.J., Jr. and Roos, D.S. (2003) OrthoMCL: identification of ortholog groups for eukaryotic genomes. Genome Res, 13, 2178-2189.
2. Zhang Z, Li J, Xiao-Qian Z, Wang J, Wong, G, Yu J (2006) KaKs_Calculator: Calculating Ka and Ks through model selection and model averaging. Geno. Prot. Bioinfo. Vol 4 No 4. 259-263.
3. Katoh K, Standley DM (2013) MAFFT Multiple Sequence Alignment Software Version 7: Improvements in Performance and Usability. Molecular Biology and Evolution Vol 30, Issue 4 772:780
4. Guillaume Collet (2012) https://github.com/gcollet/MstatX.
5. Edgar RC (2004) MUSCLE: multiple sequence alignment with high accuracy and high throughput. Nucleic Acids Res 32: 1792-1797.
6. C. Soderlund (2022) Transcriptome computational workbench (TCW): analysis of single and comparative transcriptomes. BioRxiv doi: https://doi.org/10.1101/733311