%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Analysis of the impact of genetic polymorphism (inter-individual
%% variations) on transcription factor binding sites.


\chapter{Regulatory variations (rSNPs and insertion/deletions)}


\authors{J\'er\'emy Delerce and Jacques van Helden}

This chapter presents a series of tools to analyze the impact of
genetic polymorphism (inter-individual variations) on the binding of
transcription factors.

It combines a series of tools to 
\begin{enumerate}
\item download genomes, features, and variations from the \ensembl
  database and install them on the local \RSAT server;
\item obtain sequences of variants, exended on both sides;
\item scan these variant sequences with position-specific scoring
  matrices (PSSM) in order to detect variations that may affect
  transcription factor binding.
\end{enumerate}

\section{Requirements}

The functionalities described below require a connection to the
\ensembl database, via their Perl API\footnote{API: application
  programmatic interface}. 

In principle, the Ensembl Perl modules should have been installed
together with \RSAT (see \RSAT installation guide for details).

% \section{Installing genomes and variations from \ensembl}

% \subsection{\program{install-ensembl-genome}}

% The program \program{install-ensembl-genome} downloads the complete
% genomic sequence of a given organism from the \ensembl Web site, and
% installs it on the local \RSAT site. It also installs the descriptions
% of genomic features (genes, CDS, mRNAs, ...), and the variations.

% As usually, the complete help message can be obtained with the option
% \option{-help}.

% \begin{lstlisting}
% ## Get the description of the program + all options
% install-ensembl-genome -help
% \end{lstlisting}

% Before installing a genome, it is generally a good idea to know which
% genomes are available. For this, use the option
% \option{-available\_species}.

% \begin{lstlisting}
% ## Retrieve the list of supported species on EnsEMBL
% install-ensembl-genome -v 1  -available_species
% \end{lstlisting}

% \emph{Note:} inter-individual variations are available for a subset
% only of the genomes available in \ensembl. The option
% \option{-available\_species} indicates, for each species, the
% availability (genome, features, variations). Obviously, the analysis
% of regulatory variations only makes sense for the genomes documented
% with variations.

% We can now download the complete genomic sequence for the species of
% our choice. For the sake of space and time economy, we will use a
% small genome for this manual: the budding yeast \org{Saccharomyces
%   cerevisiae}. 

% \emph{Beware}: some installation steps take a lot of time, in
% particular the installation of genomic features, because the \ensembl
% interface requires to send individual queries for each gene
% separately. The full installation can thus take several hours. This is
% not a big issue, since installing a genome is not a daily task, but it
% is worth knowing that the whole process requires a continuous
% connection during several hours.

% \begin{lstlisting}
% ## Install the genome sequences for a selected organism
% install-ensembl-genome -v 2 -species Saccharomyces_cerevisiae
% \end{lstlisting}

% The download time depends on genome size, on the speed of your
% internet connection, and on the number of genes.

% \subsection{Installing genomes from Ensembl genomes}

% The historical \ensembl project \urlref{http://www.ensembl.org/}
% was focused on vertebrate genomes + a few model organisms
% (\org{Saccharomyces cerevisiae}, \org{Drosophila melanogaster},
% \ldots).

% A more recent project called \ensemblgenomes
% \urlref{http://ensemblgenomes.org/} extends the \ensembl project to a
% wider taxonomic space.

% The program \program{install-ensembl-genome} supports the installation
% of genomes from \ensembl as well as \ensemblgenomes. By default, it
% opens a connection to the historical \ensembl database, but the option
% \option{-ensembl\_genomes} enables to install genomes from the new
% project \ensemblgenomes.

% \begin{lstlisting}
% ## Get the list of available species from the extended project
% ## EnsemblGenomes
% install-ensembl-genome -v 2 -available_species -db ensemblgenomes \
%    -o available_species_at_EnsemblGenome.txt
% \end{lstlisting}

% You can then locate your genome of interest in the file
% \file{available\_species\_at\_EnsemblGenome.txt}, and start the
% installation (don't forget the option \option{-ensembl\_genomes}.

% \begin{lstlisting}
% ## Install Escherichia coli (strain K12 MG1665) from EnsemblGenomes
% install-ensembl-genome -v 2 -db ensemblgenomes \
%    -species Escherichia_coli_str_k_12_substr_mg1655
% \end{lstlisting}


% \subsection{Downloading variations}

% The program \program{download-ensembl-variations} downloads variations
% from the \ensembl Web site, and installs it on the local \RSAT
% site. 

% This program relies on \program{wget}, which must be installed
% beforehand on your computer.

% \begin{lstlisting}
% ## Retrieve the list of supported species in the EnsEMBL variation database
% download-ensembl-variations -v 1  -available_species
% \end{lstlisting}

% We can now download all the variations available for the yeast.

% \begin{lstlisting}
% ## Retrieve the list of supported species in the EnsEMBL variation database
% download-ensembl-variations -v 1 -species Saccharomyces_cerevisiae
% \end{lstlisting}


\section{Getting variation sequences}

Before detecting regulatory variations, the first step is to obtain
the genomic sequences of these variations.  For this, variations must
have been installed on your server by the \RSAT administrator (the
installation protocol is explained in
Section~\ref{sect:download_ensembl_variations}).

To check if variations have been installed for your genome of
interest, you can proceed in the following way. 


\subsection{Checking the installation of variations on your \RSAT server}

Since the databases about polymorphic variations are growing
exponentially, the default genome installation does not include
variations. The program \program{install-ensembl-genome} includes the
option \option{-task variations}, which downloads all variations from
the Ensembl database, and exports them in one file per chromosome or
contig.

\begin{enumerate}
\item Check which versions and assemblies of your organism of interest
  have been installed on \RSAT.
\item Check if variations have been installed for these
  versions/assemblies.
\item Count the number of variations per chromosome.
\end{enumerate}

Let us assume that we are interested by the Human genome. We want to
see if our \RSAT admin installed it, and, if so, to check which
versions or assemblies are available locally.

\begin{lstlisting}
supported-organisms -taxon Homo
\end{lstlisting}

In our case, we get the following answer:

{\color{OliveGreen} \begin{footnotesize} 
\begin{verbatim}
Homo_sapiens_GRCh37
Homo_sapiens_GRCh38
\end{verbatim} \end{footnotesize}
}

Variations are installed in the organism-specific folder of the genome
directory.

\begin{lstlisting}
ls ${RSAT}/public_html/data/genomes/Homo_sapiens_GRCh38/variations
\end{lstlisting}

Variations are stored in tab-delimited text files, with one separate file
per chromosome.  The demo makefile includes a target to check the size
of variation files (count the number of lines per file).

\begin{lstlisting}
## Check the content of the variation directory for the yeast
make -f ${RSAT}/makefiles/variation-scan\_demo.mk \
   SPECIES=Homo_sapiens ASSEMBLY=GRCh38 \
   variation_stats
\end{lstlisting}

This command will indicate the location of the variation directory on
your \RSAT server, and count the number of lines for each variation
file (there is one separate file per chromosome or contig).


{\color{OliveGreen} \begin{footnotesize} 
\begin{verbatim}
Statistics about variations
        SPECIES         Homo_sapiens
        ASSEMBLY        GRCh38
        ORG             Homo_sapiens_GRCh38
        VARIATION_DIR   /data/rsat/public_html/data/genomes/Homo_sapiens_GRCh38/variations
Number of lines per variation file
... (we skip some lines)
   4027724 4.tab
   4134079 3.tab
   4666045 1.tab
   4984715 2.tab
  60448850 total
\end{verbatim} \end{footnotesize}
}

\section{File formats for variations}

Several formats have been defined to store inforamtion about genetic
variations. 


To ensure compatibility between the different sources of information
and \RSAT requirements, the program \program{convert-variation}
ensures inter-conversions betwneen different formats. The program
includes an on-line help which provides more information about the
different formats.

\begin{lstlisting}
convert-variations -h
\end{lstlisting}


We will treat an example of conversion from the GVF to varbed
format. We prepared some demo cases in the file
\file{makefiles/variation-scan\_demo.mk}.

\begin{lstlisting}
make -f makefiles/variation-scan_demo.mk
\end{lstlisting}
\tbw

\section{Detecting regulatory variations}

\subsection{Scanning a selected variation with selected matrices}

\tbw

\subsection{Obtaining a list of disease-associated variation IDs}

In order to test the retrieval of multiple variations, we will first
select a realistic case, by getting a list of Human variations
associated to some disease (for instance diabetes).

Open a connection to the dbSNP database
\urlref{http://www.ncbi.nlm.nih.gov/SNP/}, and paste the following
test in the query box:

\begin{lstlisting}
("Homo sapiens"[ORG]) AND (diabetes[Text Word]) AND ("false"[Not Reference Assembly])
\end{lstlisting}

\tbw


\subsection{Scanning a list of selected variations with a list of matrices}

\tbw

\subsection{Scanning all variations with a selected matrix}

\tbw



