Hail is an open-source, scalable framework for exploring and analyzing genetic data. Starting from sequencing or microarray data in VCF and other formats, Hail can, for example:
- generate variant annotations like call rate, Hardy-Weinberg equilibrium p-value, and population-specific allele count
- generate sample annotations like mean depth, imputed sex, and TiTv ratio
- load variant and sample annotations from text tables, JSON, VCF, VEP, and locus interval files
- generate new annotations from existing annotations and the genotypes, and use these to filter samples, variants, and genotypes
- find Mendelian violations in trios, analyze genetic similarity between samples via the GRM and IBD matrix, and compute sample scores and variant loadings using PCA
- perform association analyses with phenotypes and covariates using linear and logistic regression
All this functionality is exposed through Python and backed by distributed algorithms built on top of Apache Spark to efficiently analyze gigabyte-scale data on a laptop or terabyte-scale data on an on-prem cluster or in the cloud.
Hail is used in published research and as the core analysis platform of large-scale genomics efforts including ExAC v2 and gnomAD. The project began in Fall 2015 and is under very active development as we work toward a stable release, so we do not guarantee forward compatibility of formats and interfaces. Want to get involved in development? Check out the Github repo and chat with us in the Gitter dev room.
To get started using Hail:
- read the docs (Getting Started, Overview, Tutorial, General Reference, Command Reference, FAQ)
- join the discussion forum
- chat with the Hail team and other users in the Hail Gitter room
We encourage use of the discussion forum for user and dev support, feature requests, and sharing your Hail-powered science. Please report any suspected bugs to github issues.
The Hail team is based in the Neale lab at the Stanley Center for Psychiatric Research of the Broad Institute of MIT and Harvard and the Analytic and Translational Genetics Unit of Massachusetts General Hospital.
Contact the Hail team at
[email protected].
If you use Hail for published work, please cite the software:
and either the forthcoming manuscript describing Hail (if possible):
- Cotton Seed, Alex Bloemendal, Jonathan M Bloom, Jacqueline I Goldstein, Daniel King, Timothy Poterba, Benjamin M. Neale. Hail: An Open-Source Framework for Scalable Genetic Data Analysis. In preparation.
or the following paper which includes a brief introduction to Hail in the online methods:
- Andrea Ganna, Giulio Genovese, et al. Ultra-rare disruptive and damaging mutations influence educational attainment in the general population. Nature Neuroscience
And we'd love to hear about your work in the Science category of the discussion forum!