Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline - draft1 #1

Open
wants to merge 93 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
588f034
initial commit
Nov 14, 2024
32575a8
Merge pull request #2 from prototaxites/main
prototaxites Nov 14, 2024
6424840
Updates:
prototaxites Nov 14, 2024
5802c55
Major updates:
prototaxites Nov 15, 2024
d22ba41
Bin3C only run when not using conda profile
prototaxites Nov 15, 2024
5a46e82
Fix metator module
prototaxites Nov 21, 2024
36e7c52
Adds bin refinement with DASTool
prototaxites Nov 21, 2024
273fc26
Start adding MAGSCOT
prototaxites Nov 21, 2024
188136a
Adds:
prototaxites Nov 22, 2024
c71d485
Merge pull request #2 from sanger-tol/dev
prototaxites Nov 27, 2024
54788be
Merge branch 'dev' into d1
prototaxites Nov 27, 2024
a5eff97
Ignore hmm assets with precommit
Nov 27, 2024
3da20e6
Fix schema
Nov 27, 2024
c207428
Fix linting?
Nov 27, 2024
83f586a
Simplify HMMER inputs
Nov 28, 2024
edf5c74
Multi-threaded pyrodigal, update bwamem2_mem
Nov 28, 2024
a06c90a
Remove schema_input.json
Nov 28, 2024
5abd8cd
Add BinQC subworkflow
Nov 28, 2024
df6fcd5
Fix linting
Nov 28, 2024
422b125
Emit versions from MAGSCOT
Nov 28, 2024
ef3700c
Update test config
Nov 28, 2024
2683775
Fix missing operator
Nov 28, 2024
0e250d5
Add full test, enable binqc
Nov 28, 2024
e77aaf7
Fix module
Nov 28, 2024
4d04658
Begin working on Taxonomy WF
Nov 29, 2024
7572fe3
Initial pass at taxonomy with CheckM quality filtering
Nov 29, 2024
9a3890f
Rename DASTool bins so to avoid downstream name conflicts
Nov 29, 2024
f13797b
Fix combine bins for postbinning, DASTool pyrodigal input
Nov 29, 2024
f88023a
Fix param typo
Nov 29, 2024
c2af30e
Fix taxonomy WF, add resource requirements for GTDB
Nov 29, 2024
4f9ae2d
GTDB takes filtered bin channel now
Nov 29, 2024
ebec246
Refine bin processin & dastool bin renaming; adds gtdb_to_ncbi script…
Dec 6, 2024
59ede17
Fix linting, fix dastool input bug
Dec 6, 2024
629cdd2
Fix typo
Dec 6, 2024
8cef726
Fix contig2bin processing
Dec 6, 2024
05c8326
Changes:
Dec 11, 2024
b6f807a
Fix linting, update dastool + metabat2 modules, fix renamed modules
Dec 12, 2024
674b4ad
add skeleton summary process
Dec 12, 2024
4cac07b
Adds:
Dec 13, 2024
9950440
Update metamdbg version; use gzipped bin files
Dec 18, 2024
62bf7a4
Add Prokka to identify ncRNAs for QC purposes; add prokka summary to …
Dec 19, 2024
9693dd7
Add enable_prokka to schema and patch changes to Prokka module (will …
Dec 19, 2024
f6db65a
Changes:
Dec 20, 2024
3970d41
Fix Prokka container and update MetaBat2 module
Dec 20, 2024
9d3f933
Add basic description to README, fix remote YAML in test
Dec 20, 2024
c94be17
Add tRNAscan-SE and Infernal to replace Prokka for ncRNA-checking
Jan 6, 2025
cb720e9
Fix bugs
Jan 7, 2025
59edad5
Merge branch 'rna' into d1
Jan 7, 2025
eeba4a2
Update README.md
prototaxites Jan 9, 2025
263769c
Adds:
Jan 10, 2025
bab4d0e
Fix linting
Jan 10, 2025
747326a
Update schema
Jan 10, 2025
4c6af95
Fix schema
Jan 10, 2025
fdcd41f
Merge branch 'dev' into d1
prototaxites Jan 16, 2025
f8bdfdf
Update assembly.nf
prototaxites Jan 16, 2025
ba97b94
Fix pipeline schema
Jan 16, 2025
8a54f79
Fix test data path
Jan 16, 2025
adc83e8
Fix schema
Jan 16, 2025
99915aa
Fix typo in path
Jan 16, 2025
c21a4be
Add documentation and apply suggestions from code review.
Jan 17, 2025
616ec19
Fix linting
Jan 17, 2025
9fe7ef4
Prettier on documentation
Jan 17, 2025
41bc18f
Apply suggestions from code review
Jan 17, 2025
bb77ecd
Apply suggestions from code review
Jan 17, 2025
03ffc4e
Fix publishDir directive
Jan 17, 2025
0a13bea
Update DAS_Tool image to remove defaults dependency
Jan 24, 2025
0786837
Add TreeVal-style CRAM chunking for Hi-C read mapping
Feb 3, 2025
3c6d4d9
Fix bugs
Feb 3, 2025
06a908f
Merge pull request #14 from sanger-tol/chunk_mapping
prototaxites Feb 3, 2025
1e38b8a
Remove deleted process from config
Feb 3, 2025
89ac6a2
Process input YAML using process for output caching
Feb 3, 2025
30501cb
Fix linting
Feb 3, 2025
302678b
Remove deprecated subWF and rename YAML_INPUT to READ_YAML
Feb 4, 2025
fe4c60d
Fix unrenamed channel
Feb 4, 2025
d4fbf31
Template merge 3.2.0
Feb 4, 2025
f248455
Fix channel
Feb 4, 2025
01f15d3
Move local subworkflows to own directories
Feb 4, 2025
c4bfff9
Fix linting errors in local modules by removing switch statements
Feb 4, 2025
b7a5335
Remove for loop from read mapping subWF
Feb 5, 2025
e73ac2e
Fix languageserver syntax flags
Feb 5, 2025
468067d
Remove extraneous if from assembly workflow
Feb 5, 2025
ce670ab
Move DB preparation to pipeline initialisation subWF
Feb 5, 2025
cedb870
Fix GTDB-Tk metadata files to specific versions
Feb 5, 2025
a1b960a
Update schema
Feb 5, 2025
6061fd0
Correct path to .crai when finding indexes
Feb 5, 2025
16577d9
Reduce size of cram chunk in CI test to force chunking
prototaxites Feb 5, 2025
76462a1
Update metamdbg/asm
Feb 6, 2025
ee0592d
Fix CRAM chunking as was only aligning the first chunk; Add minumum b…
Feb 6, 2025
9e03ce3
Add resource requirements for intensive processes
Feb 6, 2025
43fc4a0
Add .vscode/ to gitignore and remove stray directory
Feb 6, 2025
8a6def1
Fix GTDB-Tk database input
Feb 6, 2025
b162a02
Update schema
Feb 6, 2025
66fc249
Add local test YAML input files + use new tolit testdata file paths
Feb 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add full test, enable binqc
  • Loading branch information
Jim Downie committed Nov 28, 2024
commit 0e250d5bb763a26711e1d50250429c606ca30a17
10 changes: 10 additions & 0 deletions assets/test_full_input.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
tolid: xyTesTing1
pacbio:
fasta:
- ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR131/014/SRR13128014/SRR13128014_subreads.fastq.gz
hic:
cram:
- https://github.com/prototaxites/lrm_testdata/raw/refs/heads/main/hic.cram
enzymes:
- DpnII
- HinfI
8 changes: 2 additions & 6 deletions conf/test_full.config
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,6 @@ params {
config_profile_description = 'Full test dataset to check pipeline function'

// Input data for full size test
// TODO nf-core: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
// TODO nf-core: Give any required params for the test so that command line flags are not needed
input = params.pipelines_testdata_base_path + 'viralrecon/samplesheet/samplesheet_full_illumina_amplicon.csv'

// Fasta references
fasta = params.pipelines_testdata_base_path + 'viralrecon/genome/NC_045512.2/GCF_009858895.2_ASM985889v3_genomic.200409.fna.gz'
// Input data
input = "${baseDir}/assets/test_full_input.yaml"
}
6 changes: 6 additions & 0 deletions workflows/longreadmag.nf
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ include { BINNING } from '../subworkflows/local/binning'
include { BIN_REFINEMENT } from '../subworkflows/local/bin_refinement'
include { PREPARE_DATA } from '../subworkflows/local/prepare_data'
include { READ_MAPPING } from '../subworkflows/local/read_mapping'
include { BIN_QC } from '../subworkflows/local/bin_qc.nf'

/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -65,6 +66,11 @@ workflow LONGREADMAG {

ch_bins = BINNING.out.bins
| mix(BIN_REFINEMENT.out.refined_bins)

if(params.enable_binqc) {
BIN_QC(ch_bins)
ch_versions = ch_versions.mix(BIN_QC.out.versions)
}
}
}
//
Expand Down
Loading