Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.6 RC2 #135

Merged
merged 100 commits into from
Jun 30, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
dd70db6
Sample subclassing messaging and more information control in interact…
vreuter Jun 14, 2017
29dc0e0
amid pull-apart of pipelines Sample subclass search
vreuter Jun 14, 2017
b9923a9
remove old examples
vreuter Jun 14, 2017
49f1e5f
better failure for nonexistent pipelines location; pull apart search …
vreuter Jun 14, 2017
500b8a9
better message
vreuter Jun 14, 2017
0f77aee
simplify edge case handling in looper run; pause Sample subclass tests
vreuter Jun 14, 2017
322fe8d
reflect separated project init in test fixture; update requiremnets; …
vreuter Jun 15, 2017
cb3a838
type cast for config value for inequality comparison
vreuter Jun 15, 2017
a195362
reflect new requirements in changelog
vreuter Jun 15, 2017
5a7eaca
version bump to rc2
nsheff Jun 15, 2017
1f26297
Remove scripts directory
nsheff Jun 15, 2017
f58fc74
Fix version requirements
nsheff Jun 15, 2017
13d553b
enable pipeline_interfaces attribute alongside pipelines_dir
nsheff Jun 15, 2017
5410cd0
move extra attribute explanation to exception message; downgrade mess…
vreuter Jun 15, 2017
80611e6
Fix import of scripts dir
nsheff Jun 16, 2017
ea3f005
Move add_sample_sheet from looper back into models
nsheff Jun 16, 2017
1ea7642
inform about subtypes found
vreuter Jun 16, 2017
9926228
reflect addition of sample sheet in project ctor
vreuter Jun 16, 2017
91da40f
trigger build
vreuter Jun 16, 2017
1af7860
amid sample subclass overhaul
vreuter Jun 16, 2017
e358383
more work on Sample subclass support
vreuter Jun 17, 2017
cddc405
clarification of some of what's going on in the pipeline construction…
vreuter Jun 17, 2017
16fe836
more explanation
vreuter Jun 17, 2017
b60cc83
first pass at finished build_pipelines implementation
vreuter Jun 18, 2017
3eb2739
creating subtype for each pipeline submission
vreuter Jun 18, 2017
79c6e39
TODO items
vreuter Jun 18, 2017
258ee99
better control of when to build pipelines; iteration over protocols; …
vreuter Jun 18, 2017
136faec
slight cleanup and comments
vreuter Jun 18, 2017
eb3d39b
first pass at creating the Sample subtype in looper; getting YAML fil…
vreuter Jun 18, 2017
f1e947f
need to finalize pipelines directory before processing locations; bet…
vreuter Jun 18, 2017
eaccdff
Project representation restrictions; num_samples not working
vreuter Jun 19, 2017
f96a7e2
num_samples working; it's samples itself that's problematic, via df attr
vreuter Jun 19, 2017
5c382e2
bump up binding of Project to Sample
vreuter Jun 19, 2017
fa7989e
cutting the Sample-to-Project cord; removing functions that have been…
vreuter Jun 19, 2017
bc3edfd
fix establishment of default compute and environment settings
vreuter Jun 19, 2017
59121db
dot to underscore
nsheff Jun 19, 2017
d1c18c4
message format
vreuter Jun 19, 2017
4db285b
Merge branch '0.6-rc2' of github.com:epigen/looper into 0.6-rc2
vreuter Jun 19, 2017
ce4e9bf
remove punctuation and match case for protocol search; ask Sample abo…
vreuter Jun 20, 2017
bd83e57
check for read_type after pipeline attributes have been set; write Sa…
vreuter Jun 20, 2017
9a8323b
base Sample construction with Project by default
vreuter Jun 20, 2017
4d725d4
first pass at new subtype import logic
vreuter Jun 20, 2017
590751b
getitem for PipelineInterface; use submission bundle creation functio…
vreuter Jun 20, 2017
8bb9fd9
fix up the matching of protocol names/keys in mappings; provide funct…
vreuter Jun 20, 2017
5a1c447
earlier pipeline locations finalization to ensure a collection; fix t…
vreuter Jun 20, 2017
be6094b
sheet function and better names/messages
vreuter Jun 20, 2017
7b6614a
docstrings and cleanup
vreuter Jun 20, 2017
b5d7fab
pin down what needs to change in the Sample independent tests around …
vreuter Jun 20, 2017
4483f4e
fix the Sample tests based on column inference method signature change
vreuter Jun 20, 2017
a171117
update pipeline locations key in tests Project config data; update th…
vreuter Jun 20, 2017
4d0f331
fix handling of initial Sample creation for Project; update Project t…
vreuter Jun 21, 2017
341230a
account for new attribute fetch call and remove outdated test
vreuter Jun 21, 2017
a35868e
ensure Sample has merged flag; remove old tests and fix others
vreuter Jun 21, 2017
1e235f0
Merge branch 'master' of github.com:epigen/looper into 0.6-rc2
vreuter Jun 21, 2017
d76c70b
use Series as raw Sample data in case subtypes are restrictive; grab …
vreuter Jun 21, 2017
9f97b2a
improve the submission counting
vreuter Jun 21, 2017
d2bd807
accumulate missing attributes and files in the required inputs confir…
vreuter Jun 21, 2017
c7b2d11
update tests to reflect new function name
vreuter Jun 21, 2017
aab1f54
check for null merge table
vreuter Jun 21, 2017
ca118fa
prevent nulls in sample YAML; generate Sample filename from subtype
vreuter Jun 21, 2017
dbfa23d
fix pipeline interface tests
vreuter Jun 22, 2017
1bdb471
trace messages
vreuter Jun 22, 2017
54c76c4
Update cluster docs
nsheff Jun 22, 2017
500e4a2
Merge branch '0.6-rc2' of github.com:epigen/looper into 0.6-rc2
nsheff Jun 22, 2017
be09c63
put merge and column derivation in proper order
vreuter Jun 22, 2017
6bb271e
Merge branch '0.6-rc2' of github.com:epigen/looper into 0.6-rc2
vreuter Jun 22, 2017
075ec4b
cleaner handling of merge call from Sample builder; return the column…
vreuter Jun 22, 2017
04036e3
MERGE HAPPENS BEFORE COLUMN DERIVATION (via set_file_paths); account …
vreuter Jun 22, 2017
73038e9
handle Sample subtype resolution and pipeline module imports; first-p…
vreuter Jun 23, 2017
6ac4f17
pull the docs changes
vreuter Jun 23, 2017
bae47df
fix import handling and improve exception catching; microtest working…
vreuter Jun 23, 2017
5588b0e
Fix doc bugs
nsheff Jun 23, 2017
b190e02
Add examples folder back in
nsheff Jun 23, 2017
53f2868
update examples to implied cols
nsheff Jun 23, 2017
780063c
remove docs from example config and simplify
nsheff Jun 23, 2017
ce9ade1
enforce __repr__ implementation and validate call safety
vreuter Jun 23, 2017
c1d4dc8
Merge branch '0.6-rc2' of github.com:epigen/looper into 0.6-rc2
vreuter Jun 23, 2017
6fbb7f5
better handling of path expansion in PipelineInterface; tests for con…
vreuter Jun 24, 2017
c1ceca9
reorganize to facilitate fixture reuse
vreuter Jun 24, 2017
86deccd
first ProtocolInterface test implemented
vreuter Jun 24, 2017
eb7ac19
better function name; more tests and expose lack of leading-dot relat…
vreuter Jun 24, 2017
89c5895
little more info about the path expansion should it ever fail; fix up…
vreuter Jun 25, 2017
db3cf18
finish protocol interface pipeline path tests, passing
vreuter Jun 25, 2017
fae6656
add path expansion tests for PipelineInterface, passing
vreuter Jun 25, 2017
8d4d5e6
add tests to check for warning message about nonexistent pipeline scr…
vreuter Jun 25, 2017
a95591d
better separation of concerns in handling of creation of submission b…
vreuter Jun 25, 2017
923a4ab
catch more exceptions and add explanation; add tests and stub others,…
vreuter Jun 26, 2017
6b9dcfb
account for possibility of null return value being introduced into th…
vreuter Jun 26, 2017
4706892
ensure that sys.modules is cleaned up; make sure that we account for …
vreuter Jun 26, 2017
be7c8a1
python 3, less verbosity, better messaging, fix format errors, random…
vreuter Jun 26, 2017
9977f14
descriptions, cleanup, better organization
vreuter Jun 26, 2017
09c6b63
add the sample name collision tests
vreuter Jun 26, 2017
36b46b8
separate concerns; add unimplemented subtype class tests
vreuter Jun 26, 2017
838669b
add the proper fixture hook for parameterization
vreuter Jun 26, 2017
e65254d
stricter subtype enforcement, cleanup, add tests
vreuter Jun 26, 2017
e310833
first pass at finishing off tests
vreuter Jun 27, 2017
33ea9c1
fix the few test mistakes
vreuter Jun 27, 2017
65b2efc
broader fixture sharing
vreuter Jun 27, 2017
f7cc27e
fixing the smokes and adding sample sheet tests
vreuter Jun 27, 2017
966286b
cleaner protocol-based selection in sheet construction; better test o…
vreuter Jun 27, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix the few test mistakes
  • Loading branch information
vreuter committed Jun 27, 2017
commit 33ea9c1d7e75c108b1cc367267e32d222f9f47a8
2 changes: 1 addition & 1 deletion looper/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -2825,7 +2825,7 @@ def class_names(cs):

# Base Sample could be imported; we want the true subtypes.
proper_subtypes = _proper_subtypes(classes, base_type)
_LOGGER.debug("%d %s subtype(s): %s", len(proper_subtypes),
_LOGGER.debug("%d proper %s subtype(s): %s", len(proper_subtypes),
base_type.__name__, class_names(proper_subtypes))

# Determine course of action based on subtype request and number found.
Expand Down
36 changes: 24 additions & 12 deletions tests/models/independent/test_ProtocolInterface.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

SUBTYPES_KEY = ProtocolInterface.SUBTYPE_MAPPING_SECTION
ATAC_PROTOCOL_NAME = "ATAC"
SAMPLE_IMPORT = "from looper.models import Sample"


class CustomExceptionA(Exception):
Expand Down Expand Up @@ -453,6 +454,7 @@ def test_subtype_is_not_Sample(
# Write out pipeline module file with non-Sample class definition.
lines = _class_definition_lines(subtype_name, name_super_type="object")
with open(pipe_path, 'w') as pipe_module_file:
pipe_module_file.write("{}\n\n".format(SAMPLE_IMPORT))
for l in lines:
pipe_module_file.write(l)

Expand Down Expand Up @@ -480,6 +482,7 @@ def test_subtype_not_implemented(
lines = _class_definition_lines("Decoy", "object") \
if decoy_class else []
with open(pipe_path, 'w') as modfile:
modfile.write("{}\n\n".format(SAMPLE_IMPORT))
for l in lines:
modfile.write(l)
conf_path = _write_config_data(
Expand All @@ -505,7 +508,7 @@ def test_matches_sample_subtype(
decoy_proto = "DECOY"

# Update the ProtocolInterface data and write it out.
atacseq_piface_data[SUBTYPES_KEY] = {
atacseq_piface_data[atac_pipe_name][SUBTYPES_KEY] = {
ATAC_PROTOCOL_NAME: subtype_name, decoy_proto: decoy_class}
conf_path = _write_config_data(
protomap={ATAC_PROTOCOL_NAME: atac_pipe_name,
Expand All @@ -518,6 +521,7 @@ def test_matches_sample_subtype(

for lines_order in itertools.permutations([legit_lines, decoy_lines]):
with open(pipe_path, 'w') as pipe_mod_file:
pipe_mod_file.write("{}\n\n".format(SAMPLE_IMPORT))
for class_lines in lines_order:
for line in class_lines:
pipe_mod_file.write(line)
Expand All @@ -543,16 +547,18 @@ def test_subtypes_list(
# Define the classes, writing them in the pipeline module file.
subtype_names = ["ArbitraryA", "PlaceholderB"]
with open(pipe_path, 'w') as pipe_module_file:
pipe_module_file.write("{}\n\n".format(SAMPLE_IMPORT))
for subtype_name in subtype_names:
# Have the classes be Sample subtypes.
for line in _class_definition_lines(
subtype_name, name_super_type=Sample.__name__):
pipe_module_file.write(line)
pipe_module_file.write("\n\n")

# Update the ProtocolInterface data.
subtype_section = subtype_names if spec_type == "single" \
else {ATAC_PROTOCOL_NAME: subtype_names}
atacseq_piface_data[SUBTYPES_KEY] = subtype_section
atacseq_piface_data[atac_pipe_name][SUBTYPES_KEY] = subtype_section

# Create the ProtocolInterface.
conf_path = _write_config_data(
Expand All @@ -568,35 +574,42 @@ def test_subtypes_list(


@pytest.mark.parametrize(
argnames="target", argvalues=["middle", "bottom"])
argnames="target", argvalues=["Leaf", "Middle"])
@pytest.mark.parametrize(
argnames="spec_type", argvalues=["single", "mapping"])
def test_sample_grandchild(
self, tmpdir, spec_type, target,
atacseq_piface_data, name_atac_pipe):
atacseq_piface_data, atac_pipe_name):
""" The subtype to be used can be a grandchild of Sample. """
pipe_path = os.path.join(tmpdir.strpath, name_atac_pipe)

pipe_path = os.path.join(tmpdir.strpath, atac_pipe_name)
intermediate_sample_subtype = "Middle"
leaf_sample_subtype = "Leaf"

intermediate_subtype_lines = _class_definition_lines(
intermediate_sample_subtype, Sample.__name__)
leaf_subtype_lines = _class_definition_lines(
leaf_sample_subtype, intermediate_sample_subtype)
with open(pipe_path, 'w') as pipe_mod_file:
pipe_mod_file.write("{}\n\n".format(SAMPLE_IMPORT))
for l in intermediate_subtype_lines:
pipe_mod_file.write(l)
pipe_mod_file.write("\n\n")
for l in leaf_subtype_lines:
pipe_mod_file.write(l)
atacseq_piface_data[SUBTYPES_KEY] = target if spec_type == "single" \
else {ATAC_PROTOCOL_NAME: target}

atacseq_piface_data[atac_pipe_name][SUBTYPES_KEY] = \
target if spec_type == "single" else \
{ATAC_PROTOCOL_NAME: target}
conf_path = _write_config_data(
protomap={ATAC_PROTOCOL_NAME: name_atac_pipe},
protomap={ATAC_PROTOCOL_NAME: atac_pipe_name},
conf_data=atacseq_piface_data, dirpath=tmpdir.strpath)

piface = ProtocolInterface(conf_path)
subtype = piface.fetch_sample_subtype(
protocol=ATAC_PROTOCOL_NAME, strict_pipe_key=name_atac_pipe,
protocol=ATAC_PROTOCOL_NAME, strict_pipe_key=atac_pipe_name,
full_pipe_path=pipe_path)

assert target == subtype.__name__


Expand Down Expand Up @@ -635,7 +648,7 @@ def atacseq_piface_data_with_subtypes(

def _class_definition_lines(name, name_super_type):
""" Create lines that define a class. """
return ["class {t}({st}):\n".format(name),
return ["class {t}({st}):\n".format(t=name, st=name_super_type),
"\tdef __init__(self, *args, **kwarggs):\n",
"\t\tsuper({t}, self).__init__(*args, **kwargs)".format(
t=name, st=name_super_type)]
Expand All @@ -650,9 +663,8 @@ def _create_module(lines_by_class, filepath):
:param str filepath: path to module file to create
:return str: path to the module file written
"""
header = "from looper.models import Sample"
lines = "\n\n".join(
[header] + ["\n".join(class_lines)
[SAMPLE_IMPORT] + ["\n".join(class_lines)
for class_lines in lines_by_class])
with open(filepath, 'w') as modfile:
modfile.write("{}\n".format(lines))
Expand Down