Unable to create HTML report (Cellbender version 0.3.0)

[TeodoraTockovska]

Hello Cellbender team!

I was browsing the troubleshooting page here, and it states to report any issues if HTML files fail to generate. My log file reported an error and it couldn't generate the HTML report. Please see the error message below:

cellbender:remove-background: Unable to create report.
cellbender:remove-background: Traceback (most recent call last):
  File "/cellbender/0.3.0/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 351, in compute_output_denoised_counts_reports_metrics
    output=html_report_file,
  File "/cellbender/0.3.0/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 82, in run_notebook_make_html
    title=('CellBender: ' + os.path.basename(output).replace('_report.html', '')),
  File "/cellbender/0.3.0/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 59, in _postprocess_html
    with open(file, mode='r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'sample1_cellbender/cellbender_feature_bc_matrix_report.html'

I ran the tool using the following command, omitting "--cuda" (ran using CPU instead of GPU):

input_file_h5_path="../../samples/sample1/outs/raw_feature_bc_matrix.h5"
cellbender remove-background --input $input_file_h5_path --output sample1_cellbender/cellbender_feature_bc_matrix.h5

[sjfleming]

Thanks for reporting this! Do you think you're seeing the same thing reported here? #273 or here? #265

Are you using Windows?

[TeodoraTockovska]

I ran Cellbender in a linux environment (on a high performance computing cluster). I do not see any other errors and every other file generated including the CSV files. I am not seeing the same issues as #273 and #265.

[sjfleming]

Okay thanks for clarifying. Is there any more to that traceback or error message? It looks like it is erroring out when I try to postprocess the HTML here

CellBender/cellbender/remove_background/report.py

Line 80 in 4990df7

_postprocess_html(

At that point, the HTML file should already exist, but it seems the error is saying the file does not exist. I am a bit confused because I would be expecting some other error message earlier on that indicates that the HTML was not produced for some other reason.

[TeodoraTockovska]

There are no other traceback or error messages. I can send you the entire log file. Is there an email that I can send it to?

[sjfleming]

Yeah that would be great! [email protected]

[sjfleming]

I think I may need to be able to side-step the postprocessing function by creating a try ... except around it. That would "solve" this issue and #273 . Though I'd still love to figure out what's causing the real problem.

[jpcartailler]

FWIW, we're also experiencing the exact same issue as @TeodoraTockovska, on a Linux cluster, running CellBender v0.3.0 (from singularity container). I agree with @sjfleming as well, the error indicates the file should already exist but doesn't, so it's probably a red herring.

(re-edit)

I re-rested using the tiny dataset generated via your generate_tiny_10x_dataset.py:

cellbender remove-background  --input tiny_raw_feature_bc_matrix.h5ad --output output/output_file.h5     --cpu-threads 24

It works in Docker, but didn't in Singularity.

The issue (in our case) is that Singularity was loading .local libraries from OUTSIDE the container, effectively breaking the environment:

ImportError: cannot import name 'ensure_async' from 'jupyter_core.utils' (/home/user/.local/lib/python3.7/site-packages/jupyter_core/utils/__init__.py)

This was preventing the temporary tmp.report.nbconvert.html file to be first created, leading to the second (reported message). I'm not sure why the previous Python messages are being buried.

In any case, for anyone else using Singularity, be sure to use the --no-home option when via the shell or exec singularity functions:

singularity exec --cleanenv --no-home /path/to/cellbender-image.sif

Problem fixed