Skip to content

Commit

Permalink
updated readme pr based on Pat's review
Browse files Browse the repository at this point in the history
  • Loading branch information
EricThomson committed Jun 2, 2023
1 parent b9ceae8 commit f8da88b
Show file tree
Hide file tree
Showing 4 changed files with 59 additions and 51 deletions.
12 changes: 8 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Second, let's say you want to improve something yourself:
- The demo notebooks
- The code base

We welcome all such contributions! To do this you need to make changes on your local version of Caiman then push this to the repository by making a **Pull Request** (PR). We will walk you through this process in the rest of the document.
We welcome all such contributions. To do this you need to make changes on your local version of Caiman then push this to the repository by making a **Pull Request** (PR). We will walk you through this process in the rest of the document.

Before you go through the work required to improve something, we recommend that you let us know your plans. E.g., open an issue, or reach out on Gitter. This way, we can avoid duplicated effort (if someone is already working on it), or wasted time (it could turn out the changes might not be feasible right now). We can usually set up a video chat on Google or Skype to talk about a feature proposal if that works for you.

Expand All @@ -28,7 +28,7 @@ The workflow for contributing to Caiman is roughly illustrated by the numbers in
1) Create a feature branch from `dev`.
2) Work on that feature until you are ready to push it to Caiman.
3) Make a PR: this is when you request that your changes become merged into `dev` at Caiman. This merge won't be immediate, you will get feedback on your code, and probably be asked to make some changes.
4) Eventually, once enough new features accumulate in the `dev` branch to merit a new release (every month or so), the `dev` branch will be merged with `main`. This will become a new version of Caiman that people download when they run `mamba install caiman`.
4) Periodically, as features accumulate in the `dev` branch (every month or so), the `dev` branch will be merged with `main`. This will become a new version of Caiman that people download when they run `mamba install caiman`.

Below we have instructions on how to do all of the above steps. While all of this may seem like a lot, some of the steps are extremely simple. Also, once you have done it once, you will have the recipe and it will be pretty easy. Finally, it is a very rewarding experience to contribute to an open source project -- we hope you'll take the plunge!

Expand Down Expand Up @@ -61,11 +61,15 @@ Once your feature branch is ready, it's time to make a PR (label 3 in the figure

git push -u origin my_feature

Then, go to your local fork of Caiman at your github repo, and a large green button will show at the top, giving you the option of making a `Pull request`. When you click it, this will send over your feature for review (be sure to push to merge with dev branch, not main, and to fill out the PR template that is auto-generated by Caiman with the information requested).
Then, go to your fork of Caiman at your personal GitHub repo, and a large green button will show at the top that gives you the option of making a pull request. Here is an example from a feature branch called `amazing_new_feature`:

<img src="docs/img/pull_request.jpg">

When you click the `Compare & pull request` button, it will bring up an interface to integrate your code at the main Caiman repo. Be sure to generate your PR targeting the `dev` branch, not `main`, and fill out the PR template that is auto-generated by Caiman with the information requested.

Note that all PRs are reviewed by other programmers. This is an important part of the process: they will almost always give comments, making suggestions or asking questions about changes using Github's pull request interface.

You may be asked to make some changes (or to think about making some changes). You will sometimes need to do more some more work on your branch and make more changes after making an initial PR. In this case, the workflow is simple: you will work within your your local `my_feature` branch as before, and run the `push` command again. Conveniently, this will automatically push the changes to the same work-in-progress PR at Caiman. Note that if your PR is open for too long, merge conflicts can start to emerge as the `dev` branch changes.
You may be asked to make some changes (or to *think* about making some changes). You will sometimes need to do more some more work on your branch and make more changes after making an initial PR. In this case, the workflow is simple: you will work within your your local `my_feature` branch as before, and run the `push` command again. Conveniently, this will automatically push the changes to the same work-in-progress PR at Caiman. Note that if your PR is open for too long, merge conflicts can start to emerge as the `dev` branch changes.

## Fourth, wait for the work to show up in main
Once your work is done, it will eventually be integrated into `main` by the developers who maintain Caiman (label 4 in the figure). This is done every month or two, and is the stage when your work will actually be visible to people who download Caiman. It's at this point your name will appear when you click on the [list Contributors](https://github.com/flatironinstitute/CaImAn/graphs/contributors) at Github -- time to give yourself a big pat on the back! We really appreciate the folks who go through all the work to help make the package better.
Expand Down
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ Jupyter will open. Click on `demo_pipeline.ipynb` to get started with a demo!
## For installation help
Caiman should install easily on Linux, Mac, and Windows. If you run into problems, we have a dedicated [installation page](./docs/source/Installation.rst): the details there should help you troubleshoot. If you don't find what you need there, *please* send us a [message on Gitter](https://app.gitter.im/#/room/#agiovann_Constrained_NMF:gitter.im), and we will help you get it sorted out.


# Demo notebooks :page_with_curl:
Once you've gotten things set up, the best way to get started learning Caiman is to select the demo notebook that most closely matches your use case, and adapt it for your particular needs.

Expand Down Expand Up @@ -67,7 +66,7 @@ In addition to the demos, there are many ways to learn more about Caiman and rec
In generaly, you don't have to need help to get involved. We love when users talk about use cases answer other people's questions, etc at Gitter and Github.

# How to contribute :hammer:
Caiman is an open-source project and improves because of contributions from users all over the world. If there is something about Caiman that you would like to improve, then you are qualified to contribute! We are always looking for more contributors, so please come read the [contributors page](./CONTRIBUTING.md) for more details about how.
Caiman is an open-source project and improves because of contributions from users all over the world. If there is something about Caiman that you would like to improve, then you are qualified to contribute. We are always looking for more contributors, so please come read the [contributors page](./CONTRIBUTING.md) for more details about how.

# Videos
There are multiple online videos by Andrea Giovannucci from past Caiman workshops/events that are an excellent start for newcomers.
Expand Down
Binary file added docs/img/pull_request.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
95 changes: 50 additions & 45 deletions docs/source/Installation.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
Installation and Updating
=========================

This page will give more detailed setup instructions for Caiman than the `main readme <../../README.md>`_. This is the place
you should come first if you run into problems and need to troubleshoot. It includes info on initial setup, as well as
updating with new releases. Note there is a Table of Contents on the left-hand side of this page: there are a lot of
sections here, so if you get lost or need help with a particular task, that should help you get oriented.
This page will give more detailed setup instructions for Caiman than the `main readme <../../README.md>`_.
This is the place you should visit first if you run into problems and need to troubleshoot. It includes
info on initial setup, as well as updating with new releases. There is a Table of Contents on the
left-hand side of this page, so if you need help with a particular task, that should help you get oriented.

If the information on this page doesn't help, please ask us for
help at `Gitter <https://app.gitter.im/#/room/#agiovann_Constrained_NMF:gitter.im/>`_,
or open an `issue <https://github.com/flatironinstitute/CaImAn/issues/>`_.

Caiman setup consists of two main steps:

Expand Down Expand Up @@ -76,9 +80,11 @@ Section 1B. Development-mode install
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Dev mode install allows you to modify the source files of Caiman and makes it easier
to contribute to the project, fix bugs etc.. The general motivation for setting up
an editable development environment is described in detail in our `contributors page <../../CONTRIBUTING.md>`_.
to contribute to the project, fix bugs etc. The general motivation for setting up
an editable development environment is described in more detail in our `contributors page <../../CONTRIBUTING.md>`_.

If you install in dev mode you will likely need to set some environment variables manually (it is
done automatically when you do the conda install): this is discussed in Section 4C.

Dev-mode install on Windows
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -90,8 +96,8 @@ Dev-mode install on Windows

The Windows installation process differs more widely from installation
on Linux or MacOSX and has different issues you may run into. Everything
you do should be from the anaconda prompt rather than from Powershell
or any other shell.
you do should be from a Conda-enabled shell (which you can find in your
Programs menu) rather than from Powershell or any other shell.

- Remove any associations you may have made between .py files and an existing python
interpreter or editor
Expand Down Expand Up @@ -322,35 +328,8 @@ To update ``caiman_data`` you can follow the following procedure:
Section 4: Miscellaneous
-------------------------

Section 4A: Setting up environment variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To make the package work *efficiently* and eliminate "crosstalk" between
different processes, some multithreading operations need to be turned off
This is for Linux and Windows and is not necessary is OSX. This process is
not needed if you installed using conda/mamba.

For **Linux (and OSX)** run these commands before launching Python:

.. code:: bash
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export VECLIB_MAXIMUM_THREADS=1
For **Windows** run the same commands, replacing the word ```export``` with the word ```set```.

The commands should be run *every time* before launching python. It is
recommended that you save these values inside your environment so you
don’t have to repeat this process every time. You can do this by
following the instructions
`here <https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#saving-environment-variables>`__.

**If you installed using the conda-forge package, this is done automatically for you.**


Section 4B: Requirements
~~~~~~~~~~~~~~~~~~~~~~~~
Section 4A: System Requirements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

32G RAM is required for a good experience, and depending on datasets, more may be necessary. Caiman is optimized for use by
multiple CPUs, so workstations or clusters with multiple CPU cores are ideal (8+ logical cores). GPU computation is not used
Expand All @@ -359,15 +338,15 @@ heavily by Caiman (though see Section 4D).
Right now, Caiman works and is supported on the following platforms:

- Linux on 64-bit x86 CPUs
- MacOS on 64-bit x86 CPUs
- MacOS on 64-bit x86 CPUs or ARM CPUs
- Windows on 64-bit x86 CPUs

ARM-based versions of Apple hardware work (if on a 16G model), but currently happen under x86 emulation and we cannot
support them as well. Support for Linux on ARM (e.g. AWS Graviton) is not available (but it may work with the port of conda,
if you compile Caiman yourself - we do not have binary packages and this is untested). If you care about this, please let us know.
Support for Linux on ARM (e.g. AWS Graviton) is not available (but it may work with the port of conda,
if you compile Caiman yourself - we do not have binary packages and this is untested). If you care about this,
please let us know.


Section 4C: Installing additional packages
Section 4B: Installing additional packages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Caiman installs through the conda-forge conda channel. Some packages are available on multiple conda channels, and in this
Expand All @@ -383,9 +362,35 @@ that they are from conda-forge, with none of them having a blank origin. If you
differences between how packages are built in different channels could lead to some packages failing to work
(e.g., OpenCV).

Section 4C: Setting up environment variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is only important for people who are doing the dev-mode install. If you
installed using the conda packaged-based install, this is done automatically.

To make the package work *efficiently* and eliminate "crosstalk" between
different processes, some multithreading operations need to be turned off
This is for Linux and Windows and is not necessary in OSX.

For **Linux** run these commands before launching Python:

.. code:: bash
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export VECLIB_MAXIMUM_THREADS=1
For **Windows** run the same commands, replacing the word ```export``` with the word ```set```.

The commands should be run *every time* before launching python. It is
recommended that you save these values inside your environment so you
don’t have to repeat this process every time. You can do this by
following the instructions
`here <https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#saving-environment-variables>`__.


Section 4D: Other topics
~~~~~~~~~~~~~~~~~~~~~~~~~

- `Running Caiman on a cluster <./Cluster.md>`_
- `Setting up Caiman to use your GPUs <./README-GPU.md>`_
- `Install quirks on some older Linux distributions <./README-Distros.md>`_
- `Running Caiman on a cluster <./CLUSTER.md>`_
- `Setting up Caiman to use your GPUs <./README-GPU.md>`_

0 comments on commit f8da88b

Please sign in to comment.