Skip to content

Commit

Permalink
build: remove all trace of DEVELOPER.
Browse files Browse the repository at this point in the history
If you previously configured with `--enable-developer` we turn that into `--enable-debugbuild`.

Signed-off-by: Rusty Russell <[email protected]>
Changelog-Removed: build: `--enable-developer` arg to configure (and DEVELOPER variables): use `./configure --enable-debugbuild` and `developer` setting at runtime.
  • Loading branch information
rustyrussell committed Sep 21, 2023
1 parent 6c15ea4 commit 3d3bf87
Show file tree
Hide file tree
Showing 16 changed files with 37 additions and 102 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/bsd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:
git config --global --add safe.directory `pwd`
git submodule update --init --recursive
./configure CC="$CC" --enable-developer --disable-valgrind
./configure CC="$CC" --disable-valgrind
cat config.vars
Expand Down
58 changes: 20 additions & 38 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,22 +77,18 @@ jobs:
fail-fast: true
matrix:
include:
- CFG: gcc-dev1
DEVELOPER: 1
- CFG: compile-gcc
VALGRIND: 1
COMPILER: gcc
- CFG: gcc-dev0
DEVELOPER: 0
- CFG: compile-gcc-O3
VALGRIND: 1
COMPILER: gcc
COPTFLAGS_VAR: COPTFLAGS="-O3 -Werror"
# While we're at it let's try to compile with clang
- CFG: clang-dev1
DEVELOPER: 1
- CFG: compile-clang
VALGRIND: 1
COMPILER: clang
- CFG: clang-sanitizers
DEVELOPER: 1
- CFG: compile-clang-sanitizers
COMPILER: clang
ASAN: 1
UBSAN: 1
Expand All @@ -112,7 +108,6 @@ jobs:
- name: Build
env:
DEVELOPER: ${{ matrix.DEVELOPER }}
COMPILER: ${{ matrix.COMPILER }}
ASAN: ${{ matrix.ASAN }}
UBSAN: ${{ matrix.UBSAN }}
Expand Down Expand Up @@ -151,9 +146,9 @@ jobs:
fail-fast: true
matrix:
include:
- CFG: gcc-dev1
- CFG: compile-gcc
VALGRIND: 1
- CFG: clang-sanitizers
- CFG: compile-clang-sanitizers
VALGRIND: 0
steps:
- name: Checkout
Expand Down Expand Up @@ -209,7 +204,7 @@ jobs:
- name: Build
run: |
./configure --enable-debugbuild --enable-fuzzing --enable-developer --disable-valgrind CC=clang
./configure --enable-debugbuild --enable-fuzzing --disable-valgrind CC=clang
make -j $(nproc) check-fuzz
integration:
Expand All @@ -227,55 +222,43 @@ jobs:
fail-fast: true
matrix:
include:
- NAME: gcc-dev1
CFG: gcc-dev1
DEVELOPER: 1
TEST_DB_PROVIDER: sqlite3
COMPILER: gcc
TEST_NETWORK: regtest
- NAME: gcc-dev0
CFG: gcc-dev0
DEVELOPER: 0
- NAME: gcc
CFG: compile-gcc
TEST_DB_PROVIDER: sqlite3
COMPILER: gcc
TEST_NETWORK: regtest
# While we're at it let's try to compile with clang
- NAME: clang-dev1
CFG: clang-dev1
DEVELOPER: 1
- NAME: clang
CFG: compile-clang
TEST_DB_PROVIDER: sqlite3
COMPILER: clang
TEST_NETWORK: regtest
# And of course we want to test postgres too
- NAME: postgres
CFG: gcc-dev1
DEVELOPER: 1
CFG: compile-gcc
COMPILER: gcc
TEST_DB_PROVIDER: postgres
TEST_NETWORK: regtest
# And don't forget about elements (like cdecker did when
# reworking the CI...)
- NAME: liquid
CFG: gcc-dev1
DEVELOPER: 1
CFG: compile-gcc
COMPILER: gcc
TEST_NETWORK: liquid-regtest
TEST_DB_PROVIDER: sqlite3
# And dual funding!
- NAME: dual-fund
CFG: gcc-dev1
CFG: compile-gcc
TEST_DB_PROVIDER: sqlite3
COMPILER: gcc
TEST_NETWORK: regtest
DEVELOPER: 1
EXPERIMENTAL_DUAL_FUND: 1
# And splicing!
- NAME: splicing
CFG: gcc-dev1
CFG: compile-gcc
TEST_DB_PROVIDER: sqlite3
COMPILER: gcc
TEST_NETWORK: regtest
DEVELOPER: 1
EXPERIMENTAL_SPLICING: 1
steps:
- name: Checkout
Expand All @@ -301,7 +284,6 @@ jobs:

- name: Test
env:
DEVELOPER: ${{ matrix.DEVELOPER }}
COMPILER: ${{ matrix.COMPILER }}
EXPERIMENTAL_DUAL_FUND: ${{ matrix.EXPERIMENTAL_DUAL_FUND }}
EXPERIMENTAL_SPLICING: ${{ matrix.EXPERIMENTAL_SPLICING }}
Expand All @@ -325,7 +307,7 @@ jobs:
BITCOIN_VERSION: "25.0"
ELEMENTS_VERSION: 22.0.2
RUST_PROFILE: release # Has to match the one in the compile step
CFG: gcc-dev1
CFG: compile-gcc
PYTEST_OPTS: --test-group-random-seed=42 --timeout=1800
needs:
- compile
Expand Down Expand Up @@ -375,10 +357,10 @@ jobs:
- name: Download build
uses: actions/download-artifact@v3
with:
name: cln-gcc-dev1.tar.bz2
name: cln-compile-gcc.tar.bz2

- name: Unpack build
run: tar -xvjf cln-gcc-dev1.tar.bz2
run: tar -xvjf cln-compile-gcc.tar.bz2

- name: Test
env:
Expand Down Expand Up @@ -448,10 +430,10 @@ jobs:
- name: Download build
uses: actions/download-artifact@v3
with:
name: cln-clang-sanitizers.tar.bz2
name: cln-compile-clang-sanitizers.tar.bz2

- name: Unpack build
run: tar -xvjf cln-clang-sanitizers.tar.bz2
run: tar -xvjf cln-compile-clang-sanitizers.tar.bz2

- name: Test
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/macos.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,5 @@ jobs:
slow_test: marks tests as slow (deselect with '-m "not slow_test"')
EOF
python3.10 -m poetry run ./configure --enable-developer --disable-valgrind --disable-compat
python3.10 -m poetry run ./configure --disable-valgrind --disable-compat
python3.10 -m poetry run make
1 change: 0 additions & 1 deletion .github/workflows/prototest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ jobs:
-e NETWORK=${{ matrix.network }} \
-e TARGET_HOST=${{ matrix.TARGET_HOST }} \
-e VALGRIND=${{ matrix.valgrind }} \
-e DEVELOPER=1 \
-e PYTEST_PAR=2 \
-e PYTEST_OPTS="--timeout=300" \
-e TEST_CMD="make check-protos" \
Expand Down
3 changes: 1 addition & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,6 @@ COPY . /tmp/lightning
RUN git clone --recursive /tmp/lightning . && \
git checkout $(git --work-tree=/tmp/lightning --git-dir=/tmp/lightning/.git rev-parse HEAD)

ARG DEVELOPER=1
ENV PYTHON_VERSION=3
RUN curl -sSL https://install.python-poetry.org | python3 -

Expand All @@ -113,7 +112,7 @@ RUN /root/.local/bin/poetry export -o requirements.txt --without-hashes --with d
RUN pip3 install -r requirements.txt

RUN ./configure --prefix=/tmp/lightning_install --enable-static && \
make DEVELOPER=${DEVELOPER} && \
make && \
/root/.local/bin/poetry run make install

FROM debian:bullseye-slim as final
Expand Down
14 changes: 2 additions & 12 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -456,20 +456,16 @@ check-protos: $(ALL_PROGRAMS)
ifeq ($(PYTEST),)
@echo "py.test is required to run the protocol tests, please install using 'pip3 install -r requirements.txt', and rerun 'configure'."; false
else
ifeq ($(DEVELOPER),1)
@(cd external/lnprototest && PYTHONPATH=$(MY_CHECK_PYTHONPATH) LIGHTNING_SRC=../.. $(PYTEST) --runner lnprototest.clightning.Runner $(PYTEST_OPTS))
else
@echo "lnprototest target requires DEVELOPER=1, skipping"
endif
endif

pytest: $(ALL_PROGRAMS) $(DEFAULT_TARGETS) $(ALL_TEST_PROGRAMS) $(ALL_TEST_GEN)
ifeq ($(PYTEST),)
@echo "py.test is required to run the integration tests, please install using 'pip3 install -r requirements.txt', and rerun 'configure'."
exit 1
else
# Explicitly hand DEVELOPER and VALGRIND so you can override on make cmd line.
PYTHONPATH=$(MY_CHECK_PYTHONPATH) TEST_DEBUG=1 DEVELOPER=$(DEVELOPER) VALGRIND=$(VALGRIND) $(PYTEST) tests/ $(PYTEST_OPTS)
# Explicitly hand VALGRIND so you can override on make cmd line.
PYTHONPATH=$(MY_CHECK_PYTHONPATH) TEST_DEBUG=1 VALGRIND=$(VALGRIND) $(PYTEST) tests/ $(PYTEST_OPTS)
endif

check-fuzz: $(ALL_FUZZ_TARGETS)
Expand Down Expand Up @@ -734,13 +730,7 @@ pyln-release: $(PYLNS:%=pyln-release-%)
pyln-release-%:
cd contrib/pyln-$* && $(MAKE) prod-release

# These must both be enabled for update-mocks
ifeq ($(DEVELOPER),1)
update-mocks: $(ALL_TEST_PROGRAMS:%=update-mocks/%.c)
else
update-mocks:
@echo Need DEVELOPER=1 to regenerate mocks >&2; exit 1
endif

$(ALL_TEST_PROGRAMS:%=update-mocks/%.c): $(ALL_GEN_HEADERS) $(EXTERNAL_LIBS) libccan.a ccan/ccan/cdump/tools/cdump-enumstr config.vars

Expand Down
8 changes: 0 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,6 @@ of the `startup_regtest.sh` file for details on how to use it.
. contrib/startup_regtest.sh
```

Note that your local nodeset will be much faster/more responsive if
you've configured your node to expose the developer options, e.g.

```bash
./configure --enable-developer
```

#### Mainnet Option
To test with real bitcoin, you will need to have a local `bitcoind` node running:

Expand Down Expand Up @@ -211,7 +204,6 @@ If you encrypt your `hsm_secret`, you will have to pass the `--encrypted-hsm` st
### Developers

Developers wishing to contribute should start with the developer guide [here](doc/contribute-to-core-lightning/coding-style-guidelines.md).
You should also configure with `--enable-developer` to get additional checks and options.

[blockstream-store-blog]: https://blockstream.com/2018/01/16/en-lightning-charge/
[std]: https://github.com/lightning/bolts
Expand Down
11 changes: 0 additions & 11 deletions common/utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -131,17 +131,6 @@ STRUCTEQ_DEF(ripemd160, 0, u);
* complained on, so we can re-test as gcc evolves. */
#define COMPILER_WANTS_INIT(compiler_versions) = 0

/* For case where we want one thing if DEVELOPER, another if not, particularly
* for function parameters.
*
* Usefully, you can refer to DEVELOPER-only fields here. */
#if DEVELOPER
/* Make sure that nondev is evaluated, and valid */
#define IFDEV(dev, nondev) ((void)(nondev), (dev))
#else
#define IFDEV(dev, nondev) (nondev)
#endif

/* Context which all wally allocations use (see common/setup.c) */
extern const tal_t *wally_tal_ctx;

Expand Down
8 changes: 0 additions & 8 deletions configure
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,7 @@ set_defaults()
PREFIX=${PREFIX:-/usr/local}
CC=${CC:-cc}
CDEBUGFLAGS=${CDEBUGFLAGS--std=gnu11 -g -fstack-protector-strong}
DEVELOPER=${DEVELOPER:-0}
DEBUGBUILD=${DEBUGBUILD:-0}
# --enable-developer forces debug build
[ "$DEVELOPER" = 0 ] || DEBUGBUILD=1
COMPAT=${COMPAT:-1}
STATIC=${STATIC:-0}
CLANG_COVERAGE=${CLANG_COVERAGE:-0}
Expand Down Expand Up @@ -208,8 +205,6 @@ usage()
echo "Options include:"
usage_with_default "--prefix=" "$PREFIX"
echo " Prefix for make install"
usage_with_default "--enable/disable-developer" "$DEVELOPER" "enable" "disable"
echo " Developer mode, good for testing"
usage_with_default "--enable/disable-debugbuild" "$DEBUGBUILD" "enable" "disable"
echo " Extra debug checks in the build, good for testing"
usage_with_default "--enable/disable-compat" "$COMPAT" "enable" "disable"
Expand Down Expand Up @@ -273,8 +268,6 @@ for opt in "$@"; do
COPTFLAGS=*) COPTFLAGS="${opt#COPTFLAGS=}";;
PYTEST=*) PYTEST="${opt#PYTEST=}";;
--prefix=*) PREFIX="${opt#--prefix=}";;
--enable-developer) DEVELOPER=1;;
--disable-developer) DEVELOPER=0;;
--enable-debugbuild) DEBUGBUILD=1;;
--disable-debugbuild) DEBUGBUILD=0;;
--enable-compat) COMPAT=1;;
Expand Down Expand Up @@ -521,7 +514,6 @@ add_var SQLITE3_LDLIBS "$SQLITE3_LDLIBS"
add_var POSTGRES_INCLUDE "$POSTGRES_INCLUDE"
add_var POSTGRES_LDLIBS "$POSTGRES_LDLIBS"
add_var VALGRIND "$VALGRIND"
add_var DEVELOPER "$DEVELOPER" $CONFIG_HEADER.$$
add_var DEBUGBUILD "$DEBUGBUILD"
add_var COMPAT "$COMPAT" $CONFIG_HEADER.$$
add_var PYTEST "$PYTEST"
Expand Down
1 change: 0 additions & 1 deletion contrib/docker/scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ export BOLTDIR=bolts
export CC=${COMPILER:-gcc}
export COMPAT=${COMPAT:-1}
export TEST_CHECK_DBSTMTS=${TEST_CHECK_DBSTMTS:-0}
export DEVELOPER=${DEVELOPER:-1}
export PATH=$CWD/dependencies/bin:"$HOME"/.local/bin:"$PATH"
export PYTEST_OPTS="--maxfail=5 --suppress-no-test-exit-code ${PYTEST_OPTS}"
export PYTEST_PAR=${PYTEST_PAR:-10}
Expand Down
6 changes: 0 additions & 6 deletions doc/beginners-guide/beginners-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,6 @@ If you want to experiment with `lightningd`, there's a script to set up a `bitco
. contrib/startup_regtest.sh
```

Note that your local nodeset will be much faster/more responsive if you've configured your node to expose the developer options, e.g.

```bash
./configure --enable-developer
```

#### Mainnet Option

To test with real bitcoin, you will need to have a local `bitcoind` node running:
Expand Down
6 changes: 3 additions & 3 deletions doc/contribute-to-core-lightning/contributor-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,20 @@ sudo apt install valgrind cppcheck shellcheck libsecp256k1-dev libpq-dev
Re-run `configure` and build using `make`:

```shell
./configure --enable-developer
./configure
make -j$(nproc)
```

## Debugging

You can build Core Lightning with `DEVELOPER=1` to use dev commands listed in `cli/lightning-cli help`. `./configure --enable-developer` will do that. You can log console messages with log_info() in lightningd and status_debug() in other subdaemons.
There are various development options enabled by running with `--developer`. You can log console messages with log_info() in lightningd and status_debug() in other subdaemons.

You can debug crashing subdaemons with the argument `--dev-debugger=channeld`, where `channeld` is the subdaemon name. It will run `gnome-terminal` by default with a gdb attached to the subdaemon when it starts. You can change the terminal used by setting the `DEBUG_TERM` environment variable, such as `DEBUG_TERM="xterm -e"` or `DEBUG_TERM="konsole -e"`.

It will also print out (to stderr) the gdb command for manual connection. The subdaemon will be stopped (it sends itself a `SIGSTOP`); you'll need to `continue` in gdb.

```shell
./configure --enable-developer
./configure
make -j$(nproc)
```

Expand Down
7 changes: 3 additions & 4 deletions doc/contribute-to-core-lightning/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,14 @@ updatedAt: "2023-07-13T05:21:39.220Z"
Tests are run with: `make check [flags]` where the pertinent flags are:

```shell
DEVELOPER=[0|1] - developer mode increases test coverage
VALGRIND=[0|1] - detects memory leaks during test execution but adds a significant delay
PYTEST_PAR=n - runs pytests in parallel
```

A modern desktop can build and run through all the tests in a couple of minutes with:

```shell Shell
make -j12 full-check PYTEST_PAR=24 DEVELOPER=1 VALGRIND=0
make -j12 full-check PYTEST_PAR=24 VALGRIND=0
```

Adjust `-j` and `PYTEST_PAR` accordingly for your hardware.
Expand Down Expand Up @@ -92,7 +91,7 @@ In order to build the Core Lightning binaries with code coverage you will need a
Then you'll need to enable support at configuration time. You likely want to enable a few sanitizers for bug detections as well as experimental features for an extended coverage (not required though).

```shell
./configure --enable-developer --enable-address-sanitizer --enable-ub-sanitizer --enable-fuzzing --disable-valgrind CC=clang && make
./configure --enable-address-sanitizer --enable-ub-sanitizer --enable-fuzzing --disable-valgrind CC=clang && make
```

The targets will be built in `tests/fuzz/` as `fuzz-` binaries, with their best known seed corpora stored in `tests/fuzz/corpora/`.
Expand Down Expand Up @@ -169,4 +168,4 @@ In order to write a new target:
- fill two functions: `init()` for static stuff and `run()` which will be called repeatedly with mutated data.
- read about [what makes a good fuzz target](https://github.com/google/fuzzing/blob/master/docs/good-fuzz-target.md).

A simple example is [`fuzz-addr`](https://github.com/ElementsProject/lightning/blob/master/tests/fuzz/fuzz-addr.c). It setups the chainparams and context (wally, tmpctx, ..) in `init()` then bruteforces the bech32 encoder in `run()`.
A simple example is [`fuzz-addr`](https://github.com/ElementsProject/lightning/blob/master/tests/fuzz/fuzz-addr.c). It setups the chainparams and context (wally, tmpctx, ..) in `init()` then bruteforces the bech32 encoder in `run()`.
Loading

0 comments on commit 3d3bf87

Please sign in to comment.