doc: Add guides and GitHub workflow for doc sync

This PR:
- adds all the guides (in markdown format) that is published at https://docs.corelightning.org/docs
- adds a github workflow to sync any future changes made to files inside the guides folder
- does not include API reference (json-rpc commands). Those will be handled in a separate PR since they're used as manpages and will require a different github workflow

Note that the guides do not exactly map to their related files in doc/, since we reorganized the overall documentation structure on readme for better readability and developer experience. For example, doc/FUZZING.md and doc/HACKING.md#Testing are merged into testing.md in the new docs. As on the creation date of this PR, content from each of the legacy documents has been synced with the new docs. Until this PR gets merged, I will continue to push any updates made to the legacy documents into the new docs.

If this looks reasonable, I will add a separate PR to clean up the legacy documents from doc/ (or mark them deprecated) to avoid redundant upkeep and maintenance.

Changelog-None

.github/workflows/rdme-docs-sync.yml

@@ -0,0 +1,23 @@
+# This GitHub Actions workflow was auto-generated by the `rdme` cli on 2023-04-22T13:16:28.430Z
+# You can view our full documentation here: https://docs.readme.com/docs/rdme
+name: ReadMe GitHub Action 🦉
+
+on:
+  push:
+    branches:
+      # This workflow will run every time you push code to the following branch: `master`
+      # Check out GitHub's docs for more info on configuring this:
+      # https://docs.github.com/actions/using-workflows/events-that-trigger-workflows
+      - master
+
+jobs:
+  rdme-docs:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repo 📚
+        uses: actions/checkout@v3
+
+      - name: Run `docs` command 🚀
+        uses: readmeio/rdme@v8
+        with:
+          rdme: docs guides/ --key=${{ secrets.README_API_KEY }} --version=23.02

doc/guides/Beginner-s Guide/beginners-guide.md

@@ -0,0 +1,76 @@
+---
+title: "Running your node"
+slug: "beginners-guide"
+excerpt: "A guide to all the basics you need to get up and running immediately."
+hidden: false
+createdAt: "2022-11-18T14:27:50.098Z"
+updatedAt: "2023-02-21T13:49:20.132Z"
+---
+## Starting `lightningd`
+
+#### Regtest (local, fast-start) option
+
+If you want to experiment with `lightningd`, there's a script to set up a `bitcoind` regtest test network of two local lightning nodes, which provides a convenient `start_ln` helper. See the notes at the top of the `startup_regtest.sh` file for details on how to use it.
+
+```bash
+. contrib/startup_regtest.sh
+```
+
+
+
+Note that your local nodeset will be much faster/more responsive if you've configured your node to expose the developer options, e.g.
+
+```bash
+./configure --enable-developer
+```
+
+
+
+#### Mainnet Option
+
+To test with real bitcoin,  you will need to have a local `bitcoind` node running:
+
+```bash
+bitcoind -daemon
+```
+
+
+
+Wait until `bitcoind` has synchronized with the network.
+
+Make sure that you do not have `walletbroadcast=0` in your `~/.bitcoin/bitcoin.conf`, or you may run into trouble.  
+Notice that running `lightningd` against a pruned node may cause some issues if not managed carefully, see [pruning](doc:bitcoin-core##using-a-pruned-bitcoin-core-node) for more information.
+
+You can start `lightningd` with the following command:
+
+```bash
+lightningd --network=bitcoin --log-level=debug
+```
+
+
+
+This creates a `.lightning/` subdirectory in your home directory: see `man -l doc/lightningd.8` (or [???](???)) for more runtime options.
+
+## Using The JSON-RPC Interface
+
+Core Lightning exposes a [JSON-RPC 2.0](https://www.jsonrpc.org/specification) interface over a Unix Domain socket; the [`lightning-cli`](ref:lightning-cli) tool can be used to access it, or there is a [python client library](???).
+
+You can use `[lightning-cli](ref:lightning-cli) help` to print a table of RPC methods; `[lightning-cli](lightning-cli) help <command>` will offer specific information on that command.
+
+Useful commands:
+
+- [lightning-newaddr](ref:lightning-newaddr): get a bitcoin address to deposit funds into your lightning node.
+- [lightning-listfunds](ref:lightning-listfunds): see where your funds are.
+- [lightning-connect](ref:lightning-connect): connect to another lightning node.
+- [lightning-fundchannel](ref:lightning-fundchannel): create a channel to another connected node.
+- [lightning-invoice](ref:lightning-invoice): create an invoice to get paid by another node.
+- [lightning-pay](ref:lightning-pay): pay someone else's invoice.
+- [lightning-plugin](ref:lightning-plugin): commands to control extensions.
+
+## Care And Feeding Of Your New Lightning Node
+
+Once you've started for the first time, there's a script called `contrib/bootstrap-node.sh` which will connect you to other nodes on the lightning network.
+
+There are also numerous plugins available for Core Lightning which add capabilities: see the [Plugins](doc:plugins) guide, and check out the plugin collection at: <https://github.com/lightningd/plugins>, including [helpme](https://github.com/lightningd/plugins/tree/master/helpme) which guides you through setting up your first channels and customising your node.
+
+For a less reckless experience, you can encrypt the HD wallet seed: see [HD wallet encryption](doc:securing-keys).

doc/guides/Beginner-s Guide/opening-channels.md

@@ -0,0 +1,50 @@
+---
+title: "Opening channels"
+slug: "opening-channels"
+hidden: false
+createdAt: "2022-11-18T16:26:57.798Z"
+updatedAt: "2023-01-31T15:07:08.196Z"
+---
+First you need to transfer some funds to `lightningd` so that it can open a channel:
+
+```shell
+# Returns an address <address>
+lightning-cli newaddr
+```
+
+
+
+`lightningd` will register the funds once the transaction is confirmed.
+
+You may need to generate a p2sh-segwit address if the faucet does not support bech32:
+
+```shell
+# Return a p2sh-segwit address
+lightning-cli newaddr p2sh-segwit
+```
+
+
+
+Confirm `lightningd` got funds by:
+
+```shell
+# Returns an array of on-chain funds.
+lightning-cli listfunds
+```
+
+
+
+Once `lightningd` has funds, we can connect to a node and open a channel. Let's assume the **remote** node is accepting connections at `<ip>` (and optional `<port>`, if not 9735) and has the node ID `<node_id>`:
+
+```shell
+lightning-cli connect <node_id> <ip> [<port>]
+lightning-cli fundchannel <node_id> <amount_in_satoshis>
+```
+
+
+
+This opens a connection and, on top of that connection, then opens a channel. 
+
+The funding transaction needs 3 confirmations in order for the channel to be usable, and 6 to be announced for others to use.
+
+You can check the status of the channel using `lightning-cli listpeers`, which after 3 confirmations (1 on testnet) should say that `state` is `CHANNELD_NORMAL`; after 6 confirmations you can use `lightning-cli listchannels` to verify that the `public` field is now `true`.

doc/guides/Beginner-s Guide/securing-keys.md

@@ -0,0 +1,10 @@
+---
+title: "Securing keys"
+slug: "securing-keys"
+hidden: false
+createdAt: "2022-11-18T16:28:08.529Z"
+updatedAt: "2023-01-31T13:52:27.300Z"
+---
+You can encrypt the `hsm_secret` content (which is used to derive the HD wallet's master key) by passing the `--encrypted-hsm` startup argument, or by using the `hsmtool` (which you can find in the `tool/` directory at the root of [Core Lightning repository](https://github.com/ElementsProject/lightning)) with the `encrypt` method. You can unencrypt an encrypted `hsm_secret` using the `hsmtool` with the `decrypt` method.
+
+If you encrypt your `hsm_secret`, you will have to pass the `--encrypted-hsm` startup option to `lightningd`. Once your `hsm_secret` is encrypted, you **will not** be able to access your funds without your password, so please beware with your password management. Also, beware of not feeling too safe with an encrypted `hsm_secret`: unlike for `bitcoind` where the wallet encryption can restrict the usage of some RPC command, `lightningd` always needs to access keys from the wallet which is thus **not locked** (yet), even with an encrypted BIP32 master seed.

doc/guides/Beginner-s Guide/sending-and-receiving-payments.md

@@ -0,0 +1,24 @@
+---
+title: "Sending and receiving payments"
+slug: "sending-and-receiving-payments"
+hidden: false
+createdAt: "2022-11-18T16:27:07.625Z"
+updatedAt: "2023-01-31T15:06:02.214Z"
+---
+Payments in Lightning are invoice based.
+
+The recipient creates an invoice with the expected `<amount>` in millisatoshi (or `"any"` for a donation), a unique `<label>` and a `<description>` the payer will see:
+
+```shell
+lightning-cli invoice <amount> <label> <description>
+```
+
+This returns some internal details, and a standard invoice string called `bolt11` (named after the [BOLT #11 lightning spec](https://github.com/lightning/bolts/blob/master/11-payment-encoding.md)).
+
+The sender can feed this `bolt11` string to the `decodepay` command to see what it is, and pay it simply using the `pay` command:
+
+```shell
+lightning-cli pay <bolt11>
+```
+
+Note that there are lower-level interfaces (and more options to these interfaces) for more sophisticated use.

doc/guides/Beginner-s Guide/watchtowers.md

@@ -0,0 +1,13 @@
+---
+title: "Watchtowers"
+slug: "watchtowers"
+excerpt: "Defend your node against breaches using a watchtower."
+hidden: false
+createdAt: "2022-11-18T16:28:27.054Z"
+updatedAt: "2023-02-02T07:13:57.111Z"
+---
+The Lightning Network protocol assumes that a node is always online and synchronised with the network. Should your lightning node go offline for some time, it is possible that a node on the other side of your channel may attempt to force close the channel with an outdated state (also known as revoked commitment). This may allow them to steal funds from the channel that belonged to you. 
+
+A watchtower is a third-party service that you can hire to defend your node against such breaches, whether malicious or accidental, in the event that your node goes offline. It will watch for breaches on the blockchain and punish the malicious peer by relaying a penalty transaction on your behalf.
+
+There are a number of watchtower services available today. One of them is the [watchtower client plugin](https://github.com/talaia-labs/rust-teos/tree/master/watchtower-plugin) that works with the [Eye of Satoshi tower](https://github.com/talaia-labs/rust-teos) (or any [BOLT13](https://github.com/sr-gi/bolt13/blob/master/13-watchtowers.md) compliant watchtower).

doc/guides/Contribute to Core Lightning/code-generation.md

@@ -0,0 +1,74 @@
+---
+title: "Code Generation"
+slug: "code-generation"
+hidden: true
+createdAt: "2023-04-22T12:29:01.116Z"
+updatedAt: "2023-04-22T12:44:47.814Z"
+---
+The CLN project has a multitude of interfaces, most of which are generated from an abstract schema:
+
+- Wire format for peer-to-peer communication: this is the binary format that is specific by the [LN spec](https://github.com/lightning/bolts). It uses the [generate-wire.py](https://github.com/ElementsProject/lightning/blob/master/tools/generate-wire.py) script to parse the (faux) CSV files that are automatically extracted from the specification and writes C source code files that are then used internally to encode and decode messages, as well as provide print functions for the messages.
+
+- Wire format for inter-daemon communication: CLN follows a multi-daemon architecture, making communication explicit across daemons. For this inter-daemon communication we use a slightly altered message format from the [LN spec](https://github.com/lightning/bolts). The changes are 
+  1. addition of FD passing semantics to allow establishing a new connection between daemons (communication uses [socketpair](https://man7.org/linux/man-pages/man2/socketpair.2.html), so no `connect`)
+  2. change the message length prefix from `u16` to `u32`, allowing for messages larger than 65Kb. The CSV files are with the respective sub-daemon and also use [generate-wire.py](https://github.com/ElementsProject/lightning/blob/master/tools/generate-wire.py) to generate encoding, decoding and printing functions
+
+- We describe the JSON-RPC using [JSON Schema](https://json-schema.org/) in the [`doc/schemas`](https://github.com/ElementsProject/lightning/tree/master/doc/schemas) directory. Each method has a `.request.json` for the request message, and a `.schema.json` for the response (the mismatch is historical and will eventually be addressed). During tests the `pytest` target will verify responses, however the JSON-RPC methods are _not_ generated (yet?). We do generate various client stubs for languages, using the `msggen`][msggen] tool. More on the generated stubs and utilities below.
+
+## Man pages
+
+The manpages are partially generated from the JSON schemas using the [`fromschema`](https://github.com/ElementsProject/lightning/blob/master/tools/fromschema.py) tool. It reads the request schema and fills in the manpage between two markers:
+
+```markdown
+[comment]: # (GENERATE-FROM-SCHEMA-START)
+...
+[comment]: # (GENERATE-FROM-SCHEMA-END)
+```
+
+
+
+> 📘 
+> 
+> Some of this functionality overlaps with [`msggen`](https://github.com/ElementsProject/lightning/tree/master/contrib/msggen) (parsing the Schemas) and [blockreplace.py](https://github.com/ElementsProject/lightning/blob/master/devtools/blockreplace.py) (filling in the template). It is likely that this will eventually be merged.
+
+## `msggen`
+
+`msggen` is used to generate JSON-RPC client stubs, and converters between in-memory formats and the JSON format. In addition, by chaining some of these we can expose a [grpc](https://grpc.io/) interface that matches the JSON-RPC interface. This conversion chain is implemented in the [grpc-plugin](https://github.com/ElementsProject/lightning/tree/master/plugins/grpc-plugin).
+
+[block:image]
+{
+  "images": [
+    {
+      "image": [
+        "https://files.readme.io/8777cc4-image.png",
+        null,
+        null
+      ],
+      "align": "center",
+      "caption": "Artifacts generated from the JSON Schemas using `msggen`"
+    }
+  ]
+}
+[/block]
+
+
+
+
+
+### `cln-rpc`
+
+We use `msggen` to generate the Rust bindings crate [`cln-rpc`](https://github.com/ElementsProject/lightning/tree/master/cln-rpc). These bindings contain the stubs for the JSON-RPC methods, as well as types for the request and response structs. The [generator code](https://github.com/ElementsProject/lightning/blob/master/contrib/msggen/msggen/gen/rust.py) maps each abstract JSON-RPC type to a Rust type, minimizing size (e.g., binary data is hex-decoded).
+
+The calling pattern follows the `call(req_obj) -> resp_obj` format, and the individual arguments are not expanded. For more ergonomic handling of generic requests and responses we also define the `Request` and `Response` enumerations, so you can hand them to a generic function without having to resort to dynamic dispatch.
+
+The remainder of the crate implements an async/await JSON-RPC client, that can deal with the Unix Domain Socket [transport](ref:lightningd-rpc) used by CLN.
+
+### `cln-grpc`
+
+The `cln-grpc` crate is mostly used to provide the primitives to build the `grpc-plugin`. As mentioned above, the grpc functionality relies on a chain of generated parts:
+
+- First `msggen` is used to generate the [protobuf file](https://github.com/ElementsProject/lightning/blob/master/cln-grpc/proto/node.proto), containing the service definition with the method stubs, and the types referenced by those stubs.
+- Next it generates the `convert.rs` file which is used to convert the structs for in-memory representation from `cln-rpc` into the corresponding protobuf structs.
+- Finally `msggen` generates the `server.rs` file which can be bound to a grpc endpoint listening for incoming grpc requests, and it will convert the request and forward it to the JSON-RPC. Upon receiving the response it gets converted back into a grpc response and sent back.
+
+![](https://files.readme.io/53b4645-image.png)