Skip to content

Commit

Permalink
Untangle cluster & wallet code.
Browse files Browse the repository at this point in the history
This is a requirement for wallet decoupling. The 'edge' node has
historically been treated like 'wallet nodes'; Yet, that's really
not the case. Edge nodes are in reality the trusted node the wallet
should connect to.
So, there's no reason why the cluster would actually be coupled to
the wallet. In order to accomodate existing code however (because
the wallet isn't yet fully decoupled), I had to separate the generation
of the cluster environment from the booting of the cluster. Thus, to
mimic the existing behavior, we first have to:

- Generate an environment for a full demo cluster (4 cores, 1 relay, 1 edge)
- Start a diminished cluster (4 cores, 1 relay and NO edge)
- Start a wallet backend as an edge, using the generate environment.

This works quite fine for now, and will be easy to modify later once the
wallet backend is truly decoupled.
  • Loading branch information
KtorZ committed Dec 5, 2018
1 parent ac9b628 commit acb294e
Show file tree
Hide file tree
Showing 10 changed files with 296 additions and 172 deletions.
19 changes: 9 additions & 10 deletions cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Cluster is starting (4 core(s), 1 relay(s), 1 edge(s))...
...relay has no health-check API.
......system start: 1539179287
......address: 127.0.0.1:3100
...wallet OK!
...edge OK!
......system start: 1539179287
......api address: 127.0.0.1:8090
......doc address: 127.0.0.1:8190
Expand All @@ -39,7 +39,7 @@ Cluster is (probably) ready!

## Configuring Nodes

_Almost anything_ from the normal CLI arguments of a node or a wallet node can be
_Almost anything_ from the normal CLI arguments of a node can be
configured via an ENV variable using an `UPPER_SNAKE_CASE` naming, correctly
prefixed with `DEMO_` with a few gotchas:

Expand All @@ -53,7 +53,6 @@ prefixed with `DEMO_` with a few gotchas:
- `--tlscert`
- `--tlskey`
- `--topology`
- `--wallet-db-path`
- `--keyfile`
Those variables actually corresponds to artifacts or location handled by the
cluster library. Messing up with one of those can break the whole cluster.
Expand All @@ -76,8 +75,8 @@ prefixed with `DEMO_` with a few gotchas:

This is the case for:
- `--listen`
- `--wallet-address`
- `--wallet-doc-address`
- `--node-api-address`
- `--node-doc-address`

For instance, one can disable TLS client authentication doing:

Expand Down Expand Up @@ -112,17 +111,17 @@ cardano-sl-cluster-demo
Spawn a demo cluster of nodes running cardano-sl, ready-to-use
Usage:
cardano-sl-cluster-demo [--no-genesis-wallets] [options]
cardano-sl-cluster-demo [options]
cardano-sl-cluster-demo --help
Options:
--cores=INT Number of core nodes to start [default: 4]
--relays=INT Number of relay nodes to start [default: 1]
--edges=INT Number of edge nodes (wallet) to start [default: 1]
--edges=INT Number of edge nodes to start [default: 1]
```

So, the components of the cluster may be tweaked by providing arguments to the CLI.
For instance, one could switch off the wallet node by doing:
For instance, one could switch off the edge node by doing:

```
$> stack exec -- cardano-sl-cluster-demo --edges 0
Expand All @@ -136,10 +135,10 @@ common parent thread, they all eventually end up sharing the same logging state.

This result in a non-friendly behavior where, every logging handler get replaced by the one
from the next node started by the cluster. So once started, every log entry gets logged _as-if_
they were logged by the last node started (with default parameters, the wallet node).
they were logged by the last node started (with default parameters, the edge node).

This is why you won't find any log (apart from a couple of lines on start-up) inside log files,
and duplicated lines in one of them (likely `wallet.log.pub`); fixing this is non-trivial.
and duplicated lines in one of them (likely `edge.log.pub`); fixing this is non-trivial.


<p align="center">
Expand Down
41 changes: 11 additions & 30 deletions cluster/demo/Main.hs → cluster/app/demo/Main.hs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{-| Demo cluster of wallet nodes. See cluster/README.md -}
{-| Demo cluster nodes. See cluster/README.md -}

{-# LANGUAGE QuasiQuotes #-}

Expand All @@ -18,11 +18,10 @@ import System.Console.Docopt (Arguments, Docopt, docopt,
import System.IO (BufferMode (..), hSetBuffering, stdout)

import Cardano.Cluster (MaxWaitingTime (..), NodeName (..),
NodeType (..), RunningNode (..), startCluster,
waitForNode)
NodeType (..), RunningNode (..), mkNamedNodes,
startCluster, waitForNode)
import Cardano.Cluster.Util (unsafeIntFromString)
import Cardano.Wallet.API.V1.Types (SyncPercentage, WalletImport (..))
import Cardano.Wallet.Client (WalletClient (..))
import Pos.Node.API (SyncPercentage)


-- | Command-Line Interface specification. See http://docopt.org/
Expand All @@ -33,13 +32,13 @@ cardano-sl-cluster-demo
Spawn a demo cluster of nodes running cardano-sl, ready-to-use

Usage:
cardano-sl-cluster-demo [--no-genesis-wallets] [options]
cardano-sl-cluster-demo [options]
cardano-sl-cluster-demo --help

Options:
--cores=INT Number of core nodes to start [default: 4]
--relays=INT Number of relay nodes to start [default: 1]
--edges=INT Number of edge nodes (wallet) to start [default: 1]
--cores=INT Number of core nodes to start [default: 4]
--relays=INT Number of relay nodes to start [default: 1]
--edges=INT Number of edge nodes to start [default: 1]
|]


Expand Down Expand Up @@ -87,18 +86,13 @@ main = void $ do
<> "\n......address: " <> toText (env ! "LISTEN")
return handle

RunningWalletNode (NodeName nodeId) env client keys handle -> do
RunningEdgeNode (NodeName nodeId) env client handle -> do
putText "..." >> waitForNode client (MaxWaitingTime 90) printProgress

unless (args `isPresent` (longOption "no-genesis-wallets")) $ do
putTextFromStart "...Importing genesis wallets"
forM_ keys (importWallet client . WalletImport Nothing)

putTextFromStart $ "..." <> nodeId <> " OK!"
putTextLn
$ "\n......system start: " <> toText (env ! "SYSTEM_START")
<> "\n......api address: " <> toText (env ! "WALLET_ADDRESS")
<> "\n......doc address: " <> toText (env ! "WALLET_DOC_ADDRESS")
<> "\n......api address: " <> toText (env ! "NODE_API_ADDRESS")
<> "\n......doc address: " <> toText (env ! "NODE_DOC_ADDRESS")
return handle
putTextLn "Cluster is (probably) ready!"

Expand All @@ -109,19 +103,6 @@ main = void $ do
getArgInt args =
unsafeIntFromString . fromJust . getArg args . longOption

-- | Create a list of named nodes of the given type
mkNamedNodes :: NodeType -> Int -> [(NodeName, NodeType)]
mkNamedNodes NodeCore 1 = [("core", NodeCore)]
mkNamedNodes NodeRelay 1 = [("relay", NodeRelay)]
mkNamedNodes NodeEdge 1 = [("wallet", NodeEdge)]
mkNamedNodes typ n = zip (mkIndexedName typ <$> iterate (+1) 0) (replicate n typ)

-- | Create a @NodeName@ from the given @NodeType@ and index
mkIndexedName :: NodeType -> Int -> NodeName
mkIndexedName NodeCore n = NodeName ("core" <> show n)
mkIndexedName NodeRelay n = NodeName ("relay" <> show n)
mkIndexedName NodeEdge n = NodeName ("wallet" <> show n)

putTextFromStart :: Text -> IO ()
putTextFromStart txt = do
hSetCursorColumn stdout 0 >> clearFromCursorToLineEnd
Expand Down
88 changes: 88 additions & 0 deletions cluster/app/prepare-environment/Main.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
{-# LANGUAGE QuasiQuotes #-}

module Main where

import Universum hiding (init, keys)

import Control.Lens (at)
import qualified Data.Map.Strict as Map
import Data.Maybe (fromJust)
import Formatting (build, sformat, (%))
import System.Console.Docopt (Arguments, Docopt, argument, docopt,
exitWithUsage, getArg, isPresent, longOption,
parseArgsOrExit)
import System.Environment (getEnvironment)

import Cardano.Cluster (NodeType (..), mkNamedNodes)
import Cardano.Cluster.Environment (Artifact, initializeArtifact,
prepareEnvironment, withStateDirectory)
import Cardano.Cluster.Util (stripFilterPrefix, unsafeIntFromString)


-- | Command-Line Interface specification. See http://docopt.org/
cli :: Docopt
cli = [docopt|
cardano-sl-prepare-environment

Generate a default environment for a given cluster configuration

Usage:
cardano-sl-prepare-environment <prefix> [options]
cardano-sl-prepare-environment --help

Options:
--cores=INT Number of core nodes to start [default: 4]
--relays=INT Number of relay nodes to start [default: 1]
--edges=INT Number of edge nodes to start [default: 1]
|]


main :: IO ()
main = do
args <- parseArgsOrExit cli =<< getArgs
when (args `isPresent` (longOption "help")) $ exitWithUsage cli

let nCores = getOptInt args "cores"
let nRelays = getOptInt args "relays"
let nEdges = getOptInt args "edges"
let prefix = getArgString args "prefix"
let nodes = mconcat
[ mkNamedNodes NodeCore nCores
, mkNamedNodes NodeRelay nRelays
, mkNamedNodes NodeEdge nEdges
]

putTextLn $ sformat
("Generating environment for ("%build%" core(s), "%build%" relay(s), "%build%" edge(s))...")
nCores nRelays nEdges

env0 <- (Map.fromList . stripFilterPrefix prefix) <$> getEnvironment
withStateDirectory (env0 ^. at "STATE_DIR") $ \stateDir -> do
forM_ nodes $ \node@(_, nodeType) -> do
let (artifacts, _) = prepareEnvironment node nodes stateDir env0
let (genesis, topology, logger, tls) = artifacts

case nodeType of
NodeCore -> do
void (init genesis >> init topology >> init logger)

NodeRelay -> do
void (init topology >> init logger)

NodeEdge -> do
void (init topology >> init logger >> init tls)
putTextLn $ sformat
("Environment generated in: "%build)
stateDir
where
init :: Artifact a b -> IO b
init = initializeArtifact

-- | Args are defaulted to something, so we know they exist
getOptInt :: Arguments -> String -> Int
getOptInt args =
unsafeIntFromString . fromJust . getArg args . longOption

getArgString :: Arguments -> String -> String
getArgString args =
fromJust . getArg args . argument
33 changes: 27 additions & 6 deletions cluster/cardano-sl-cluster.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,9 @@ test-suite cardano-sl-cluster-test
Cardano.Cluster.Environment.Spec

build-depends: base

, cardano-sl-cluster
, cardano-sl-core
, cardano-sl-infra

, async
, containers
, lens
Expand All @@ -107,17 +105,40 @@ executable cardano-sl-cluster-demo
TypeApplications
ScopedTypeVariables

hs-source-dirs: demo
hs-source-dirs: app/demo
main-is: Main.hs

build-depends: base

, cardano-sl
, cardano-sl-cluster
, cardano-wallet

, cardano-sl-node
, ansi-terminal
, async
, containers
, docopt
, formatting
, lens
, universum


executable cardano-sl-cluster-prepare-environment
ghc-options: -threaded -O2 -rtsopts
default-language: Haskell2010
default-extensions: DeriveGeneric
LambdaCase
NoImplicitPrelude
OverloadedStrings
TupleSections
TypeApplications
ScopedTypeVariables

hs-source-dirs: app/prepare-environment
main-is: Main.hs

build-depends: base
, cardano-sl-cluster
, containers
, docopt
, formatting
, lens
, universum
Loading

0 comments on commit acb294e

Please sign in to comment.