[epoch] Do not start Narwhal if epoch mismatch (MystenLabs#7452)

We currently read narwhal committee from the system object during
startup.
This can be wrong if we are in a crash recovery state: we have executed
the last transaction of the epoch (and hence on-chain system state
object is already at next epoch), but we haven't reconfigured Sui yet.
This could lead to Narwhal and Sui to be at different epoch, which is
fatal and could lead to all kinds of issues.
A proper fix is likely to put Narwhal committee in the epoch store, just
like the Sui committee, but that requires some proper refactoring and
will take some time.
This PR is a quick fix: we simply don't start Narwhal if they don't
match, and we rely on the reconfig process to start it properly.

crates/sui-node/src/lib.rs

@@ -52,7 +52,7 @@ use sui_types::messages::QuorumDriverResponse;
 use tokio::sync::{watch, Mutex};
 use tokio::task::JoinHandle;
 use tower::ServiceBuilder;
-use tracing::info;
+use tracing::{info, warn};
 use typed_store::DBMetrics;
 pub mod admin;
 mod handle;
@@ -499,7 +499,7 @@ impl SuiNode {
         );
 
         let consensus_handler = Arc::new(ConsensusHandler::new(
-            epoch_store,
+            epoch_store.clone(),
             checkpoint_service.clone(),
             state.transaction_manager().clone(),
             state.db(),
@@ -521,13 +521,19 @@ impl SuiNode {
             .address;
         let worker_cache = system_state.get_current_epoch_narwhal_worker_cache(transactions_addr);
 
-        narwhal_manager
-            .start(
-                committee.clone(),
-                SharedWorkerCache::from(worker_cache),
-                consensus_handler,
-            )
-            .await;
+        if committee.epoch == epoch_store.epoch() {
+            narwhal_manager
+                .start(
+                    committee.clone(),
+                    SharedWorkerCache::from(worker_cache),
+                    consensus_handler,
+                )
+                .await;
+        } else {
+            warn!(
+                "Current Sui epoch doesn't match the system state epoch. Not starting Narwhal yet"
+            );
+        }
 
         Ok(ValidatorComponents {
             validator_server_handle,