Skip to content

Commit

Permalink
Add module AST cache. (FuelLabs#4733)
Browse files Browse the repository at this point in the history
## Description

This adds a module AST cache which speeds up the parsing and type
checking stages for modules that have not changed.
To make this work, when first parsing, we save the tree of dependencies
and hashes for each module.
Then when compiling again, we check the cache and re-use if possible.

To make this work for LSP, I've had to revert the change that was done
earlier to re-create the engines for each compilation step.
This may lead to memory bloating, which was the reason that change was
done in the first place IIRC, so I'll be investigating a garbage
collection step that LSP can run after each compilation.

On the LSP `did_change` test, that compiles a single file, makes a
change, and re-parses again, this drops the time from 6s (3s + 3s for
the second parse) to 3s, as expected, since the core and standard
library are re-used instead of being parsed and analyzed again.

## Checklist

- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
  • Loading branch information
tritao authored Aug 25, 2023
1 parent 15e0ec4 commit 255350e
Show file tree
Hide file tree
Showing 11 changed files with 241 additions and 50 deletions.
3 changes: 0 additions & 3 deletions forc-pkg/src/pkg.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1798,7 +1798,6 @@ pub fn compile(
namespace,
Some(&sway_build_config),
&pkg.name,
&mut metrics,
),
Some(sway_build_config.clone()),
metrics
Expand Down Expand Up @@ -2650,7 +2649,6 @@ pub fn check(
)?
.include_tests(include_tests);

let mut metrics = PerformanceData::default();
let input = manifest.entry_string()?;
let handler = Handler::default();
let programs_res = sway_core::compile_to_ast(
Expand All @@ -2660,7 +2658,6 @@ pub fn check(
dep_namespace,
Some(&build_config),
&pkg.name,
&mut metrics,
);

let programs = match programs_res.as_ref() {
Expand Down
3 changes: 0 additions & 3 deletions sway-core/src/ir_generation/const_eval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1027,16 +1027,13 @@ mod tests {
let mut md_mgr = MetadataManager::default();
let core_lib = namespace::Module::default();

let mut performance_data = sway_utils::PerformanceData::default();

let r = crate::compile_to_ast(
&handler,
&engines,
std::sync::Arc::from(format!("library; {prefix} fn f() -> u64 {{ {expr}; 0 }}")),
core_lib,
None,
"test",
&mut performance_data,
);

let (errors, _warnings) = handler.consume();
Expand Down
4 changes: 4 additions & 0 deletions sway-core/src/language/parsed/module.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ use crate::{
use super::ParseTree;
use sway_types::Span;

pub type ModuleHash = u64;

/// A module and its submodules in the form of a tree.
#[derive(Debug, Clone)]
pub struct ParseModule {
Expand All @@ -18,6 +20,8 @@ pub struct ParseModule {
pub module_kind_span: Span,
/// an empty span at the beginning of the file containing the module
pub span: Span,
/// an hash used for caching the module
pub hash: ModuleHash,
}

/// A library module that was declared as a `mod` of another module.
Expand Down
8 changes: 7 additions & 1 deletion sway-core/src/language/programs.rs
Original file line number Diff line number Diff line change
@@ -1,24 +1,30 @@
use sway_error::handler::ErrorEmitted;
use sway_utils::PerformanceData;

use super::{lexed::LexedProgram, parsed::ParseProgram, ty::TyProgram};

/// Contains the lexed, parsed, and typed compilation stages of a program.
/// Contains the lexed, parsed, typed compilation stages of a program, as well
/// as compilation metrics.
#[derive(Clone, Debug)]
pub struct Programs {
pub lexed: LexedProgram,
pub parsed: ParseProgram,
pub typed: Result<TyProgram, ErrorEmitted>,
pub metrics: PerformanceData,
}

impl Programs {
pub fn new(
lexed: LexedProgram,
parsed: ParseProgram,
typed: Result<TyProgram, ErrorEmitted>,
metrics: PerformanceData,
) -> Programs {
Programs {
lexed,
parsed,
typed,
metrics,
}
}
}
Loading

0 comments on commit 255350e

Please sign in to comment.