Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a way for entries generated by tests to be associated with their related function declaration #3509

Merged
merged 12 commits into from
Dec 7, 2022

Conversation

kayagokalp
Copy link
Member

@kayagokalp kayagokalp commented Dec 5, 2022

closes #3508.

To continue with improving in-language testing (like the linked issues), we need to have a way of getting the original function declaration for the test that we are running from forc-test. Currently forc-test only works on FinalizedEntrys generated by the tests and we cannot get the function declaration from the entry point. Here I am introducing an optional field to the FinalizedEntry for tests. If the FinalizedEntry is generated from a test function, corresponding FinalizedEntry's declaration id is populated and can be retrieved. To pass this declaration id information I needed to preserve it in the IR generation step to do that I added a metadata for it and populate it for the test function declarations. So for test function declarations we are inserting their declaration index to the metadata. I did not insert the DeclarationId since it does not implement Hash, and we are already inserting the span for the function declaration. Since DeclarationId is basically the index + span, storing only the index was enough. After the IR generation before creating an entry point I construct the DeclarationId from index and span (which comes from the metadata) and pass that to FinalizedEntry. This way in the IR generation step we are not losing the declaration index + span (and thus, the declaration id) and can retrieve the test function's declaration later on.

unblocks #3490.
unblocks #3266.

@kayagokalp kayagokalp added compiler: ir IRgen and sway-ir including optimization passes compiler: frontend Everything to do with type checking, control flow analysis, and everything between parsing and IRgen forc-test Everything related to the `forc-test` lib and `forc test` command. labels Dec 5, 2022
@kayagokalp kayagokalp self-assigned this Dec 5, 2022
Copy link
Member Author

@kayagokalp kayagokalp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up collecting declaration ids only for test functions, so I added an optional test decl id field to FinalizedEntry. With this PR, I can retrieve DeclarationId for a specific test function from forc-test which makes it possible for later PRs to get the TyFunctionDeclaration from the declaration engine.

@kayagokalp kayagokalp marked this pull request as ready for review December 5, 2022 13:22
@kayagokalp kayagokalp requested review from mitchmindtree and a team December 5, 2022 13:22
sway-core/src/metadata.rs Outdated Show resolved Hide resolved
sway-core/src/metadata.rs Outdated Show resolved Hide resolved
@kayagokalp
Copy link
Member Author

kayagokalp commented Dec 5, 2022

Looks like the metadata addition needs to be somehow reflected in the ir generation test checks. I am trying to understand the structure there right now. Removed the weird logic that I added for optional decl index, and it also fixed CI as I don't add unnecessary/garbage metadata now. 🥳

@mohammadfawaz
Copy link
Contributor

Two things:

@kayagokalp
Copy link
Member Author

kayagokalp commented Dec 5, 2022

cc @mohammadfawaz Of course, moved to PR description.

I am looking for how to (and where to) add a test for this 👍

@mohammadfawaz
Copy link
Contributor

cc @mohammadfawaz Of course, to continue with improving in-language testing (like the linked issues), we need to have a way of getting the original function declaration for the test that we are running from forc-test. Currently forc-test only works on FinalizedEntrys generated by the tests and we cannot get the function declaration from the entry point. Here I am introducing an optional field to the FinalizedEntry for tests. If the FinalizedEntry is generated from a test function, corresponding FinalizedEntry's declaration id is populated and can be retrieved. To pass this declaration id information I needed to preserve it in the IR generation step to do that I added a metadata for it and populate it for the test function declarations. So for test function declarations we are inserting their declaration index to the metadata. I did not insert the DeclarationId since it does not implement Hash, and we are already inserting the span for the function declaration. Since DeclarationId is basically the index + span, storing only the index is enough. After the IR generation before creating an entry point I construct the DeclarationId from index and span (which comes from the metadata) and pass that to FinalizedEntry. This way in the IR generation step we are not losing the declaration index + span (and thus, the declaration id) and can retrieve the test function's declaration later on.

I am looking for how to (and where to) add a test for this 👍

Excellent, thank you! Just wanted to see this description in the Description field for this PR for future reference.

Copy link
Contributor

@otrho otrho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good overall. 👍

test/src/ir_generation/mod.rs Show resolved Hide resolved
test/src/ir_generation/tests/test_metadata.sw Outdated Show resolved Hide resolved
@otrho
Copy link
Contributor

otrho commented Dec 6, 2022

It should just work, since the metadata are supposed to be opaque to the IR in general, but having a file with test functions and the decl_id metadata tag in here would be good.

@kayagokalp
Copy link
Member Author

Thanks for the comments @otrho! I added a serialize test and made the ir_generation test more explicit (added checks related to the decl_id). I think this should be good go

@kayagokalp kayagokalp requested a review from otrho December 6, 2022 09:15
@kayagokalp kayagokalp enabled auto-merge (squash) December 6, 2022 09:17
@otrho otrho requested a review from a team December 6, 2022 22:42
Copy link
Contributor

@mohammadfawaz mohammadfawaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks! Thanks for adding the test.. it just makes it clear what you're adding in this change and also prevents regression in case someone makes a change that causes the metadata to drop.

It seems that this metadata is attached to the top level test function so there is no risk of it being lost since those functions are never changed (never inlined for example). This is good!

@otrho it would be nice to have a place where all the metadata types are explained along with their legal use cases, like LLVM: https://llvm.org/docs/LangRef.html#dicompileunit

Copy link
Contributor

@mitchmindtree mitchmindtree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌 :shipit:

@kayagokalp kayagokalp merged commit aa95047 into master Dec 7, 2022
@kayagokalp kayagokalp deleted the kayagokalp/3508 branch December 7, 2022 01:04
kayagokalp added a commit that referenced this pull request Dec 8, 2022
…ld revert (#3490)

## Summary

This PR adds `#[test(should_revert)]` tests to forc-test, enabling users
to write reverting cases in their unit tests.
Example:

```rust
script;    

fn main() {}  
fn my_func() {}  
  
#[test(should_revert)]  
fn my_test_func() {  
  my_func();  
  revert(0)  
}
```

## Implementation Details

Since #3509 is merged, this PR only adds the required section to
`forc-test` so that it can detect if the given entry point's
corresponding function declarations has `#[test(should_revert)]`
attribute. If that is the case the passing condition is selected as
`TestPassCondition::ShouldRevert` otherwise it defaults to
`TestPassCondition::ShouldNotRevert`.

I also added a safety check for erroneous declarations such as
`#[test(foo)]`. Since we just have `should revert` tests right now, I am
checking if there is an argument provided with the test attirbute. If
there is none, test is considered to be
`TestPassCondition::ShouldNotRevert`, tl-dr; default is non reverting
tests. If there is an argument and it is `should_revert`, test is
considered to be `TestPassCondition::ShouldRevert` and finally if there
is an argument but it is not `should_revert` an error is produced.

A simple should revert test with `revert(0)` added to unit tests as
well.
sdankel pushed a commit that referenced this pull request Dec 14, 2022
…their related function declaration (#3509)

closes #3508.

To continue with improving in-language testing (like the linked issues),
we need to have a way of getting the original function declaration for
the test that we are running from `forc-test`. Currently `forc-test`
only works on `FinalizedEntry`s generated by the tests and we cannot get
the function declaration from the entry point. Here I am introducing an
optional field to the `FinalizedEntry` for tests. If the
`FinalizedEntry` is generated from a test function, corresponding
`FinalizedEntry`'s declaration id is populated and can be retrieved. To
pass this declaration id information I needed to preserve it in the IR
generation step to do that I added a metadata for it and populate it for
the test function declarations. So for test function declarations we are
inserting their declaration index to the metadata. I did not insert the
`DeclarationId` since it does not implement `Hash`, and we are already
inserting the span for the function declaration. Since `DeclarationId`
is basically the index + span, storing only the index was enough. After
the IR generation before creating an entry point I construct the
`DeclarationId` from index and span (which comes from the metadata) and
pass that to `FinalizedEntry`. This way in the IR generation step we are
not losing the declaration index + span (and thus, the declaration id)
and can retrieve the test function's declaration later on.

unblocks #3490.
unblocks #3266.
sdankel pushed a commit that referenced this pull request Dec 14, 2022
…ld revert (#3490)

## Summary

This PR adds `#[test(should_revert)]` tests to forc-test, enabling users
to write reverting cases in their unit tests.
Example:

```rust
script;    

fn main() {}  
fn my_func() {}  
  
#[test(should_revert)]  
fn my_test_func() {  
  my_func();  
  revert(0)  
}
```

## Implementation Details

Since #3509 is merged, this PR only adds the required section to
`forc-test` so that it can detect if the given entry point's
corresponding function declarations has `#[test(should_revert)]`
attribute. If that is the case the passing condition is selected as
`TestPassCondition::ShouldRevert` otherwise it defaults to
`TestPassCondition::ShouldNotRevert`.

I also added a safety check for erroneous declarations such as
`#[test(foo)]`. Since we just have `should revert` tests right now, I am
checking if there is an argument provided with the test attirbute. If
there is none, test is considered to be
`TestPassCondition::ShouldNotRevert`, tl-dr; default is non reverting
tests. If there is an argument and it is `should_revert`, test is
considered to be `TestPassCondition::ShouldRevert` and finally if there
is an argument but it is not `should_revert` an error is produced.

A simple should revert test with `revert(0)` added to unit tests as
well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler: frontend Everything to do with type checking, control flow analysis, and everything between parsing and IRgen compiler: ir IRgen and sway-ir including optimization passes forc-test Everything related to the `forc-test` lib and `forc test` command.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce a way for entries generated by tests to be associated with their related function declaration
4 participants