Skip to content

Commit

Permalink
Merge pull request tidymodels#493 from tidymodels/RC-1.3.0
Browse files Browse the repository at this point in the history
Release Candidate 1.3.0
  • Loading branch information
EmilHvitfeldt authored Jan 20, 2024
2 parents 80f4fbf + 0c105d5 commit 66dce0f
Show file tree
Hide file tree
Showing 13 changed files with 114 additions and 86 deletions.
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Type: Package
Package: yardstick
Title: Tidy Characterizations of Model Performance
Version: 1.2.0.9003
Version: 1.3.0.9000
Authors@R: c(
person("Max", "Kuhn", , "[email protected]", role = "aut"),
person("Davis", "Vaughan", , "[email protected]", role = "aut"),
Expand All @@ -17,7 +17,7 @@ URL: https://github.com/tidymodels/yardstick,
https://yardstick.tidymodels.org
BugReports: https://github.com/tidymodels/yardstick/issues
Depends:
R (>= 3.4.0)
R (>= 3.6.0)
Imports:
cli,
dplyr (>= 1.1.0),
Expand Down
14 changes: 12 additions & 2 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# yardstick (development version)

# yardstick 1.3.0

## New Metrics

* The Brier score for survival data was added with `brier_survival()`.

* The Integrated Brier score for survival data was added with `brier_survival_integrated()`.
Expand All @@ -12,10 +16,12 @@ calculated with `roc_curve_survival()`.
* Time-Dependent ROC AUC estimation for right-censored data can now be
calculated with `roc_auc_survival()`.

* `metric_set()` can now be used with a combination of dynamic and static survival metrics.
## Improvements

* `demographic_parity()`, `equalized_odds()`, and `equal_opportunity()` are new metrics for measuring model fairness. Each is implemented with the `new_groupwise_metric()` constructor, a general interface for defining group-aware metrics that allows for quickly and flexibly defining fairness metrics with the problem context in mind.

* `metric_set()` can now be used with a combination of dynamic and static survival metrics.

* Added a print method for metrics and metric sets (#435).

* All warnings and errors have been updated to use the cli package for increased clarity and consistency. (#456, #457, #458)
Expand All @@ -24,8 +30,12 @@ calculated with `roc_auc_survival()`.

* Clarifying documentation about how `event_level` always default to `"first`. (#432)

## Bug Fixes

* Metrics now throw more informative error if `estimate` argument is wrongly used. (#443)

## Breaking Changes

* Curve metrics now throw an informative error instead of returning `NA` when missing values are found and `na_rm = FALSE`. (#344)

# yardstick 1.2.0
Expand Down Expand Up @@ -193,7 +203,7 @@ calculated with `roc_auc_survival()`.
the new explicit argument, `event_level`. All metric functions that previously
supported changing the "event" level have gained this new argument.
The global option was a historical design decision that can be classified as
a case of a [hidden argument](https://design.tidyverse.org/args-hidden.html#args-hidden).
a case of a hidden argument.
Existing code that relied on this global option will continue to work in this
version of yardstick, however you will now get a once-per-session warning
that requests that you update to instead use the explicit `event_level`
Expand Down
12 changes: 6 additions & 6 deletions R/surv-brier_survival.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@
#'
#' @param ... The column identifier for the survival probabilities this
#' should be a list column of data.frames corresponding to the output given when
#' predicting with {censored} model. This should be an unquoted column name
#' although this argument is passed by expression and supports
#' [quasiquotation][rlang::quasiquotation] (you can unquote column names). For
#' `_vec()` functions, the dots are not used.
#' predicting with [censored](https://censored.tidymodels.org/) model. This
#' should be an unquoted column name although this argument is passed by
#' expression and supports [quasiquotation][rlang::quasiquotation] (you can
#' unquote column names). For `_vec()` functions, the dots are not used.
#'
#' @param estimate A list column of data.frames corresponding to the output
#' given when predicting with {censored} model. See the details for more
#' information regarding format.
#' given when predicting with [censored](https://censored.tidymodels.org/)
#' model. See the details for more information regarding format.
#'
#' @details
#'
Expand Down
2 changes: 1 addition & 1 deletion README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ options(width = 100, digits = 3)
# yardstick <img src="man/figures/logo.png" align="right"/>

<!-- badges: start -->
[![Coverage Status](https://img.shields.io/codecov/c/github/tidymodels/yardstick/main.svg)](https://codecov.io/github/tidymodels/yardstick?branch=main)
[![Coverage Status](https://img.shields.io/codecov/c/github/tidymodels/yardstick/main.svg)](https://app.codecov.io/github/tidymodels/yardstick?branch=main)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/yardstick)](https://CRAN.R-project.org/package=yardstick)
[![Downloads](https://cranlogs.r-pkg.org/badges/yardstick)](https://CRAN.R-project.org/package=yardstick)
[![R-CMD-check](https://github.com/tidymodels/yardstick/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/tidymodels/yardstick/actions/workflows/R-CMD-check.yaml)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<!-- badges: start -->

[![Coverage
Status](https://img.shields.io/codecov/c/github/tidymodels/yardstick/main.svg)](https://codecov.io/github/tidymodels/yardstick?branch=main)
Status](https://img.shields.io/codecov/c/github/tidymodels/yardstick/main.svg)](https://app.codecov.io/github/tidymodels/yardstick?branch=main)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/yardstick)](https://CRAN.R-project.org/package=yardstick)
[![Downloads](https://cranlogs.r-pkg.org/badges/yardstick)](https://CRAN.R-project.org/package=yardstick)
[![R-CMD-check](https://github.com/tidymodels/yardstick/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/tidymodels/yardstick/actions/workflows/R-CMD-check.yaml)
Expand Down
9 changes: 9 additions & 0 deletions cran-comments.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
## Submission 1.3.0

Final additions to make survival metrics possible.

We checked 36 reverse dependencies, comparing R CMD check results across CRAN and dev versions of this package.

* We saw 0 new problems
* We failed to check 0 packages

## Submission 1.2.0

Major refactor done to allow new types of metrics to be used.
Expand Down
12 changes: 6 additions & 6 deletions man/brier_survival.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions man/brier_survival_integrated.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions man/roc_auc_survival.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions man/roc_curve_survival.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

61 changes: 33 additions & 28 deletions revdep/README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,46 @@
# Platform

|field |value |
|:--------|:------------------------------------------------|
|version |R version 4.2.1 (2022-06-23) |
|os |macOS Monterey 12.6 |
|system |aarch64, darwin20 |
|ui |RStudio |
|language |(EN) |
|collate |en_US.UTF-8 |
|ctype |en_US.UTF-8 |
|tz |America/Los_Angeles |
|date |2023-04-19 |
|rstudio |2023.05.0-daily+206 Mountain Hydrangea (desktop) |
|pandoc |2.17.1.1 @ /opt/homebrew/bin/pandoc |
|field |value |
|:--------|:------------------------------------------------------------|
|version |R version 4.3.1 (2023-06-16) |
|os |macOS Sonoma 14.2.1 |
|system |aarch64, darwin20 |
|ui |X11 |
|language |(EN) |
|collate |en_US.UTF-8 |
|ctype |en_US.UTF-8 |
|tz |America/Los_Angeles |
|date |2024-01-19 |
|pandoc |3.1.3 @ /Users/emilhvitfeldt/miniforge3/bin/ (via rmarkdown) |

# Dependencies

|package |old |new |Δ |
|:----------|:-----|:----------|:--|
|yardstick |1.1.0 |1.1.0.9001 |* |
|cli |NA |3.6.1 |* |
|dplyr |NA |1.1.1 |* |
|fansi |NA |1.0.4 |* |
|hardhat |NA |1.3.0.9000 |* |
|lifecycle |NA |1.0.3 |* |
|pillar |NA |1.9.0 |* |
|rlang |NA |1.1.0 |* |
|tibble |NA |3.2.1 |* |
|tidyselect |NA |1.2.0 |* |
|utf8 |NA |1.2.3 |* |
|vctrs |NA |0.6.1 |* |
|yardstick |1.2.0 |1.2.0.9003 |* |
|cli |3.6.2 |3.6.2 | |
|dplyr |1.1.4 |1.1.4 | |
|fansi |1.0.6 |1.0.6 | |
|generics |0.1.3 |0.1.3 | |
|glue |1.7.0 |1.7.0 | |
|hardhat |1.3.0 |1.3.0 | |
|lifecycle |1.0.4 |1.0.4 | |
|magrittr |2.0.3 |2.0.3 | |
|pillar |1.9.0 |1.9.0 | |
|pkgconfig |2.0.3 |2.0.3 | |
|R6 |2.5.1 |2.5.1 | |
|rlang |1.1.3 |1.1.3 | |
|tibble |3.2.1 |3.2.1 | |
|tidyselect |1.2.0 |1.2.0 | |
|utf8 |1.2.4 |1.2.4 | |
|vctrs |0.6.5 |0.6.5 | |
|withr |3.0.0 |3.0.0 | |

# Revdeps

## New problems (1)

|package |version |error |warning |note |
|:-----------|:-------|:------|:-------|:----|
|[sknifedatar](problems.md#sknifedatar)|0.1.2 |__+1__ | | |
|package |version |error |warning |note |
|:---------|:-------|:------|:-------|:----|
|[modeltime](problems.md#modeltime)|1.2.8 |__+1__ | |1 |

4 changes: 2 additions & 2 deletions revdep/cran.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## revdepcheck results

We checked 32 reverse dependencies, comparing R CMD check results across CRAN and dev versions of this package.
We checked 36 reverse dependencies, comparing R CMD check results across CRAN and dev versions of this package.

* We saw 1 new problems
* We failed to check 0 packages
Expand All @@ -10,6 +10,6 @@ Issues with CRAN packages are summarised below.
### New problems
(This reports the first line of each new failure)

* sknifedatar
* modeltime
checking examples ... ERROR

48 changes: 26 additions & 22 deletions revdep/problems.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,45 @@
# sknifedatar
# modeltime

<details>

* Version: 0.1.2
* GitHub: https://github.com/rafzamb/sknifedatar
* Source code: https://github.com/cran/sknifedatar
* Date/Publication: 2021-06-01 08:00:02 UTC
* Number of recursive dependencies: 213
* Version: 1.2.8
* GitHub: https://github.com/business-science/modeltime
* Source code: https://github.com/cran/modeltime
* Date/Publication: 2023-09-02 15:10:02 UTC
* Number of recursive dependencies: 249

Run `revdep_details(, "sknifedatar")` for more info
Run `revdepcheck::revdep_details(, "modeltime")` for more info

</details>

## Newly broken

* checking examples ... ERROR
```
Running examples in ‘sknifedatar-Ex.R’ failed
Running examples in ‘modeltime-Ex.R’ failed
The error most likely occurred in:
> ### Name: multieval
> ### Title: Evaluation of multiple metrics and predictions
> ### Aliases: multieval
> ### Name: parallel_start
> ### Title: Start parallel clusters using 'parallel' package
> ### Aliases: parallel_start parallel_stop
>
> ### ** Examples
>
> set.seed(123)
...
20. │ └─tidyselect:::vars_select_eval(...)
21. │ └─tidyselect:::walk_data_tree(expr, data_mask, context_mask)
22. │ └─tidyselect:::as_indices_sel_impl(...)
23. │ └─tidyselect:::as_indices_impl(...)
24. │ └─vctrs::vec_as_subscript(x, logical = "error", call = call, arg = arg)
25. ├─dplyr::bind_rows(.)
26. │ └─rlang::list2(...)
27. ├─dplyr::mutate(., modelo = y)
28. └─rlang::cnd_signal(x)
>
> # Starts 2 clusters
> parallel_start(2)
Error in serverSocket(port = port) :
creation of server socket failed: port 11904 cannot be opened
Calls: parallel_start -> <Anonymous> -> makePSOCKcluster -> serverSocket
Execution halted
```
## In both
* checking dependencies in R code ... NOTE
```
Namespaces in Imports field not imported from:
‘tidymodels’ ‘tidyverse’
All declared Imports should be used.
```

0 comments on commit 66dce0f

Please sign in to comment.