Skip to content

Commit

Permalink
Merge pull request abess-team#113 from AnrWang/master
Browse files Browse the repository at this point in the history 
Update gallery
  • Loading branch information
@Mamba413
Mamba413 authored Jul 30, 2024
2 parents 4ca44b3 + 945944b commit 4f821a0
Show file tree
Hide file tree
Showing 8 changed files with 143 additions and 134 deletions.
44 changes: 24 additions & 20 deletions docs/source/gallery/GeneralizedLinearModels/Inverse-gaussian-regression.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"metadata": {},
"source": [
"\n",
"# Inverse Gaussian regression\n"
"## Inverse Gaussian regression\n"
]
},
{
Expand All @@ -35,7 +35,17 @@
"id": "e9f957ec",
"metadata": {},
"source": [
"## Inverse Gaussian Regression or wald regression\n",
"### Introduction\n",
"\n",
"The inverse Gaussian distribution is especially suitable for modeling data that are positively skewed and non-negative. It was first introduced by Louis Bachelier in 1900 [[1]](#refer-anchor-1). Its properties make it useful for modeling the time until an event occurs (like failure time in reliability studies) and for life data analysis.\n",
"Next are some applications of it.\n",
"- **Reliability Engineering**: The inverse Gaussian distribution is commonly used to model the lifetimes of components subject to wear and tear, such as mechanical parts and electronic devices. Its suitability for skewed data makes it a valuable tool in predicting failure times and planning maintenance schedules.\n",
"- **Medical Research**: It is used to model survival times and the time until recurrence of diseases. For instance, the time until the recurrence of a disease after treatment can be effectively modeled using the inverse Gaussian distribution.\n",
"- **Finance**: In finance, the inverse Gaussian distribution can be used to model stock returns and interest rates, which are often positively skewed. It provides a more accurate representation of the time to default for credit risk modeling compared to symmetric distributions.\n",
"- **Queueing Theory**: It is used to model the time between arrivals of customers in queueing systems. The distribution’s ability to handle skewness makes it a good fit for systems where the arrival times are not evenly distributed.\n",
"\n",
"The inverse Gaussian distribution and its regression model provide powerful tools for analyzing positively skewed continuous data. By leveraging the properties of this distribution, practitioners can gain valuable insights into the underlying processes governing their data, whether in engineering, medicine, finance, or other fields. Effective parameter estimation and model evaluation are crucial for making accurate predictions and informed decisions based on the model.\n",
"\n",
"The inverse Gaussian distribution is a continuous probability distribution with probability density function:\n",
"$$\n",
"f(y|\\mu,\\lambda) = \\sqrt{\\frac{\\lambda}{2\\pi y^3}}\\exp\\left(-\\frac{\\lambda(y-\\mu)^2}{2\\mu^2y}\\right)\n",
Expand Down Expand Up @@ -65,7 +75,9 @@
"id": "724a582d",
"metadata": {},
"source": [
"### Import necessary packages "
"### Data for inverse Gaussian regression\n",
"\n",
"We import necessary packages and set a seed."
]
},
{
Expand All @@ -81,15 +93,6 @@
"import numpy as np"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "1b93ed63",
"metadata": {},
"source": [
"### Set a seed"
]
},
{
"cell_type": "code",
"execution_count": 5,
Expand All @@ -106,7 +109,6 @@
"id": "d3b0b805",
"metadata": {},
"source": [
"### Generate the data\n",
"\n",
"Firstly, we define a data generator function to provide a way to generate suitable dataset for this task.\n",
"\n",
Expand Down Expand Up @@ -236,9 +238,9 @@
"id": "66f1826a",
"metadata": {},
"source": [
"### Define inverse gaussian regression loss\n",
"### Solve the problem\n",
"\n",
"Secondly, we define the loss function `inverse_gaussian_regression_loss` accorting to [1](#loss) that matches the data generating function `make_inverse_gaussian_regression_data`."
"Next, we define the loss function `inverse_gaussian_regression_loss` accorting to [1](#loss) that matches the data generating function `make_inverse_gaussian_regression_data`."
]
},
{
Expand Down Expand Up @@ -292,8 +294,7 @@
"id": "484bd3e9",
"metadata": {},
"source": [
"### Use skscope to solve the sparse inverse gaussian regression problem\n",
"\n",
"We use `skscope` to solve the sparse inverse gaussian regression problem.\n",
"After defining the data generation and loss function, we can call `ScopeSolver` to solve the sparse-constrained optimization problem.\n"
]
},
Expand Down Expand Up @@ -371,7 +372,6 @@
"id": "5181485b",
"metadata": {},
"source": [
"### More on the results\n",
"\n",
"We can plot the sparse signal recovering from the noisy observations to visualize the results."
]
Expand Down Expand Up @@ -424,14 +424,18 @@
"source": [
"### References\n",
"\n",
"<div id=\"refer-anchor-1\"></div>\n",
"\n",
"- [1] Bachelier, L. (1900). Théorie de la spéculation. In Annales scientifiques de l'École normale supérieure (Vol. 17, pp. 21-86).\n",
"\n",
"<div id=\"refer-1\"></div>\n",
"\n",
"- [1] Wikipedia, \"Inverse Gaussian distribution\".\n",
"- [2] Wikipedia, \"Inverse Gaussian distribution\".\n",
"https://en.wikipedia.org/wiki/Inverse_Gaussian_distribution\n",
"\n",
"<div id=\"refer-2\"></div>\n",
"\n",
"- [2] Numpy, \"np.random.wald\".\n",
"- [3] Numpy, \"np.random.wald\".\n",
"https://numpy.org/doc/stable/reference/random/generated/numpy.random.wald.html#numpy.random.wald\n"
]
}
Expand Down
91 changes: 38 additions & 53 deletions docs/source/gallery/GeneralizedLinearModels/gamma-regression.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,5 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "7f5e5d54",
"metadata": {},
"source": [
"# Gamma regression\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
Expand All @@ -27,45 +18,56 @@
"## Gamma regression"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ba8fc9",
"metadata": {},
"source": [
"We would like to use an example to show how the sparse-constrained optimization for gamma regression works in our program."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "55dcc923",
"metadata": {},
"source": [
"### Introduction\n",
"Gamma regression can be used when you have positive continuous response variables such as payments for insurance claims,\n",
"or the lifetime of a redundant system.\n",
"It is well known that the density of Gamma distribution can be represented as a function of\n",
"a mean parameter ($\\mu$) and a shape parameter ($\\alpha$), respectively,\n",
"\n",
"Gamma regression is a type of generalized linear model used when the response variable is continuous and strictly positive. It is particularly useful for modeling data such as insurance claim amounts or the lifetime of systems. Gamma regression is widely used in various fields to model positive continuous response variables. Some specific applications include:\n",
"\n",
"- **Insurance**: Estimating the amount of insurance claims based on policyholder characteristics.\n",
"- **Survival Analysis**: Modeling the time until an event occurs, such as the lifetime of a machine or system.\n",
"- **Health Economics**: Predicting healthcare costs based on patient demographics and medical history.\n",
"\n",
"The Gamma distribution, characterized by a mean parameter ($\\mu$) and a shape parameter ($\\alpha$), is defined as follows:\n",
"\n",
"$$\n",
"\\begin{align}f(y \\mid \\mu, \\alpha)=\\frac{1}{y \\Gamma(\\alpha)}\\left(\\frac{\\alpha y}{\\mu}\\right)^{\\alpha} e^{-\\alpha y / \\mu} {I}_{(0, \\infty)}(y),\\end{align}\n",
"f(y \\mid \\mu, \\alpha) = \\frac{1}{y \\Gamma(\\alpha)} \\left( \\frac{\\alpha y}{\\mu} \\right)^{\\alpha} e^{-\\alpha y / \\mu} I_{(0, \\infty)}(y),\n",
"$$\n",
"where $I(\\cdot)$ denotes the indicator function. In the Gamma regression model,\n",
"response variables are assumed to follow Gamma distributions. Specifically,\n",
"\n",
"\\begin{align}y_i \\sim Gamma(\\mu_i, \\alpha),\\end{align}\n",
"where $I(\\cdot)$ denotes the indicator function. In the context of Gamma regression, the response variable $y$ is assumed to follow a Gamma distribution:\n",
"\n",
"$$\n",
"y_i \\sim \\text{Gamma}(\\mu_i, \\alpha),\n",
"$$\n",
"\n",
"with the mean parameter $\\mu_i$ modeled as a function of the explanatory variables $\\mathbf{x}_i$:\n",
"\n",
"where $1/\\mu_i = x_i^T\\beta$.\n",
"$$\n",
"\\frac{1}{\\mu_i} = \\mathbf{x}_i^\\top \\boldsymbol{\\beta}.\n",
"$$\n",
"\n",
"With $n$ independent data of the explanatory variables $x$ and the response variable $y$, we can estimate $\\beta$ by minimizing the negative log-likelihood function under sparsity constraint:\n",
"The Gamma regression model relates the mean of the response variable to the predictors through a link function. The canonical link function for Gamma regression is the inverse link function, which ensures that the mean $\\mu_i$ is always positive. Given $n$ independent observations of the explanatory variables $\\mathbf{x}$ and the response variable $y$, the parameter vector $\\boldsymbol{\\beta}$ can be estimated by minimizing the negative log-likelihood function under a sparsity constraint:\n",
"\n",
"<a id='loss'></a>\n",
"$$\n",
"\\arg \\min _{\\beta \\in R^p} L(\\beta):=-\\frac{1}{n} \\sum_{i=1}^n\\left\\{-\\alpha \\left( y_i x_i^T \\beta - \\log \\left(x_i^T \\beta\\right)\\right) + \\alpha \\log \\alpha + \\left(\\alpha - 1\\right) \\log y - \\log \\Gamma \\left(\\alpha\\right) \\right\\}, \\text { s.t. }\\|\\beta\\|_0 \\leq s .\\tag{1}\n",
"\\arg \\min_{\\boldsymbol{\\beta} \\in \\mathbb{R}^p} L(\\boldsymbol{\\beta}) := -\\frac{1}{n} \\sum_{i=1}^n \\left\\{ -\\alpha \\left( y_i \\mathbf{x}_i^\\top \\boldsymbol{\\beta} - \\log \\left( \\mathbf{x}_i^\\top \\boldsymbol{\\beta} \\right) \\right) + \\alpha \\log \\alpha + (\\alpha - 1) \\log y_i - \\log \\Gamma(\\alpha) \\right\\},\n",
"$$\n",
"subject to the constraint $\\|\\boldsymbol{\\beta}\\|_0 \\leq s$.\n",
"\n",
"Here is example python code for solving sparse gamma regression problem:\n"
"In Gamma regression, the coefficients $\\boldsymbol{\\beta}$ have a multiplicative effect on the mean response variable. Specifically, for a unit increase in the predictor $x_j$, the mean response changes by a factor of $e^{\\beta_j}$, given the inverse link function.\n"
]
},
{
"cell_type": "markdown",
"id": "dd37dbe8",
"metadata": {},
"source": [
"### An example\n",
"\n",
"We would like to use an example to show how the sparse-constrained optimization for gamma regression works in our program."
]
},
{
Expand All @@ -74,7 +76,7 @@
"id": "2a3dae62",
"metadata": {},
"source": [
"### Import necessary packages "
"First, we import some necessary packages and set a seed."
]
},
{
Expand All @@ -90,15 +92,6 @@
"from skscope.numeric_solver import convex_solver_LBFGS"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c325811b",
"metadata": {},
"source": [
"### Set a seed"
]
},
{
"cell_type": "code",
"execution_count": 3,
Expand All @@ -115,11 +108,8 @@
"id": "36c71007",
"metadata": {},
"source": [
"### Generate the data\n",
"\n",
"Firstly, we define a data generator function to provide a way to generate suitable dataset for this task.\n",
"\n",
"The model: \n",
"Then, we define a data generator function to provide a way to generate suitable dataset for this task.\n",
"We consider the model: \n",
"* $y \\sim \\text{Gamma}(\\mu, \\alpha), \\mu = 1/(x^T \\beta + \\epsilon), \\beta \\sim U[1, 2], \\mu / \\alpha\\sim U[0.1, 100.1]$ in shape-scale definition.\n",
"\n"
]
Expand Down Expand Up @@ -183,7 +173,7 @@
"id": "867ac8fa",
"metadata": {},
"source": [
"We then use this function to generate a data set containg 500 observations and set only 5 of the 500 variables to have effect on the expectation of the response. "
"Next, we use this function to generate a data set containg 500 observations and set only 5 of the 500 variables to have effect on the expectation of the response. "
]
},
{
Expand Down Expand Up @@ -245,8 +235,6 @@
"id": "e74f586f",
"metadata": {},
"source": [
"### Define the loss function to calculate negative log-likelihood of Gamma regression\n",
"\n",
"Secondly, we define the loss function `gamma_loss` accorting to [1](#loss) that matches the data generating function `make_gamma_data`."
]
},
Expand Down Expand Up @@ -300,8 +288,7 @@
"id": "56f9e16e",
"metadata": {},
"source": [
"### Use skscope to solve the sparse gamma regression problem\n",
"\n",
"We use `skscope` to solve the sparse gamma regression problem.\n",
"After defining the data generation and loss function, we can call `ScopeSolver` to solve the sparse-constrained optimization problem."
]
},
Expand Down Expand Up @@ -398,8 +385,6 @@
"id": "ad64d6ca",
"metadata": {},
"source": [
"### More on the results\n",
"\n",
"We can plot the sparse signal recovering from the noisy observations to visualize the results."
]
},
Expand Down
30 changes: 28 additions & 2 deletions docs/source/gallery/GeneralizedLinearModels/logistic-regression.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,28 +25,42 @@
"source": [
"### Introduction\n",
"\n",
"Logistic regression is an important model to solve classification problem, which is expressed specifically as:\n",
"Logistic regression is a widely used statistical method for analyzing datasets where the outcome variable is binary. It was first introduced by David Cox in 1958 [[1]](#refer-anchor-1). Logistic regression models the probability of a binary response based on one or more predictor variables and is widely applied in various fields including medicine, finance, marketing, and social sciences. Here are some examples of its practical applications:\n",
"\n",
"- **Medical Diagnosis**: Logistic regression is used to predict the probability of a patient having a particular disease based on diagnostic tests and patient characteristics.\n",
"\n",
"- **Credit Scoring**: It helps in assessing the likelihood of a borrower defaulting on a loan based on their financial history and demographic information.\n",
"\n",
"- **Customer Behavior Prediction**: In marketing, logistic regression can be used to predict whether a customer will purchase a product based on their past behavior and demographic data.\n",
"\n",
"Logistic regression defines a probability distribution over the binary outcome $ y $, given the predictor variables $ x $. The probability of the outcome being 1 or 0 is modeled using the logistic function:\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"& P(y=1 \\mid x)=\\frac{1}{1+\\exp \\left(-x^T \\beta\\right)}, \\\\\n",
"& P(y=0 \\mid x)=\\frac{1}{1+\\exp \\left(x^T \\beta\\right)},\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"where $\\beta$ is an unknown parameter vector that to be estimated. Since we expect only a few explanatory variables contributing to predicting $y$, we assume $\\beta$ is sparse vector with sparsity level $s$.\n",
"\n",
"With $n$ independent data of the explanatory variables $x$ and the response variable $y$, we can estimate $\\beta$ by minimizing the negative log-likelihood function under sparsity constraint:\n",
"\n",
"<a id='loss'></a>\n",
"$$\n",
"\\arg \\min _{\\beta \\in R^p} L(\\beta):=-\\frac{1}{n} \\sum_{i=1}^n\\left\\{y_i x_i^T \\beta-\\log \\left(1+\\exp \\left(x_i^T \\beta\\right)\\right)\\right\\}, \\text { s.t. }\\|\\beta\\|_0 \\leq s \\tag{1}\n",
"$$ "
"$$ \n",
"\n",
"The coefficients $\\beta$ in logistic regression have an intuitive interpretation. Each coefficient $\\beta_j$ represents the log-odds change in the probability of the outcome per unit increase in the predictor variable $x_j$. The odds ratio is given by $e^{\\beta_j}$."
]
},
{
"cell_type": "markdown",
"id": "c6ce97f7",
"metadata": {},
"source": [
"### Examples\n",
"\n",
"We first import necessary packages. "
]
},
Expand Down Expand Up @@ -438,6 +452,18 @@
"source": [
"We can see that opening warm-start strategy accelerates the computation. "
]
},
{
"cell_type": "markdown",
"id": "d2542b43",
"metadata": {},
"source": [
"### Reference\n",
"\n",
"<div id=\"refer-anchor-1\"></div>\n",
"\n",
"- [1] Cox, D. R. (1958). The regression analysis of binary sequences. Journal of the Royal Statistical Society Series B: Statistical Methodology, 20(2), 215-232."
]
}
],
"metadata": {
Expand Down
Loading

0 comments on commit 4f821a0

Please sign in to comment.