Skip to content

Commit

Permalink
Merge pull request TikhonJelvis#161 from TikhonJelvis/appendix6-fix
Browse files Browse the repository at this point in the history
quick fix
  • Loading branch information
coverdrive authored Sep 23, 2021
2 parents 18a1b64 + 2587d03 commit 91e64c7
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion appendix6/appendix6.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ We represent function approximations by parameterized functions $f: \mathcal{X}

#### $D[\mathbb{R}]$ as an Affine Space $\mathcal{P}$

When performing Stochastic Gradient Descent or Batch Gradient Descent, parameters $\bm{p} \in D[\mathbb{R}]$ of a function approximation $f: \mathcal{X} \times D[\mathbb{R}] \rightarrow \mathbb{R}$ are updated using an appropriate linear combination of gradients of $f$ with respect to $w$ (at specific values of $x \in \mathcal{X}$). Hence, the parameters domain $D[\mathbb{R}]$ can be treated as an affine space (call it $\mathcal{P}$) whose associated vector space (over scalars field $\mathbb{R}$) is the set of gradients of $f$ with respect to parameters $\bm{p} \in D[\mathbb{R}]$ (denoted as $\nabla_{\bm{p}} f(x, \bm{p})$), evaluated at specific values of $x \in \mathcal{X}$, with addition operation defined as element-wise real-numbered addition and scalar multiplication operation defined as element-wise multiplication with real-numbered scalars. We refer to this Affine Space $\mathcal{P}$ as the *Parameters Space* and we refer to it's associated vector space (of gradients) as the *Gradient Space* $\mathcal{G}$. Since each *point* in $\mathcal{P}$ and each *translation* in $\mathcal{G}$ is an element in $D[\mathcal{R}]$, the $\oplus$ operation is element-wise real-numbered addition.
When performing Stochastic Gradient Descent or Batch Gradient Descent, parameters $\bm{p} \in D[\mathbb{R}]$ of a function approximation $f: \mathcal{X} \times D[\mathbb{R}] \rightarrow \mathbb{R}$ are updated using an appropriate linear combination of gradients of $f$ with respect to $w$ (at specific values of $x \in \mathcal{X}$). Hence, the parameters domain $D[\mathbb{R}]$ can be treated as an affine space (call it $\mathcal{P}$) whose associated vector space (over scalars field $\mathbb{R}$) is the set of gradients of $f$ with respect to parameters $\bm{p} \in D[\mathbb{R}]$ (denoted as $\nabla_{\bm{p}} f(x, \bm{p})$), evaluated at specific values of $x \in \mathcal{X}$, with addition operation defined as element-wise real-numbered addition and scalar multiplication operation defined as element-wise multiplication with real-numbered scalars. We refer to this Affine Space $\mathcal{P}$ as the *Parameters Space* and we refer to it's associated vector space (of gradients) as the *Gradient Space* $\mathcal{G}$. Since each *point* in $\mathcal{P}$ and each *translation* in $\mathcal{G}$ is an element in $D[\mathbb{R}]$, the $\oplus$ operation is element-wise real-numbered addition.

We define the gradient function
$$G: \mathcal{X} \rightarrow (\mathcal{P} \rightarrow \mathcal{G})$$
Expand Down

0 comments on commit 91e64c7

Please sign in to comment.