Skip to content

Commit

Permalink
Fidelity docs (google#1473)
Browse files Browse the repository at this point in the history 
* added fidelity description

* changed scenario order
  • Loading branch information
@elalish
elalish authored Aug 21, 2020
1 parent f4ec94a commit 7c21de7
Show file tree
Hide file tree
Showing 2 changed files with 150 additions and 84 deletions.
160 changes: 80 additions & 80 deletions packages/render-fidelity-tools/test/config.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
{
"name": "babylon",
"description": "Babylon"
},
},
{
"name": "dspbr-pt",
"description": "dspbr-pt"
Expand Down Expand Up @@ -159,6 +159,55 @@
"radius": 3
}
},
{
"name": "khronos-MetalRoughSpheres",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/MetalRoughSpheres/glTF/MetalRoughSpheres.gltf",
"orbit": {
"radius": 12
},
"renderSkybox": true
},
{
"name": "khronos-MetalRoughSpheres-HDR",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/MetalRoughSpheres/glTF/MetalRoughSpheres.gltf",
"lighting": "../../../shared-assets/environments/spruit_sunrise_1k_HDR.hdr",
"orbit": {
"radius": 12
},
"renderSkybox": true,
"exclude": [
"dspbr-pt"
]
},
{
"name": "khronos-MetalRoughSpheres-LDR",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/MetalRoughSpheres/glTF/MetalRoughSpheres.gltf",
"lighting": "../../../shared-assets/environments/spruit_sunrise_1k_LDR.jpg",
"orbit": {
"radius": 12
},
"exclude": [
"dspbr-pt",
"babylon"
],
"renderSkybox": true
},
{
"name": "khronos-EnvironmentTest",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/EnvironmentTest/glTF/EnvironmentTest.gltf",
"dimensions": {
"width": 1024
},
"target": {
"y": 3.8
},
"orbit": {
"radius": 20
},
"exclude": [
"dspbr-pt"
]
},
{
"name": "khronos-VC",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/VC/glTF-Binary/VC.glb",
Expand Down Expand Up @@ -208,55 +257,37 @@
]
},
{
"name": "khronos-MetalRoughSpheres",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/MetalRoughSpheres/glTF/MetalRoughSpheres.gltf",
"orbit": {
"radius": 12
"name": "khronos-AlphaBlendModeTest",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/AlphaBlendModeTest/glTF-Binary/AlphaBlendModeTest.glb",
"dimensions": {
"width": 1024,
"height": 512
},
"renderSkybox": true
},
{
"name": "khronos-MetalRoughSpheres-HDR",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/MetalRoughSpheres/glTF/MetalRoughSpheres.gltf",
"lighting": "../../../shared-assets/environments/spruit_sunrise_1k_HDR.hdr",
"orbit": {
"radius": 12
"target": {
"y": 1.1
},
"renderSkybox": true,
"exclude":[
"dspbr-pt"
]
},
{
"name": "khronos-MetalRoughSpheres-LDR",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/MetalRoughSpheres/glTF/MetalRoughSpheres.gltf",
"lighting": "../../../shared-assets/environments/spruit_sunrise_1k_LDR.jpg",
"orbit": {
"radius": 12
"radius": 6
},
"exclude": [
"dspbr-pt",
"babylon"
],
"renderSkybox": true
"dspbr-pt"
]
},
{
"name": "DirectionalLightTest",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/EnvironmentTest/glTF/EnvironmentTest.gltf",
"lighting": "../../../shared-assets/environments/spot1Lux.hdr",
"name": "khronos-BoomBoxWithAxes",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/BoomBoxWithAxes/glTF/BoomBoxWithAxes.gltf",
"dimensions": {
"width": 1024
"height": 1024
},
"target": {
"y": 3.8
"y": 0.01
},
"orbit": {
"radius": 20
"radius": 0.05
},
"exclude": [
"dspbr-pt"
],
"renderSkybox": true
]
},
{
"name": "khronos-2CylinderEngine",
Expand Down Expand Up @@ -326,22 +357,6 @@
"dspbr-pt"
]
},
{
"name": "khronos-EnvironmentTest",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/EnvironmentTest/glTF/EnvironmentTest.gltf",
"dimensions": {
"width": 1024
},
"target": {
"y": 3.8
},
"orbit": {
"radius": 20
},
"exclude": [
"dspbr-pt"
]
},
{
"name": "khronos-GearboxAssy",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/GearboxAssy/glTF/GearboxAssy.gltf",
Expand Down Expand Up @@ -414,7 +429,7 @@
"phi": 60,
"radius": 2.38
},
"exclude":[
"exclude": [
"dspbr-pt"
]
},
Expand All @@ -427,7 +442,7 @@
"orbit": {
"radius": 18.4
},
"exclude":[
"exclude": [
"dspbr-pt"
]
},
Expand All @@ -449,7 +464,7 @@
"orbit": {
"radius": 4.25
},
"exclude":[
"exclude": [
"dspbr-pt"
]
},
Expand All @@ -459,7 +474,7 @@
"orbit": {
"radius": 4.25
},
"exclude":[
"exclude": [
"dspbr-pt"
]
},
Expand All @@ -471,7 +486,7 @@
"phi": 60,
"radius": 23
},
"exclude":[
"exclude": [
"dspbr-pt"
]
},
Expand Down Expand Up @@ -736,37 +751,22 @@
"renderSkybox": true
},
{
"name": "khronos-AlphaBlendModeTest",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/AlphaBlendModeTest/glTF-Binary/AlphaBlendModeTest.glb",
"dimensions": {
"width": 1024,
"height": 512
},
"target": {
"y": 1.1
},
"orbit": {
"radius": 6
},
"exclude":[
"dspbr-pt"
]
},
{
"name": "khronos-BoomBoxWithAxes",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/BoomBoxWithAxes/glTF/BoomBoxWithAxes.gltf",
"name": "DirectionalLightTest",
"model": "../../../shared-assets/models/glTF-Sample-Models/2.0/EnvironmentTest/glTF/EnvironmentTest.gltf",
"lighting": "../../../shared-assets/environments/spot1Lux.hdr",
"dimensions": {
"height": 1024
"width": 1024
},
"target": {
"y": 0.01
"y": 3.8
},
"orbit": {
"radius": 0.05
"radius": 20
},
"exclude":[
"exclude": [
"dspbr-pt"
]
],
"renderSkybox": true
}
]
}
74 changes: 70 additions & 4 deletions packages/render-fidelity-tools/test/results-viewer.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,79 @@
<!-- Web Components polyfill is required to support Edge and Firefox < 63: -->
<script src="../node_modules/@webcomponents/webcomponentsjs/webcomponents-loader.js"></script>
<style>
body {
margin: 1em;
background-color: #f7f7f7;
}
@import url('https://fonts.googleapis.com/css?family=Rubik:400,500');
body {
margin: 1em;
background-color: #f7f7f7;
font-family: 'Rubik', sans-serif;
font-size: 16px;
line-height: 24px;
color: rgba(0,0,0,.87);
font-weight: 400;
-webkit-font-smoothing: antialiased;
}

p {
max-width: 1000px;
text-align: left;
}
</style>
</head>
<body>
<div align="center">
<h2>Render Fidelity Comparison Results</h1>

<p>The purpose of <a href="https://www.khronos.org/gltf/"
target="_blank">glTF</a> is to standardize Physically-Based Rendering (PBR)
materials such that you can be confident your model will appear as intended
in any lighting environment in any renderer. This is a very ambitious goal,
as real-time rendering at this level of quality is still very much an area
of active research with improvements being made constantly. This site
demonstrates where we are on that path to convergence and highlights areas
that could still use improvement. We are comparing the most popular web
renderers: <a href="https://threejs.org/" target="_blank">three.js</a>
(represented by &lt;model-viewer&gt;), <a
href="https://google.github.io/filament/webgl/"
target="_blank">filament.js</a>, and <a href="https://www.babylonjs.com/"
target="_blank">babylon.js</a>. If any other renderers would like to be
included, please open a PR adding them to the <a
href="https://github.com/google/model-viewer/tree/master/packages/render-fidelity-tools/src/components/renderers"
target="_blank">render-fidelity-tools</a>
package.</p>

<p>In addition, we are showing a ground-truth render provided by <a
href="https://github.com/DassaultSystemes-Technology/dspbr-pt"
target="_blank">dspbr-pt</a>, which is a very slow path-tracing renderer
based on three.js. The advantage it provides is simplicity: without the
speed requirement it can be made clear and readable and as it runs with more
and more samples it approaches the physical result with far fewer
approximations than are necessary in real-time renderers.</p>

<p>The first set of scenarios are for demonstrating material capabilities,
so these are primarily fidelity tests. Following these are simpler cases
that test glTF spec conformance, but are not particularly interesting for
fidelity. At the bottom are the Furnace Test and the Directional Light Test.
The Furnace Test checks energy conservation and the correct result is the
whole 7x7 grid of metal-rough spheres being invisible (indistinguishable
from the background). The top (metal) and bottom (dielectric) rows are
passing, while the mixtures lose some energy. The Directional Light Test
checks the worst-case scenario for environmental lighting approximations:
the case where a single bright pixel represents a directional light. The
Filament version uses an actual directional light, so this is the ground
truth.</p>

<p>In &lt;model-viewer&gt;, we do not consider rendering changes to be breaking
changes, as our quality is incrementally improving with every release.
However, the difference between the renders you see here bounds how much you
can expect our rendering to change going forward. Note the largest
differences are in the handling of transparent materials, as this is quite
difficult to get right, so the largest changes will likely be here.</p>

<p>We show a simple logarithmic metric on an average of the pixel
differences, excluding the transparent background. This is <b>not</b> a
perceptual metric, so take it with a grain of salt. It is mostly there to
help us identify regressions.</p>
</div>

<image-comparison-app src="./results/config.json"></image-comparison-app>

Expand Down

0 comments on commit 7c21de7

Please sign in to comment.