Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
@ZPdesu
ZPdesu committed Jun 2, 2021
1 parent dc97c84 commit afa242f
Show file tree
Hide file tree
Showing 13 changed files with 502 additions and 1 deletion.
33 changes: 32 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
@@ -1 +1,32 @@
# Barbershop
# Barbershop: GAN-based Image Compositing using Segmentation Masks
![teaser](assets/docs/teaser.png)

> [**Barbershop: GAN-based Image Compositing using Segmentation Masks**](https://zpdesu.github.io/Barbershop/)<br/>
[Peihao Zhu](https://github.com/ZPdesu),
[Rameen Abdal](https://github.com/RameenAbdal),
[John Femiani](https://scholar.google.com/citations?user=rS1xJIIAAAAJ&hl=en),
[Peter Wonka](http://peterwonka.net/)<br/>


> [arXiv](https://arxiv.org/abs/) | [BibTeX](#bibtex) | [Project Page](https://zpdesu.github.io/Barbershop/) | [Video](https://youtu.be/h_BK3e9BvbU)

> **Abstract** Seamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion which cause coupling between different parts of the image. Even though recent work on GANs enables synthesis of realistic hair or faces, it remains difficult to combine them into a single, coherent, and plausible image rather than a disjointed set of image patches. We present a novel solution to image blending, particularly for the problem of hairstyle transfer, based on GAN-inversion. We propose a novel latent space for image blending which is better at preserving detail and encoding spatial information, and propose a new GAN-embedding algorithm which is able to slightly modify images to conform to a common segmentation mask. Our novel representation enables the transfer of the visual properties from multiple reference images including specific details such as moles and wrinkles, and because we do image blending in a latent-space we are able to synthesize images that are coherent. Our approach avoids blending artifacts present in other approaches and finds a globally consistent image. Our results demonstrate a significant improvement over the current state of the art in a user study, with users preferring our blending solution over 95 percent of the time.


## Description
Official Implementation of Barbershop.


## Updates

**2/6/2021** Add project page.



## BibTeX

```
```

Binary file added docs/Barbershop.pdf
View file
Binary file not shown.
3 changes: 3 additions & 0 deletions docs/_config.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
theme: jekyll-theme-cayman
title: DeepView
description: [CVPR 2019 Paper]
38 changes: 38 additions & 0 deletions docs/_layout/default.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<!DOCTYPE html>
<html lang="{{ site.lang | default: "en-US" }}">
<head>
<meta charset="UTF-8">

{% seo %}
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#157878">
<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="{{ '/assets/css/style.css?v=' | append: site.github.build_revision | relative_url }}">
</head>
<body>
<section class="page-header">
<h1 class="project-name">{{ site.title | default: site.github.repository_name }}</h1>
<h2 class="project-tagline">{{ site.description | default: site.github.project_tagline }}</h2>
</section>

<section class="main-content">
{{ content }}

<footer class="site-footer">
<span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a>.</span>
</footer>
</section>

{% if site.google_analytics %}
<script type="text/javascript">
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', '{{ site.google_analytics }}', 'auto');
ga('send', 'pageview');
</script>
{% endif %}
</body>
</html>
6 changes: 6 additions & 0 deletions docs/assets/css/style.scss
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@import "{{ site.theme }}";

header {
display: none;
}

Binary file added docs/assets/teaser.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 11 additions & 0 deletions docs/bootstrap.min.css
View file

Large diffs are not rendered by default.

303 changes: 303 additions & 0 deletions docs/index.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,303 @@
<!doctype html><meta charset=utf-8>

<head>
<!-- Global site tag (gtag.js) - Google Analytics -->
<!-- <script async src="https://www.googletagmanager.com/gtag/js?id=UA-152598381-4"></script> -->
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());

gtag('config', 'UA-152598381-4');
</script>

<meta property="twitter:image" content="https://zpdesu.github.io/Barbershop/thumbs/tr_paper-1.jpg" />
<meta property="og:image" content="https://zpdesu.github.io/Barbershop/thumbs/tr_paper-1.jpg"/>
<meta property="og:type" content="website"/>
<meta property="og:url" content="https://zpdesu.github.io/Barbershop/" />
<meta property="og:title" content="Barbershop: GAN-based Image Compositing using Segmentation Masks" />
<meta property="og:description" content="Barbershop: GAN-based Image Compositing using Segmentation Masks" />
</head>

<title>Barbershop: GAN-based Image Compositing using Segmentation Masks</title>

<style>
body {
font-family: Roboto, "Roboto", sans-serif;
background: #F0F0F0;
color: #696969;
/* text-align: center; */
/* padding: 0px 40px 40px 40px; */
}
h1 {
display: block;
font-size: 2em;
margin-top: 0.67em;
margin-bottom: 0px;
margin-left: 0;
margin-right: 0;
font-weight: normal;
color: #000;
}
h2 {
display: block;
font-size: 1.5em;
margin-top: 0.67em;
margin-bottom: 0.67em;
margin-left: 0;
margin-right: 0;
font-weight: normal;
color: #000;
}
h3 {
display: block;
font-size: 1em;
margin-top: 0.67em;
margin-bottom: 0.67em;
margin-left: 0;
margin-right: 0;
font-weight: normal;
}
h4 {
display: block;
font-size: 1.1em;
margin-top: 0.0em;
margin-bottom: 0.67em;
margin-left: 0;
margin-right: 0;
font-weight: normal;
color: #000;
}
hr {
border-top: 1px solid black;
}
.first-letter {
font-weight: 350;
display: inline;
font-size: 1.75em;
margin-block-start: 1em;
margin-block-end: 1em;
margin-inline-start: 0px;
margin-inline-end: 0px;
}
.paper-thumbnail {
box-shadow: 1px 2px 5px 2px rgba(0, 0, 0, .15);
width: 230px;
}
.grid-container {
display: grid;
grid-template-columns: auto auto auto auto;
text-align: center;
margin: 0px -10px 0px -10px;
}
.grid-container > div {
text-align: center;
}
.logos {
margin: 0px 10px 10px 0;
}
.ib {
display: inline-block;
margin: 00px 20px 20px 0px;
}
.logos-container {
margin: 20px 0px 50px 0px;
width: 100%;
float: left;
position: relative;
min-width: 800px;
}
.google-logo {
text-align: center;
position: relative;
}

.video {
margin: 20px 0px 0px 0px;
width: 100%;
float: left;
position: relative;
}
.abstract {
margin: 50px 0px 0px 0px;
width: 100%;
float: left;
position: relative;
}
.data {
margin: 40px 0px 80px 0px;
width: 100%;
float: left;
position: relative;
}
.superhero {
width: 1024px;
position: relative;
margin: auto;
padding-left: 80px;
padding-right: 80px;
overflow: auto;
/* min-width: 800px; */
}
.title {
margin: 50px 0px 0px 0px;
width: 100%;
float: left;
position: relative;
text-align: center;
/* line-height: 40pt; */
}
.title-line {
margin: 20px 0px 20px 0px;
}
.authors {
/* margin: 25px 0px 0px 0px; */
/* width: 400px;
height: 1100px; */
padding-left: 15%;
padding-right: 15%;
float: left;
position: relative;
text-align: center;
}
.paper {
width: 100%;
float: left;
position: relative;
}
h1.title-name {
font-size: 40px;
font-weight: 300;
margin-bottom: 0px;
display: inline-block;
}
h1.name {
font-size: 40px;
font-weight: 300;
margin-bottom: 10px;
display: inline-block;
}
.email {
display: block;
font-size: 19px;
font-weight: 300;
color: #81899C;
margin-top: 0px;
display: inline-block;
}

.lead {
font-size: 20px;
font-weight: 300;
margin-top: 0px;
line-height: 30px;
display: inline-block;
}
.names {
font-size: 1.05em;
font-weight: 300;
margin-top: 0px;
line-height: 20pt;
display: inline-block;
text-align: center;
}
.video-container {
position: relative;
padding-bottom: 56.25%; /* 16:9 */
height: 0;
}
.video-container iframe {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
}
.footer {
width: 100%;
height: 65px;
float: left;
margin-top: 25px;
}
.footer-googleai {
width: 20%;
float: left;
}
.footer-google {
width: 20%;
float: right;
text-align: right;
}
a.pagelink:link,
a.pagelink:visited,
a.pagelink:active { color: #81899C; text-decoration: none; }
a.pagelink:hover { color: #006699; text-decoration: underline; }
</style>

<body>
<div class="superhero">
<div class="title">
<h1 class="title-name"><strong>Barbershop:</strong></h1>
<h2>GAN-based Image Compositing using Segmentation Masks</h2>
<div class="title-line">
<hr style = "width: 32%;">
</div>
</div>
<div class ="authors">
<h2 class="names"><a href="https://github.com/ZPdesu"> Peihao Zhu</a>,
<a href="https://github.com/RameenAbdal"> Rameen Abdal</a>,
<a href="https://scholar.google.com/citations?user=rS1xJIIAAAAJ&hl=en"> John Femiani</a>,
<a href="http://peterwonka.net/"> Peter Wonka</a></h2>
</div>

<div class="logos-container">

<div class ="google-logo">
<h2 class="names"> <a href="https://s2021.siggraph.org/">arXiv</a> <a href="https://github.com/ZPdesu/Barbershop">Code</a></h2>
</div>

</div>

<div class="paper">
<div class="grid-container">
<div class="item1"><a href="Barbershop.pdf"><img src="thumbs/page-01.png" class="paper-thumbnail"></a></div>
<div class="item2"><a href="Barbershop.pdf"><img src="thumbs/page-04.png" class="paper-thumbnail"></a></div>
<div class="item3"><a href="Barbershop.pdf"><img src="thumbs/page-09.png" class="paper-thumbnail"></a></div>
<div class="item4"><a href="Barbershop.pdf"><img src="thumbs/page-11.png" class="paper-thumbnail"></a></div>
</div>
<br>
Click to view the paper.
</div>

<div class="video">
<h2 class="section-title">Video</h2>
<div class="video-container" style="margin-top: 50px;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/h_BK3e9BvbU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>

</div>

<div class="abstract">
<h2>Abstract</h2>
<p style="text-align: justify;">
<span class="first-letter">S</span>eamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion which cause coupling between different parts of the image. Even though recent work on GANs enables synthesis of realistic hair or faces, it remains difficult to combine them into a single, coherent, and plausible image rather than a disjointed set of image patches. We present a novel solution to image blending, particularly for the problem of hairstyle transfer, based on GAN-inversion. We propose a novel latent space for image blending which is better at preserving detail and encoding spatial information, and propose a new GAN-embedding algorithm which is able to slightly modify images to conform to a common segmentation mask. Our novel representation enables the transfer of the visual properties from multiple reference images including specific details such as moles and wrinkles, and because we do image blending in a latent-space we are able to synthesize images that are coherent. Our approach avoids blending artifacts present in other approaches and finds a globally consistent image. Our results demonstrate a significant improvement over the current state of the art in a user study, with users preferring our blending solution over 95 percent of the time.</p>
</div>
<br>

<div class="section list" >
<h2>BibTex</h2>
<div class="section bibtex">
<pre>
</pre>
</div>
</div>


</div>






</body>
Loading

0 comments on commit afa242f

Please sign in to comment.