Skip to content
Permalink
Browse files

clean room rewrite

  • Loading branch information...
duhaime committed Dec 7, 2017
1 parent 11489e3 commit de36df3ab5991ff02c100c6486920bfb778cdcc9
@@ -1,7 +1,5 @@
.DS_Store
data/
data.tar.gz
output/
.DS_Store
*.pem
*.swp
utils/image_vectors/
utils/texture-list.txt
*.swp
@@ -1,67 +1,35 @@
# TSNE Image Browser

This repository hosts source code used to identify and display 10,000 similar images in a WebGL-powered TSNE image browser.
This repository hosts source code that visualizes tens of thousands of images in a two dimensional projection in which similar images are clustered together. The image analysis leverages Tensorflow's Inception bindings, and the visualization layer leverages a custom performant WebGL viewer.

![App preview](/assets/images/preview.png?raw=true)
![App preview](./assets/images/preview.png?raw=true)

## Dependencies

The scripts in `utils/` rely upon the Python packages identified in `utils/requirememts.txt`. If you create a virtual environment or conda environment, you can run `pip install utils/requirements.txt` to resolve these dependencies.
To install the Python dependencies, you can run (ideally in a virtual environment):
```bash
pip install -r assets/requirements.txt --user
```

Image resizing utilities require ImageMagick compiled with jpg support:
`brew uninstall imagemagick && brew install imagemagick`
```bash
brew uninstall imagemagick && brew install imagemagick
```

The html viewer requires a WebGL-enabled browser.

## Quickstart

If you have a WebGL-enabled browser, you can start a local web server and see the application by running:
If you have a WebGL-enabled browser and a directory full of images to process, you can prepare the data for the viewer by installing the dependencies above then running:

```bash
python imageplot.py "path/to/images/*.jpg"
```
git clone https://github.com/YaleDHLab/tsne-images-webgl
cd tsne-images-webgl

wget https://s3-us-west-2.amazonaws.com/lab-apps/meserve-kunhardt/tsne-map/data.tar.gz
tar -zxf data.tar.gz
To see the results of this process, you can start a web server by running:

# Python3
python -m http.server 7051
# Python3
python -m SimpleHTTPServer 7051
```bash
python server.py
```

The viewer will then be available on `localhost:7051`.

## Data Processing

The following table gives a quick overview of the utilities in `utils/`:

| File | Use |
| ------------- | ------------- |
| classify_images.py | Generates one numpy vector for each input image |
| cluster_vectors.py | Builds a 2d TSNE model with input image vectors |
| get_nearest_neighbors.py | Finds 100 nearest neighbors for each input image vector |
| make_montage.py | Generates one large image file from many small images |
| resize_thumbs.py | Resizes all images in a target directory to 128x128px |
| select_images_to_display.py | Selects a subset of all images to create good-looking clusters |

To process new data, one can run:
```
cd tsne-three-js/utils
pip install -r requirements.txt
# get the nearest neighbors for each image
python get_nearest_neighbors.py
# cherrypick a few thousand images for the tsne map
python select_images_to_display.py
# project the cherry-picked images into a 2d space
python cluster_vectors.py
# make a montage of the cherry-picked images
python make_montage.py
# upload the assets to some s3 bucket
python upload_data_to_s3.py
The visualization will then be available on port 5000.
@@ -0,0 +1,122 @@
* {
margin: 0;
padding: 0;
color: #fff;
font-family: 'Open Sans';
}

body,
html,
canvas {
background: #000;
overflow: hidden;
}

h2 {
font-weight: 400;
font-size: 1em;
line-height: 1.05em;
text-align: left;
margin: 10px 0;
}

/***
* Header
***/

.header {
height: 60px;
width: 100%;
position: fixed;
top: 0px;
z-index: 1;
overflow: hidden;
box-shadow: 2px 1px 5px #000;
}

.header,
.logo {
background: #333;
}

.logo {
width: 40px;
padding: 10px 10px 10px 14px;
height: 40px;
}

.app-name,
.tagline {
display: inline-block;
height: 100%;
vertical-align: top;
padding: 18px 34px;
box-sizing: border-box;
font-weight: 300;
}

.app-name {
background: #444;
color: #fff;
font-size: 1.3em;
letter-spacing: 0.1em;
}

.tagline {
color: #999;
font-size: 1em;
padding-top: 20px;
font-weight: 400;
letter-spacing: .025em;
}

@media(max-width: 335px) {
.header .logo {
display: none;
}

.header .app-name {
width: 100%;
text-align: center;
padding: 18px 0;
}
}

/**
* Nav
**/

nav {
position: absolute;
top: 0;
left: 0;
width: 193px;
min-height: 100%;
max-height: 100%;
overflow: auto;
box-sizing: border-box;
padding: 72px 10px 0 10px;
background: rgba(17,17,17,0.9);
box-shadow: 1px 0px 2px rgba(0, 0, 0, 0.7);
transition: transform .5s;
}

nav.hidden {
transform: translateX(-210px);
}

.hotspot {
padding: 6px 2px;
cursor: pointer;
display: inline-block;
text-align: center;
width: 80px;
text-transform: uppercase;
font-size: 13px;
}

.hotspot .background-image {
padding-bottom: 110%;
background-position: 50% 11%;
margin: 0 5%;
}
Binary file not shown.
@@ -0,0 +1,31 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Generator: Adobe Illustrator 19.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 512.001 512.001" style="enable-background:new 0 0 512.001 512.001;" xml:space="preserve">
<g>
<path style="fill:#777777;" d="M124.477,378.183L9.347,495.779c21.628,21.628,56.695,21.628,78.324,0L119,464.45
c21.628,21.628,56.696,21.628,78.324,0l28.375-28.375C187.373,426.203,151.825,405.106,124.477,378.183z"/>
<path style="fill:#888888;" d="M447.27,55.383h-55.383V177.22c0.002-40.997,22.277-76.788,55.383-95.939
c16.293-9.425,35.207-14.822,55.383-14.822c0-6.982,0-11.077,0-11.077V0C472.065,0,447.27,24.796,447.27,55.383z"/>
</g>
<path style="fill:#777777;" d="M336.504,55.383C336.504,24.796,311.708,0,281.121,0v66.46c20.176,0,39.091,5.397,55.383,14.822
c33.107,19.153,55.383,54.946,55.383,95.945V55.383H336.504z"/>
<path style="fill:#888888;" d="M391.887,209.772v-27.116c0-3.312,0-5.432,0-5.432c0-40.997-22.276-76.791-55.383-95.942
c-16.293-9.425-35.207-14.822-55.383-14.822v155.073C311.213,191.443,357.554,187.532,391.887,209.772z M314.351,143.996
c0-12.234,9.918-22.153,22.153-22.153s22.153,9.919,22.153,22.153c0,12.235-9.918,22.153-22.153,22.153
S314.351,156.231,314.351,143.996z"/>
<path style="fill:#777777;" d="M391.887,177.221v32.551c5.151,3.336,10.037,7.246,14.55,11.76h96.216V66.46
c-20.176,0-39.091,5.397-55.383,14.822C414.164,100.434,391.888,136.225,391.887,177.221z M469.423,143.996
c0,12.235-9.918,22.153-22.153,22.153s-22.153-9.918-22.153-22.153c0-12.234,9.918-22.153,22.153-22.153
C459.504,121.843,469.423,131.762,469.423,143.996z"/>
<path style="fill:#8F5543;" d="M281.121,221.532l-10.2,10.2L281.121,221.532z"/>
<path style="fill:#676767;" d="M406.438,221.533c34.606,34.606,34.606,90.712,0,125.319c-69.21,69.21-181.422,69.21-250.633,0.002
l-31.329,31.33c27.348,26.923,62.896,48.02,101.221,57.892c17.714,4.562,36.285,6.989,55.423,6.989
c122.349,0,221.532-99.182,221.532-221.531C502.653,221.533,406.437,221.532,406.438,221.533z"/>
<path style="fill:#929191;" d="M281.121,221.532l125.318,125.319c34.606-34.606,34.606-90.713,0-125.319
c-4.515-4.514-9.401-8.425-14.551-11.761C357.554,187.532,311.213,191.443,281.121,221.532z"/>
<path style="fill:#777777;" d="M406.438,346.851L281.12,221.533l-10.199,10.2L155.802,346.851
C225.017,416.062,337.228,416.061,406.438,346.851z"/>
<circle style="fill:#ffffff;" cx="336.507" cy="143.996" r="22.153"/>
<circle style="fill:#ffffff;" cx="447.274" cy="143.996" r="22.153"/>
</svg>
BIN +460 KB (220%) assets/images/preview.png
Binary file not shown.
@@ -0,0 +1,6 @@
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-101732875-1', 'auto');
ga('send', 'pageview');

0 comments on commit de36df3

Please sign in to comment.
You can’t perform that action at this time.