Permalink
Please
sign in to comment.
Showing
with
2,023 additions
and 46,337 deletions.
- +3 −5 .gitignore
- +16 −48 README.md
- +122 −0 assets/css/style.css
- BIN assets/images/dh-mark.png
- +31 −0 assets/images/owl.svg
- BIN assets/images/preview.png
- +6 −0 assets/js/ga.js
- +0 −406 assets/js/image-viewer.js
- 0 assets/js/{AjaxTextureLoader.js → texture-loader.js}
- +3 −2 assets/{vendor → }/js/trackball-controls.js
- +563 −0 assets/js/tsne-webgl.js
- +912 −0 assets/js/tween.js
- +0 −66 assets/js/worker-manager.js
- +0 −43 assets/js/worker.js
- +6 −0 assets/requirements.txt
- +0 −78 assets/vendor/js/detector.js
- +0 −1 assets/vendor/js/stats.js
- +0 −44,110 assets/vendor/js/three-r86.js
- +0 −874 assets/vendor/js/three-r86.min.js
- +273 −0 imageplot.py
- +72 −143 index.html
- +16 −0 server.py
- +0 −282 utils/classify_images.py
- +0 −65 utils/cluster_vectors.py
- +0 −15 utils/deploy_to_s3.py
- +0 −58 utils/get_nearest_neighbors.py
- +0 −52 utils/make_montages.py
- +0 −8 utils/requirements.txt
- +0 −19 utils/resize_thumbs.py
- +0 −55 utils/select_images_to_display.py
- +0 −7 utils/upload_data_to_s3.py
@@ -1,7 +1,5 @@ | |||
.DS_Store | |||
data/ | |||
data.tar.gz | |||
output/ | |||
.DS_Store | |||
*.pem | |||
*.swp | |||
utils/image_vectors/ | |||
utils/texture-list.txt | |||
*.swp |
@@ -1,67 +1,35 @@ | |||
# TSNE Image Browser | |||
|
|||
This repository hosts source code used to identify and display 10,000 similar images in a WebGL-powered TSNE image browser. | |||
This repository hosts source code that visualizes tens of thousands of images in a two dimensional projection in which similar images are clustered together. The image analysis leverages Tensorflow's Inception bindings, and the visualization layer leverages a custom performant WebGL viewer. | |||
|
|||
![App preview](/assets/images/preview.png?raw=true) | |||
![App preview](./assets/images/preview.png?raw=true) | |||
|
|||
## Dependencies | |||
|
|||
The scripts in `utils/` rely upon the Python packages identified in `utils/requirememts.txt`. If you create a virtual environment or conda environment, you can run `pip install utils/requirements.txt` to resolve these dependencies. | |||
To install the Python dependencies, you can run (ideally in a virtual environment): | |||
```bash | |||
pip install -r assets/requirements.txt --user | |||
``` | |||
|
|||
Image resizing utilities require ImageMagick compiled with jpg support: | |||
`brew uninstall imagemagick && brew install imagemagick` | |||
```bash | |||
brew uninstall imagemagick && brew install imagemagick | |||
``` | |||
|
|||
The html viewer requires a WebGL-enabled browser. | |||
|
|||
## Quickstart | |||
|
|||
If you have a WebGL-enabled browser, you can start a local web server and see the application by running: | |||
If you have a WebGL-enabled browser and a directory full of images to process, you can prepare the data for the viewer by installing the dependencies above then running: | |||
|
|||
```bash | |||
python imageplot.py "path/to/images/*.jpg" | |||
``` | |||
git clone https://github.com/YaleDHLab/tsne-images-webgl | |||
cd tsne-images-webgl | |||
|
|||
wget https://s3-us-west-2.amazonaws.com/lab-apps/meserve-kunhardt/tsne-map/data.tar.gz | |||
tar -zxf data.tar.gz | |||
To see the results of this process, you can start a web server by running: | |||
|
|||
# Python3 | |||
python -m http.server 7051 | |||
# Python3 | |||
python -m SimpleHTTPServer 7051 | |||
```bash | |||
python server.py | |||
``` | |||
|
|||
The viewer will then be available on `localhost:7051`. | |||
|
|||
## Data Processing | |||
|
|||
The following table gives a quick overview of the utilities in `utils/`: | |||
|
|||
| File | Use | | |||
| ------------- | ------------- | | |||
| classify_images.py | Generates one numpy vector for each input image | | |||
| cluster_vectors.py | Builds a 2d TSNE model with input image vectors | | |||
| get_nearest_neighbors.py | Finds 100 nearest neighbors for each input image vector | | |||
| make_montage.py | Generates one large image file from many small images | | |||
| resize_thumbs.py | Resizes all images in a target directory to 128x128px | | |||
| select_images_to_display.py | Selects a subset of all images to create good-looking clusters | | |||
|
|||
To process new data, one can run: | |||
``` | |||
cd tsne-three-js/utils | |||
pip install -r requirements.txt | |||
# get the nearest neighbors for each image | |||
python get_nearest_neighbors.py | |||
# cherrypick a few thousand images for the tsne map | |||
python select_images_to_display.py | |||
# project the cherry-picked images into a 2d space | |||
python cluster_vectors.py | |||
# make a montage of the cherry-picked images | |||
python make_montage.py | |||
# upload the assets to some s3 bucket | |||
python upload_data_to_s3.py | |||
The visualization will then be available on port 5000. |
@@ -0,0 +1,122 @@ | |||
* { | |||
margin: 0; | |||
padding: 0; | |||
color: #fff; | |||
font-family: 'Open Sans'; | |||
} | |||
|
|||
body, | |||
html, | |||
canvas { | |||
background: #000; | |||
overflow: hidden; | |||
} | |||
|
|||
h2 { | |||
font-weight: 400; | |||
font-size: 1em; | |||
line-height: 1.05em; | |||
text-align: left; | |||
margin: 10px 0; | |||
} | |||
|
|||
/*** | |||
* Header | |||
***/ | |||
|
|||
.header { | |||
height: 60px; | |||
width: 100%; | |||
position: fixed; | |||
top: 0px; | |||
z-index: 1; | |||
overflow: hidden; | |||
box-shadow: 2px 1px 5px #000; | |||
} | |||
|
|||
.header, | |||
.logo { | |||
background: #333; | |||
} | |||
|
|||
.logo { | |||
width: 40px; | |||
padding: 10px 10px 10px 14px; | |||
height: 40px; | |||
} | |||
|
|||
.app-name, | |||
.tagline { | |||
display: inline-block; | |||
height: 100%; | |||
vertical-align: top; | |||
padding: 18px 34px; | |||
box-sizing: border-box; | |||
font-weight: 300; | |||
} | |||
|
|||
.app-name { | |||
background: #444; | |||
color: #fff; | |||
font-size: 1.3em; | |||
letter-spacing: 0.1em; | |||
} | |||
|
|||
.tagline { | |||
color: #999; | |||
font-size: 1em; | |||
padding-top: 20px; | |||
font-weight: 400; | |||
letter-spacing: .025em; | |||
} | |||
|
|||
@media(max-width: 335px) { | |||
.header .logo { | |||
display: none; | |||
} | |||
|
|||
.header .app-name { | |||
width: 100%; | |||
text-align: center; | |||
padding: 18px 0; | |||
} | |||
} | |||
|
|||
/** | |||
* Nav | |||
**/ | |||
|
|||
nav { | |||
position: absolute; | |||
top: 0; | |||
left: 0; | |||
width: 193px; | |||
min-height: 100%; | |||
max-height: 100%; | |||
overflow: auto; | |||
box-sizing: border-box; | |||
padding: 72px 10px 0 10px; | |||
background: rgba(17,17,17,0.9); | |||
box-shadow: 1px 0px 2px rgba(0, 0, 0, 0.7); | |||
transition: transform .5s; | |||
} | |||
|
|||
nav.hidden { | |||
transform: translateX(-210px); | |||
} | |||
|
|||
.hotspot { | |||
padding: 6px 2px; | |||
cursor: pointer; | |||
display: inline-block; | |||
text-align: center; | |||
width: 80px; | |||
text-transform: uppercase; | |||
font-size: 13px; | |||
} | |||
|
|||
.hotspot .background-image { | |||
padding-bottom: 110%; | |||
background-position: 50% 11%; | |||
margin: 0 5%; | |||
} |
Binary file not shown.
@@ -0,0 +1,31 @@ | |||
<?xml version="1.0" encoding="iso-8859-1"?> | |||
<!-- Generator: Adobe Illustrator 19.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) --> | |||
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" | |||
viewBox="0 0 512.001 512.001" style="enable-background:new 0 0 512.001 512.001;" xml:space="preserve"> | |||
<g> | |||
<path style="fill:#777777;" d="M124.477,378.183L9.347,495.779c21.628,21.628,56.695,21.628,78.324,0L119,464.45 | |||
c21.628,21.628,56.696,21.628,78.324,0l28.375-28.375C187.373,426.203,151.825,405.106,124.477,378.183z"/> | |||
<path style="fill:#888888;" d="M447.27,55.383h-55.383V177.22c0.002-40.997,22.277-76.788,55.383-95.939 | |||
c16.293-9.425,35.207-14.822,55.383-14.822c0-6.982,0-11.077,0-11.077V0C472.065,0,447.27,24.796,447.27,55.383z"/> | |||
</g> | |||
<path style="fill:#777777;" d="M336.504,55.383C336.504,24.796,311.708,0,281.121,0v66.46c20.176,0,39.091,5.397,55.383,14.822 | |||
c33.107,19.153,55.383,54.946,55.383,95.945V55.383H336.504z"/> | |||
<path style="fill:#888888;" d="M391.887,209.772v-27.116c0-3.312,0-5.432,0-5.432c0-40.997-22.276-76.791-55.383-95.942 | |||
c-16.293-9.425-35.207-14.822-55.383-14.822v155.073C311.213,191.443,357.554,187.532,391.887,209.772z M314.351,143.996 | |||
c0-12.234,9.918-22.153,22.153-22.153s22.153,9.919,22.153,22.153c0,12.235-9.918,22.153-22.153,22.153 | |||
S314.351,156.231,314.351,143.996z"/> | |||
<path style="fill:#777777;" d="M391.887,177.221v32.551c5.151,3.336,10.037,7.246,14.55,11.76h96.216V66.46 | |||
c-20.176,0-39.091,5.397-55.383,14.822C414.164,100.434,391.888,136.225,391.887,177.221z M469.423,143.996 | |||
c0,12.235-9.918,22.153-22.153,22.153s-22.153-9.918-22.153-22.153c0-12.234,9.918-22.153,22.153-22.153 | |||
C459.504,121.843,469.423,131.762,469.423,143.996z"/> | |||
<path style="fill:#8F5543;" d="M281.121,221.532l-10.2,10.2L281.121,221.532z"/> | |||
<path style="fill:#676767;" d="M406.438,221.533c34.606,34.606,34.606,90.712,0,125.319c-69.21,69.21-181.422,69.21-250.633,0.002 | |||
l-31.329,31.33c27.348,26.923,62.896,48.02,101.221,57.892c17.714,4.562,36.285,6.989,55.423,6.989 | |||
c122.349,0,221.532-99.182,221.532-221.531C502.653,221.533,406.437,221.532,406.438,221.533z"/> | |||
<path style="fill:#929191;" d="M281.121,221.532l125.318,125.319c34.606-34.606,34.606-90.713,0-125.319 | |||
c-4.515-4.514-9.401-8.425-14.551-11.761C357.554,187.532,311.213,191.443,281.121,221.532z"/> | |||
<path style="fill:#777777;" d="M406.438,346.851L281.12,221.533l-10.199,10.2L155.802,346.851 | |||
C225.017,416.062,337.228,416.061,406.438,346.851z"/> | |||
<circle style="fill:#ffffff;" cx="336.507" cy="143.996" r="22.153"/> | |||
<circle style="fill:#ffffff;" cx="447.274" cy="143.996" r="22.153"/> | |||
</svg> |
Binary file not shown.
@@ -0,0 +1,6 @@ | |||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ | |||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), | |||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) | |||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); | |||
ga('create', 'UA-101732875-1', 'auto'); | |||
ga('send', 'pageview'); |
Oops, something went wrong.
0 comments on commit
de36df3