Phobrain

Photo classification with TensorFlow For Poets

Thanks to Google for making this easy-to-use package available. The main improvement I can suggest is to make a Docker image that uses SSE and AVX speedups. Keywords are based on an attentional model for an interactive slide show, and I was curious to see how well colors, counts, and geometrical properties would be handled.
N: 15K photos, 400x400. Multiple keywords may apply to a given training photo. Tests were run on a 2012 laptop.

The models applied to photos.

Summary

Model	Categories	Photos (including dupes)	Train Accuracy	Cross entropy	Test Accuracy
Colors	4	1,834	92.0	0.378042	64.7
Counts	3	1,346	97.0	0.177031	76.1
Face/faces	2	1,720	95.0	0.179471	75.7
Geom: all	6	1,943	86.0	0.497265	53.6
Geom: all but 'square'	5	1,872	91.0	0.442485	59.0
Geom: all but 'square'&'triangle'	4	1,710	90.0	0.342200	69.4
Geom: only 'angular' & 'curve'	2	1,117	92.0	0.218759	80.7
Geom: all but 'angular' & 'curve'	4	826	94.0	0.245161	56.2
Graffiti/mural	2	803	93.0	0.153973	81.0
Juxtaposition	6	1,196	94.0	0.418914	40.8
Cases > 300	27	14,476	41.0	1.868935	32.4
Cases > 300, 50K steps	27	14,476	57.0	1.359802	26.4
Cases > 300, 500K steps	27	14,476	72.0	0.717565	19.3

Colors

Keyword	Cases
blue	270
green	286
yellow	405
red	873

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 19:22:50.109142: Step 4999: Train accuracy = 92.0%
2017-07-03 19:22:50.109875: Step 4999: Cross entropy = 0.378042
2017-07-03 19:22:50.445518: Step 4999: Validation accuracy = 58.0% (N=100)
Final test accuracy = 64.7% (N=184)

Counts

Keyword	Cases
pair	455
trio	151
array	740

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 20:58:50.476725: Step 4999: Train accuracy = 97.0%
2017-07-03 20:58:50.476855: Step 4999: Cross entropy = 0.177031
2017-07-03 20:58:50.828164: Step 4999: Validation accuracy = 62.0% (N=100)
Final test accuracy = 76.1% (N=134)

Face vs. faces

Keyword	Cases
face	1039
faces	681

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-04 05:05:54.234981: Step 4999: Train accuracy = 95.0%
2017-07-04 05:05:54.235100: Step 4999: Cross entropy = 0.179471
2017-07-04 05:05:54.632160: Step 4999: Validation accuracy = 73.0% (N=100)
Final test accuracy = 75.7% (N=169)

Geometry

Keyword	Cases
angular	570
curve	547
circle	291
triangle	162
square	71
radiate	302

Case	Cases	Train Accuracy	Cross entropy	Test Accuracy
All	1,943	86.0	0.497265	53.6
All but 'square'	1,872	91.0	0.442485	59.0
All but 'square'&'triangle'	1,710	90.0	0.342200	69.4
Only 'angular' & 'curve'	1,117	92.0	0.218759	80.7
All but 'angular' & 'curve'	826	94.0	0.245161	56.2

All

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 23:18:17.186763: Step 4999: Train accuracy = 86.0%
2017-07-03 23:18:17.186880: Step 4999: Cross entropy = 0.497265
2017-07-03 23:18:17.593359: Step 4999: Validation accuracy = 57.0% (N=100)
Final test accuracy = 53.6% (N=194)

Without 'square'

2017-07-03 23:57:37.833185: Step 4999: Train accuracy = 91.0%
2017-07-03 23:57:37.833313: Step 4999: Cross entropy = 0.442485
2017-07-03 23:57:38.271913: Step 4999: Validation accuracy = 44.0% (N=100)
Final test accuracy = 59.0% (N=188)

Without 'square' and 'triangle'

2017-07-04 00:37:58.211023: Step 4999: Train accuracy = 90.0%
2017-07-04 00:37:58.211443: Step 4999: Cross entropy = 0.342200
2017-07-04 00:37:58.542919: Step 4999: Validation accuracy = 59.0% (N=100)
Final test accuracy = 69.4% (N=173)

Only 'angular' and 'curve'

2017-07-04 01:22:54.607147: Step 4999: Train accuracy = 92.0%
2017-07-04 01:22:54.607258: Step 4999: Cross entropy = 0.218759
2017-07-04 01:22:54.992293: Step 4999: Validation accuracy = 67.0% (N=100)
Final test accuracy = 80.7% (N=114)

All but 'angular' and 'curve'

2017-07-04 02:02:11.049362: Step 4999: Train accuracy = 94.0%
2017-07-04 02:02:11.049476: Step 4999: Cross entropy = 0.245161
2017-07-04 02:02:11.494333: Step 4999: Validation accuracy = 66.0% (N=100)
Final test accuracy = 56.2% (N=80)

Graffiti & murals

Keyword	Cases
graffiti	428
mural	375

2017-07-04 02:46:25.944766: Step 4999: Train accuracy = 93.0%
2017-07-04 02:46:25.944878: Step 4999: Cross entropy = 0.153973
2017-07-04 02:46:26.332304: Step 4999: Validation accuracy = 83.0% (N=100)
Final test accuracy = 81.0% (N=79)

Juxtaposition

Keyword	Cases
juxtapose	284
juxtapose_align	388
juxtapose_ontop	168
juxtapose_opposite	148
juxtapose_size	92
juxtapose_texture	116

2017-07-04 05:53:51.393635: Step 4999: Train accuracy = 94.0%
2017-07-04 05:53:51.393787: Step 4999: Cross entropy = 0.418914
2017-07-04 05:53:51.790802: Step 4999: Validation accuracy = 40.0% (N=100)
Final test accuracy = 40.8% (N=130)

Cases > 300

Keyword	Cases
angular	570
array	740
costume	541
curve	547
downtown	442
face	1039
faces	681
flowers	310
graffiti	428
hang	313
juxtapose_align	388
man	1168
mural	375
orange	312
pair	455
pink	310
radiate	302
red	873
reflection	351
sculpture	606
shadow	705
sign	570
smile	498
texture	312
tree	397
woman	838
yellow	405

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 22:22:28.178970: Step 4999: Train accuracy = 41.0%
2017-07-03 22:22:28.179091: Step 4999: Cross entropy = 1.868935
2017-07-03 22:22:28.681500: Step 4999: Validation accuracy = 35.0% (N=100)
Final test accuracy = 32.4% (N=1461)

--how_many_training_steps=50000 --learning_rate=0.005

2017-07-04 17:15:52.153930: Step 49999: Train accuracy = 57.0%
2017-07-04 17:15:52.154050: Step 49999: Cross entropy = 1.359802
2017-07-04 17:15:52.543695: Step 49999: Validation accuracy = 20.0% (N=100)
Final test accuracy = 26.4% (N=1461)

--how_many_training_steps=500000 --learning_rate=0.005

2017-07-09 09:40:44.933220: Step 499999: Train accuracy = 72.0%
2017-07-09 09:40:44.933405: Step 499999: Cross entropy = 0.717565
2017-07-09 09:40:45.371540: Step 499999: Validation accuracy = 21.0% (N=100)
Final test accuracy = 19.3% (N=1461)

The models applied to photos.

Lost in the venal void our dreams deflate
By easy stages through green atmosphere,
Imagination's bright balloon is late, like the blue whale,
In coming up for air.

It is not known what genus of the wild
Black plums of thought best wrinkle, twitch and flow
Into sweet wisdom's prune -- for in the mild
Orchards of love there is no need to know.

— Mervyn Peake, Titus Alone