Phobrain

Photo classification with TensorFlow For Poets

Google's TensorFlow For Poets

Thanks to Google for making this easy-to-use package available. The main improvement I can suggest is to make a Docker image that uses SSE and AVX speedups. Keywords are based on an attentional model for an interactive slide show, and I was curious to see how well colors, counts, and geometrical properties would be handled.
N: 15K photos, 400x400. Multiple keywords may apply to a given training photo. Tests were run on a 2012 laptop.

The models applied to photos.

Summary

ModelCategoriesPhotos
(including dupes)
Train AccuracyCross entropyTest Accuracy
Colors 41,83492.00.37804264.7
Counts 31,34697.00.17703176.1
Face/faces 21,72095.00.17947175.7
Geom: all 61,94386.00.49726553.6
Geom: all but 'square' 51,87291.00.44248559.0
Geom: all but 'square'&'triangle' 41,71090.00.34220069.4
Geom: only 'angular' & 'curve' 21,11792.00.21875980.7
Geom: all but 'angular' & 'curve' 482694.00.24516156.2
Graffiti/mural 280393.00.15397381.0
Juxtaposition 61,19694.00.41891440.8
Cases > 300 2714,47641.01.86893532.4
Cases > 300, 50K steps 2714,47657.01.35980226.4
Cases > 300, 500K steps 2714,47672.00.71756519.3


Colors

KeywordCases
blue270
green286
yellow405
red873

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 19:22:50.109142: Step 4999: Train accuracy = 92.0%
2017-07-03 19:22:50.109875: Step 4999: Cross entropy = 0.378042
2017-07-03 19:22:50.445518: Step 4999: Validation accuracy = 58.0% (N=100)
Final test accuracy = 64.7% (N=184)

Counts

KeywordCases
pair455
trio151
array740

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 20:58:50.476725: Step 4999: Train accuracy = 97.0%
2017-07-03 20:58:50.476855: Step 4999: Cross entropy = 0.177031
2017-07-03 20:58:50.828164: Step 4999: Validation accuracy = 62.0% (N=100)
Final test accuracy = 76.1% (N=134)

Face vs. faces

KeywordCases
face1039
faces681

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-04 05:05:54.234981: Step 4999: Train accuracy = 95.0%
2017-07-04 05:05:54.235100: Step 4999: Cross entropy = 0.179471
2017-07-04 05:05:54.632160: Step 4999: Validation accuracy = 73.0% (N=100)
Final test accuracy = 75.7% (N=169)

Geometry

KeywordCases
angular570
curve547
circle291
triangle162
square71
radiate302


CaseCases Train AccuracyCross entropyTest Accuracy
All1,94386.00.49726553.6
All but 'square'1,87291.00.44248559.0
All but 'square'&'triangle'1,71090.00.34220069.4
Only 'angular' & 'curve' 1,11792.00.21875980.7
All but 'angular' & 'curve' 82694.00.24516156.2

All

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 23:18:17.186763: Step 4999: Train accuracy = 86.0%
2017-07-03 23:18:17.186880: Step 4999: Cross entropy = 0.497265
2017-07-03 23:18:17.593359: Step 4999: Validation accuracy = 57.0% (N=100)
Final test accuracy = 53.6% (N=194)

Without 'square'

2017-07-03 23:57:37.833185: Step 4999: Train accuracy = 91.0%
2017-07-03 23:57:37.833313: Step 4999: Cross entropy = 0.442485
2017-07-03 23:57:38.271913: Step 4999: Validation accuracy = 44.0% (N=100)
Final test accuracy = 59.0% (N=188)

Without 'square' and 'triangle'

2017-07-04 00:37:58.211023: Step 4999: Train accuracy = 90.0%
2017-07-04 00:37:58.211443: Step 4999: Cross entropy = 0.342200
2017-07-04 00:37:58.542919: Step 4999: Validation accuracy = 59.0% (N=100)
Final test accuracy = 69.4% (N=173)

Only 'angular' and 'curve'

2017-07-04 01:22:54.607147: Step 4999: Train accuracy = 92.0%
2017-07-04 01:22:54.607258: Step 4999: Cross entropy = 0.218759
2017-07-04 01:22:54.992293: Step 4999: Validation accuracy = 67.0% (N=100)
Final test accuracy = 80.7% (N=114)

All but 'angular' and 'curve'

2017-07-04 02:02:11.049362: Step 4999: Train accuracy = 94.0%
2017-07-04 02:02:11.049476: Step 4999: Cross entropy = 0.245161
2017-07-04 02:02:11.494333: Step 4999: Validation accuracy = 66.0% (N=100)
Final test accuracy = 56.2% (N=80)

Graffiti & murals

KeywordCases
graffiti428
mural375
2017-07-04 02:46:25.944766: Step 4999: Train accuracy = 93.0%
2017-07-04 02:46:25.944878: Step 4999: Cross entropy = 0.153973
2017-07-04 02:46:26.332304: Step 4999: Validation accuracy = 83.0% (N=100)
Final test accuracy = 81.0% (N=79)

Juxtaposition

KeywordCases
juxtapose284
juxtapose_align388
juxtapose_ontop168
juxtapose_opposite148
juxtapose_size92
juxtapose_texture116
2017-07-04 05:53:51.393635: Step 4999: Train accuracy = 94.0%
2017-07-04 05:53:51.393787: Step 4999: Cross entropy = 0.418914
2017-07-04 05:53:51.790802: Step 4999: Validation accuracy = 40.0% (N=100)
Final test accuracy = 40.8% (N=130)

Cases > 300

KeywordCases
angular570
array740
costume541
curve547
downtown442
face1039
faces681
flowers310
graffiti428
hang313
juxtapose_align388
man1168
mural375
orange312
pair455
pink310
radiate302
red873
reflection351
sculpture606
shadow705
sign570
smile498
texture312
tree397
woman838
yellow405

--how_many_training_steps=5000 --learning_rate=0.005

2017-07-03 22:22:28.178970: Step 4999: Train accuracy = 41.0%
2017-07-03 22:22:28.179091: Step 4999: Cross entropy = 1.868935
2017-07-03 22:22:28.681500: Step 4999: Validation accuracy = 35.0% (N=100)
Final test accuracy = 32.4% (N=1461)

--how_many_training_steps=50000 --learning_rate=0.005

2017-07-04 17:15:52.153930: Step 49999: Train accuracy = 57.0%
2017-07-04 17:15:52.154050: Step 49999: Cross entropy = 1.359802
2017-07-04 17:15:52.543695: Step 49999: Validation accuracy = 20.0% (N=100)
Final test accuracy = 26.4% (N=1461)

--how_many_training_steps=500000 --learning_rate=0.005

2017-07-09 09:40:44.933220: Step 499999: Train accuracy = 72.0%
2017-07-09 09:40:44.933405: Step 499999: Cross entropy = 0.717565
2017-07-09 09:40:45.371540: Step 499999: Validation accuracy = 21.0% (N=100)
Final test accuracy = 19.3% (N=1461)



The models applied to photos.

You're not doing it right!

Lost in the venal void our dreams deflate
By easy stages through green atmosphere,
Imagination's bright balloon is late, like the blue whale,
In coming up for air.

It is not known what genus of the wild
Black plums of thought best wrinkle, twitch and flow
Into sweet wisdom's prune -- for in the mild
Orchards of love there is no need to know.


         — Mervyn Peake,
Titus Alone


© 2017 Photoriot.