A review of Deep Learning models for Image Sentiment Analysis

Alex Egg,

1. Xu, Cetintas, Chih, Li, Visual Sentiment Prediction with Deep Convolutional Neural Networks, Nov 2014

Transer Learning on AlexNet w/ LogRegression

64-74% AUC depending on dataset. Simplest model

2. You et al. Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks, AAAI Jan 2015

Proposes 3, deep CNN based models for image sentiment analysis. First a standard CNN that is trained on a Filcker sentiment, SentiBank, dataset for a baseline. Then train a CNN based model that adapts the training examples in second pass (similar to boosting) to beat the baseline in evaluation metrics. Then evaluates the network against a new dataset curated from 1.5k twitter images which shows how the model generalizes well. Then they introduce a section on transfer learning which I didn’t quite follow…

“Given the distinct nature of visual sentiment analysis and object recognition, the authors in [2] explore the possibility of designing a new architecture specific for the former task, training a network with 2 convolutional and 4 fully connected layers. However, there is very little rationale given for why they configured their network in this way except for the last two fully connected layers. “
— [3]

78.3% Accuracy

Dataset here:


3. Campos, Salvador Diving Deep into Sentiment: Understanding Fine tuned CNNs for Visual Sentiment Prediction, Aug 2015

They like the transfer learning idea, however, they don’t like the way [1] did it w/ an unmodified AlexNet+LogReg. They like the idea of how [2] did it, but they don’t like their Deep CNN arch as they dont’ explain/justify it.

Experiments w/ differnt types of transfer learning on AlexNet. See large improvements using dataset augmentation and suggests the best transfer learning arch on AlexNet is to replace the last FC layer w/ a 2 neuron layer and train that.

83% Accuracy


4. Campos, V., Jou, B., & Giro-i-Nieto, X. (2017, February). From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction. Image and Vision Computing.

Just a contineation of teh analysis in [3]

Permalink: a-review-of-deep-learning-models-for-image-sentiment-analysis


Last edited by Alex Egg, 2017-09-22 20:55:04
View Revision History