Discovery of Social Relationships in Consumer Photo Collections using Markov Logic

Parag Singla and Henry Kautz and Jiebo Luo and Andrew Gallagher


We identify the social relationships between individuals in consumer photos. Consumer photos generally do not contain a random gathering of strangers but rather groups of friends and families. Detecting and identifying these relationships are important steps towards understanding consumer image collections. Similar to the approach that a human might use, we use a rule-based system to quantify the domain knowledge (e.g. children tend to be photographed more often than adults; parents tend to appear with their kids). The weight of each rule reflects its importance in the overall prediction model. Learning and inference are based on a sound mathematical formulation using the theory developed in the area of statistical relational models. In particular, we use the language called Markov Logic [14]. We evaluate our model using cross validation on a set of about 4500 photos collected from 13 different users. Our experiments show the potential of our approach by improving the accuracy (as well as other statistical measures) over a set of two different relationship prediction tasks when compared with different baselines. We conclude with directions for future work.


Paper (PDF)