Face photo-sketch recognition

There exist several types of Face Recognition Systems (FRSs), with traditional FRSs typically operating on photographs taken in the visible light spectrum with a digital camera. Much research has recently been devoted to Heterogeneous Face Recognition (HFR), in which processing is performed using face images spanning different image modalities. One important use of such algorithms is in the comparison of mug-shot photographs with sketches obtained from eyewitness accounts of criminals, which has been described as perhaps the most challenging type of scenario in HFR since sketches often do not resemble the corresponding photograph. Apart from the difference in image appearance between photos and sketches, face sketches contain several differences in terms of shape and texture when compared to the corresponding facial photograph. Reasons for this include memory loss of witness, inaccurate descriptions (including exaggeration of any discriminant facial features) and the type of method used to depict the subject (hand-drawn sketches/software-generated sketches). All this leads to a large modality gap between the images to be compared, and in fact normal Commercial Off-the-Shelf (COTS) FRSs have been shown to perform poorly.

Work done in this area:

  1. Eigenpatches intra-modality algorithm and fusion of intra- and inter-modality algorithms, described in Fusion of Intra- and Inter-Modality Algorithms for Face-Sketch Recognition (paper)
  2. LGMS inter-modality algorithm, described in ‘Face Photo-Sketch Recognition with log-Gabor Filters and Statistical Correlation Coefficients’ (paper)
  3. UoM-SGFS: A viewed software-generated composite sketch database created using EFIT-V, described in ‘A Large-Software-Generated Face Composite Sketch Database’ (paper)
  4. The DEEP (face) Photo-Sketch System (DEEPS) framework, described in ‘Matching Software-Generated Sketches to Face Photos with a Very Deep CNN, Morphed Faces, and Transfer Learning’ (paper) and ‘Forensic Face Photo-Sketch Recognition Using a Deep Learning-Based Architecture’ (paper)

Link to Papers

Note:

Face photographs in main image taken from the Color FERET database:

National Institute of Standards and Technology (NIST), “The Color FERET
Database version 2,” last visited on Mar. 17, 2015. [Online]. Available:
http://www.nist.gov/itl/iad/ig/colorferet.cfm