Documentation :: Recognition How-To

How Face Recognition works
Before Recognition: raining the face index
Performing Face Recognition
Limitations and Privacy
Tips for improving face recognition

How Face Recognition works

Face.com's face recognition is service which allows computers to analyze facial information found in photos, and attempt to identify faces against a known set of users. We target the hardest problem in this field: identifying 'faces in the crowd', appearing in everyday photos, and match them against a large set of known faces. The technology we're offering through our APIs scales to many users, and was built with consumer applications in mind: fun, productivity, and privacy.

The process of face recognition requires an index of known faces before recognition can be performed. This process is also called 'training', and it works by adding face tags of known users to the index (or training-set), and then processing these photos to create a new index entry. More saved tags per user means higher accuracy. Once you've got your users all set up in the index, you can search for them in photos.

The face.com index supports users of known social networks as well as private namespaces. We currently support facebook and twitter user indexing, which are shared across all API users, and include data access and privacy enforcements using valid user credentials. With Facebook and Twitter user IDs, only access to friends of the currently logged-in user is permitted. Private namespaces are suitable for building an application-specific index of faces who do not necessarily have exiting profiles in facebook or twitter, and are not be shared with other users of the APIs. Training and enforcement of privacy policies in Private namespaces is entirely up to the developer, and must correspond to our Terms.

Before Recognition: training the face.com index

Overview

Before Face Recognition can be used, you'll need to train the face.com index:

Call faces.detect on photos of the user you want to add to the index
Call tags.save with the temporary tag IDs returned by faces.detect, along with the relevant user ID
Call faces.train with the relevant user ID(s) to update the index.

Note: You can skip steps 1 and 2 for most facebook users, see next section.

Training Facebook users index

When working with facebook users, you can save time by calling faces.train on one or more user IDs. Using the user credentials you provide us, we will automatically fetch recent tagging data from facebook, and apply detection + tags.save before calling the training procedure. Please note that not all users in facebook have tagged photo data, and that some users choose to block access to this data. As with all facebook and twitter index usage, you must provide valid credentials of the currently logged-in user in order to access the user's friends information, including photo tags.

Note: When using detection and training for facebook users, you are not limited to facebook photos. You may provide tagging information for facebook users from ANY photo. For instance, you may detect and save faces that appear in other photo sharing services such as flickr, and apply those to a facebook user's index.

Training Twitter or Private Namespace index

For Twitter and Private user namespaces, follow steps 1-3 in the overview.

Index Status

At any time, you may call faces.status to get an updated view of the index information. This will allow you to tune and improve results as required by your application.

Removing bad tags

During the life cycle of your application you may encounter wrong tagging information, and possibly support a way for users to correct tags. For this purpose, use the tags.remove method with the relevant tag ID, and then follow with a faces.train call to update the index.

Temporary Tag IDs VS. Permanent Tag IDs

Every response of faces.detect and faces.recognize includes a list of detected face tags. Each face tag is identified with a temporary tag ID (tag is not confirmed, i.e. confirmed: false, and tid value is prefixed with 'TEMP_') that can be used in calls to tags.save. Temporary tag IDs are stored in our cache for a limited session time, and shouldn't be used as a reference for future calls other than tags.save. Once tags are saved, a new permanent tag ID is produced, the tag's 'confirmed' status changes to 'true', and the tag id can be saved for future reference.

Performing Face Recognition

Once you've got your users all trained up in the index, you can start sending faces.recognize calls for those users.

All calls to faces.recognize must include the list of users you're looking for within the passed photos.

In addition to providing specific UID(s) for search, we support special user lists which you can pass as well. These include:

friends@facebook.com - facebook friends of the currently logged-in user
all@namespace.com - all-users in a private namespace
*<user>@namespace - all users tagged-by <user> - this is a great way to automatically keep the recognition search list focused only on what was used by the current user. For this to work with private namespaces, you can use the extended form of tags.save to specify the 'tagger ID' (ID of the user performing the tagging).

Limitations & Privacy

Recognizing 1000's of users is supported, but will take more time to complete, and may result in a higher number of errors. It is recommended that you provide a high-relevancy list of users where possible.
When passing URLs to faces.detect and faces.recognize, make sure these URLs are available for public download. If your photos are placed in a private location, either POST the photos or use temporary public URLs for the duration of the calls.
The Maximum width or height of a photo is 900 pixels. You may POST or send links to higher-size photos, but those will be resized internally for better performance.
The Maximum image file size we currently accept is 5MB. Please contact us if this limit prevents you developing the application you want.
When passing facebook or twitter user IDs, you'll need to pass the current users' credentials, and for facebook connect apps this also means you'll need to set your facebook's API key and secret. We'll use these credentials to authenticate the request, make sure friend relationships are in place, and that the user IDs are accessible to the signed-in user. See Auth section for more details.
When passing your own domain's user IDs, there's no need to provide additional credentials. However, please make sure user's privacy is maintained (see our terms).

Tips for improving face recognition

The quality of recognition results is influenced by the index quality, the search input, and interpreting the results in a manner most suitable for your service:

Training Set - a single face tag is sufficient to support training and recognition, but the more face tags a user has, the better accuracy gets for that user. We recommend you use 10 or more tags for best accuracy, especially if recognizing against a large set of users (1000+ friends). Use tags.save and faces.train regularly, possibly as part of a tagging or confirmation UX of your service.
Number of UIDs per call - although the percentage of false positives remains the same, the actual number of mistakes is expected to increase as you increase the number of UIDs per search. Where possible, passing higher relevancy users will or the *user@namespace notation can result in higher accuracy and faster response times.
Confidence level - when performing recognitions, you get multiple results per detected face, and may select either the top selection or multiple choices. With each detected face tag, we also provide a threshold score, which can be used to screen out low-confidence scores. Note that the confidence threshold we provide is specific to the call you made, and may change if, for instance, you change the number of UIDs in your search.
Photo Quality - the face.com can handle any JPG photo, and was built to handle web resolutions and everyday conditions. That said, extremely low resolution, lighting, smearing, focus, and other issues may result in lower accuracy.