How to detect eyes in a photo in Salesforce Apex

Any good photo app needs to start with identifying facial landmark features. This generally requires an AI trained in Deep Learning, and we all know how difficult and time consuming that can be. What if there was an easier way? We’ve got just the thing.

Our first task is to obtain our API client. This can be downloaded here, and then unzipped into our project folder.

Now that our client is accessible, we can invoke the function faceLocateWithLandmarks.

SwagFaceApi api = new SwagFaceApi();SwagClient client = api.getClient();// Configure API key authorization: ApikeyApiKeyAuth Apikey = (ApiKeyAuth) client.getAuthentication('Apikey');Apikey.setApiKey('YOUR API KEY');Map<String, Object> params = new Map<String, Object>{'imageFile' => Blob.valueOf('Sample text file\nContents')};try {// cross your fingersSwagFaceLocateWithLandmarksResponse result = api.faceLocateWithLandmarks(params);System.debug(result);} catch (Swagger.ApiException e) {// ...handle your exceptions}

Pretty easy so far. Let’s try out our function on this photo:

Image for post
Image for post

Here is the result.

{"Successful": true,"Faces": [{"LeftX": 567,"TopY": 400,"RightX": 782,"BottomY": 615,"LeftEyebrow": [{"X": 673,"Y": 425},{"X": 683,"Y": 418},{"X": 694,"Y": 415},{"X": 706,"Y": 416},{"X": 663,"Y": 462}],"RightEyebrow": [null,null,null,null,null],"LeftEye": [{"X": 611,"Y": 483},{"X": 624,"Y": 477},{"X": 637,"Y": 481},{"X": 628,"Y": 488},{"X": 614,"Y": 493},{"X": 680,"Y": 462}],"RightEye": [{"X": 687,"Y": 451},{"X": 697,"Y": 446},{"X": 706,"Y": 450},{"X": 702,"Y": 456},{"X": 692,"Y": 460},{"X": 663,"Y": 566}],"BottomAndSidesOfFace": [{"X": 566,"Y": 554},{"X": 580,"Y": 577},{"X": 600,"Y": 599},{"X": 625,"Y": 613},{"X": 652,"Y": 621},{"X": 680,"Y": 624},{"X": 708,"Y": 622},{"X": 731,"Y": 614},{"X": 744,"Y": 599},{"X": 748,"Y": 579},{"X": 749,"Y": 553},{"X": 746,"Y": 530},{"X": 744,"Y": 507},{"X": 739,"Y": 485},{"X": 730,"Y": 464},{"X": 717,"Y": 445},{"X": 571,"Y": 479}],"NoseBridge": [{"X": 672,"Y": 472},{"X": 682,"Y": 482},{"X": 693,"Y": 492},{"X": 675,"Y": 521}],"NoseBottom": [{"X": 687,"Y": 519},{"X": 698,"Y": 517},{"X": 704,"Y": 511},{"X": 709,"Y": 505},{"X": 600,"Y": 494}],"LipsInnerOutline": [{"X": 697,"Y": 544},{"X": 707,"Y": 541},{"X": 715,"Y": 537},{"X": 729,"Y": 539},{"X": 719,"Y": 559},{"X": 711,"Y": 564},{"X": 700,"Y": 566}],"LipsOuterOutline": [{"X": 678,"Y": 546},{"X": 694,"Y": 534},{"X": 705,"Y": 532},{"X": 712,"Y": 527},{"X": 725,"Y": 528},{"X": 734,"Y": 536},{"X": 733,"Y": 561},{"X": 725,"Y": 575},{"X": 716,"Y": 580},{"X": 705,"Y": 583},{"X": 686,"Y": 581},{"X": 670,"Y": 564}]}],"FaceCount": 1,"ErrorDetails": null}

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store