*The views expressed in this article are my own and based only on an initial read of publicly available descriptions of the iPhone X. The blog is for entertainment and informational. I am not a professional or expert in technology, but look forward to learning more about this technology in the near future.
Botox® and iPhones are very popular. Now these two phenomena will interface in an unusual way with Apple’s latest iPhone X. Among other technological features, the iPhone X has the ability with its front cameras and software to recognize its owner. The phone apparently not only examines the appearance of your face but also its shape, which then allows its owner to unlock the phone.
How does the iPhone X recognize your face?
Dropping the Touch ID system, the iPhone X uses a Face ID recognition system to unlock and use the phone. However, will the iPhone X still recognize its owner after he or she visits the plastic surgeon? Can facial plastic surgery change you significantly enough to affect the Face ID system of recognition? We don’t know for certain, as Apple hasn’t reported data specifically on this topic. It makes reasonable sense to be prepared and have a backup method to unlock and use your brand new phone!
From what we’ve gathered via simple online research, you can unlock your iPhone X just by looking at it screen. Apple recently released some limited information on how its Face ID system works:
“With a simple glance, Face ID securely unlocks iPhone X. It provides intuitive and secure authentication enabled by the TrueDepth camera system, which uses advanced technologies to accurately map the geometry of your face. Face ID confirms attention by detecting the direction of your gaze, then uses neural networks for matching and anti-spoofing so you can unlock your phone with a glance. Face ID automatically adapts to changes in your appearance, and carefully safeguards the privacy and security of your biometric data.”
According to Apple, there is an approximate probability of 1 in 1,000,000 that a random person could unlock your iPhone X. The probability of a false match is higher for twins and siblings, but Apple hasn’t provided statistics on these groups. Apple hasn’t provided on any information either on facial recognition after cosmetic surgery. Children under the age of 13 are tougher to accurately recognize with Face ID, according to Apple, since “distinct facial features may not have fully developed”. Most parents would disagree!
So how does Apple’s new phone recognize a person’s face? A mathematical representation is created of your face from its camera and software, accounting for variables such as hats, scarves, and glasses. According to the the paper:
“Once it confirms the presence of an attentive face, the TrueDepth camera projects and reads over 30,000 infrared dots to form a depth map of the face, along with a 2D infrared image. This data is used to create a sequence of 2D images and depth maps, which are digitally signed and sent to the Secure Enclave. To counter both digital and physical spoofs, the TrueDepth camera randomizes the sequence of 2D images and depth map captures, and projects a device-specific random pattern. A portion of the A11 Bionic chip’s neural engine—protected within the Secure Enclave—transforms this data into a mathematical representation and compares that representation to the enrolled facial data. This enrolled facial data is itself a mathematical representation of your face captured across a variety of poses.”
It’s reassuring to know that the iPhone X already has a method to account for natural changes that occur to one’s face over time. Wrinkles, dark circles, droopy skin, and blemishes develop gradually from sun damage and natural aging. Once you’ve initially registered your face with the iPhone X, Face ID will update its mathematical representation by taking new images from successful logins, then deleting older images to keep a current representation of its owner. According to the paper
“To improve unlock performance and keep pace with the natural changes of your face and look, Face ID augments its stored mathematical representation over time. Upon successful unlock, Face ID may use the newly calculated mathematical representation—if its quality is sufficient—for a finite number of additional unlocks before that data is discarded. Conversely, if Face ID fails to recognize you, but the match quality is higher than a certain threshold and you immediately follow the failure by entering your passcode, Face ID takes another capture and augments its enrolled Face ID data with the newly calculated mathematical representation. This new Face ID data is discarded after a finite number of unlocks and if you stop matching against it. These augmentation processes allow Face ID to keep up with dramatic changes in your facial hair or makeup use, while minimizing false acceptance.”
So, the Face ID facial recognition system (1) uses a three dimensional map in addition to analyzing the 2d image, and (2) learns over time to account for gradual changes in a person’s face.
Outline of central face region that might be utilized in facial recognition software. Before and after brow lift surgery by Dr. Chaboki.
Facial Plastic Surgery and the iPhone X
A three dimensional map of your face is generated and analyzed with the new iPhone X, so we speculate that procedures that add tissue volume (ex. cheek implants), reduce tissue volume (ex. buccal fat reduction), or lifts tissue (ex. lower eyelid surgery) may possibly affect its facial recognition system. For plastic surgery patients, procedures involving the center of the face are more likely to change the front view and subsequently may more likely affect facial recognition. The human brain processes facial recognition by looking at the central face. We suspect the following aesthetic treatments might be more likely than others to fool the Face ID of the iPhone X:
- facial fillers – Juvederm®, Restylane®, Belotero®
- buccal fat reduction
- liquid facelift
- cheek augmentation and lip enhancement
- face implants
- cheek and chin surgery
- rhinoplasty
- eyelid surgery
- fat transfer
- brow lift
We suspect that cosmetic surgery that affects the region outside the center of the face or views from the side are less likely to affect facial recognition systems, including possibly the Face ID system of the iPhone X. Procedures that may not affect facial recognition systems may include:
- Botox or Dysport®, unless sculpting the face with jaw reduction
- ear surgery
- neck liposuction
- Kybella®
- neck lift
- lower facelift
Does Plastic Surgery alter Facial Recognition of the iPhone X?
We don’t know and are only speculating based on the limited information that is publicly available.
Most cosmetic surgery patients prefer subtle, gradual changes. As a result, we suspect these patients shouldn’t have an issue with facial recognition. For those patients who go for dramatic plastic surgery, especially on multiple facial areas at the same time, then be prepared to possibly not be recognized by your new phone.
Before and after brow lift surgery by Dr. Chaboki.
Plastic surgery is considered challenging in the field of face recognition research. We could possibly explore these questions further in a more scientific method. The Face ID system or other methods of facial recognition may possibly be used in future products, so this question will continue to linger long after the iPhone X.
So…. it says it grabs the new photo of you if you successfully input the passcode. What if your partner or friend knew the passcode and put it in? Doesn’t that mean the phone will think you’re a new person and it will only unlock for the friend
Good question.
Based on information from the Apple website, the owner must initially setup of FaceID under settings: looking at camera, place your face inside a digital frame, then gently move your head in clockwise fashion to allow the phone to get a complete map of your face. Once registered, another person would realistically need the passcode in order to open the phone, since there is an approximate probability of 1 in 1,000,000 that a random person could unlock your iPhone X with their face alone.