安卓人脸检测
随着Google Play服务7.8的发布，Google引入了移动视觉API，可以实现脸部检测、条形码检测和文本检测。在本教程中，我们将开发一个安卓人脸检测应用程序，可以在图像中检测出人脸。
Android人脸识别
Android面部识别API通过使用眼睛、鼻子、耳朵、脸颊和嘴巴等一些标志物来跟踪照片和视频中的脸部。与检测单个特征不同，API会一次性检测整个脸部，然后再检测标志物和分类（如果定义了）。此外，该API还可以检测各种角度的脸部。
Android面部检测 – 特征点识别
地标是面部中的一个有趣的点。左眼、右眼和鼻子底部都是地标的例子。以下是目前可以通过API找到的地标。
- 左眼和右眼
左耳和右耳
左耳尖和右耳尖
鼻子的底部
左脸颊和右脸颊
嘴角的左边和右边
口腔的底部
当使用“左”和“右”时，它们是相对于对象的。例如，LEFT_EYE地标是对象的左眼，而不是在查看图像时位于左边的眼睛。
分类
分类确定某种面部特征是否存在。Android Face API目前支持两种分类。
- eyes open : getIsLeftEyeOpenProbability() and getIsRightEyeOpenProbability() method are used.
- smiling : getIsSmilingProbability() method is used.
脸部朝向
脸部的定位是使用欧拉角来确定的。这些欧拉角指的是脸部围绕X轴、Y轴和Z轴旋转的角度。欧拉角Y告诉我们脸部是朝左还是朝右看。
- Euler Z tells us if the face is rotated/slated
- Euler X tells us if the face is looking up or down (currently not supported)
Note: If a probability can’t be computed, it’s set to -1. Let’s jump into the business end of this tutorial. Our application shall contain a few sample images along with the functionality to capture your own image. Note: The API supports face detection only. Face Recognition isn’t available with the current Mobile Vision API.
Android人脸检测示例项目的结构
安卓人脸检测代码
在你的应用的build.gradle文件中添加以下依赖。
compile 'com.google.android.gms:play-services-vision:11.0.4'
请按照下面所示，在AndroidManifest.xml文件的application标签内添加以下元数据。
<meta-data
android:name="com.google.android.gms.vision.DEPENDENCIES"
android:value="face"/>
这样，Vision库就能知道你打算在应用程序中检测人脸。在AndroidManifest.xml文件的manifest标签中添加以下权限来获取相机权限。
<uses-feature
android:name="android.hardware.camera"
android:required="true"/>
<uses-permission
android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
下面提供了activity_main.xml布局文件的代码。
<?xml version="1.0" encoding="utf-8"?>
<ScrollView xmlns:android="https://schemas.android.com/apk/res/android"
xmlns:app="https://schemas.android.com/apk/res-auto"
xmlns:tools="https://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent">
<android.support.constraint.ConstraintLayout xmlns:app="https://schemas.android.com/apk/res-auto"
xmlns:tools="https://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="wrap_content"
tools:context="com.Olivia.facedetectionapi.MainActivity">
<ImageView
android:id="@+id/imageView"
android:layout_width="300dp"
android:layout_height="300dp"
android:layout_marginTop="8dp"
android:src="@drawable/sample_1"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toTopOf="parent" />
<Button
android:id="@+id/btnProcessNext"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginTop="8dp"
android:text="PROCESS NEXT"
app:layout_constraintHorizontal_bias="0.501"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toBottomOf="@+id/imageView" />
<ImageView
android:id="@+id/imgTakePic"
android:layout_width="250dp"
android:layout_height="250dp"
android:layout_marginTop="8dp"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toBottomOf="@+id/txtSampleDescription"
app:srcCompat="@android:drawable/ic_menu_camera" />
<Button
android:id="@+id/btnTakePicture"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginTop="8dp"
android:text="TAKE PICTURE"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toBottomOf="@+id/imgTakePic" />
<TextView
android:id="@+id/txtSampleDescription"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginBottom="8dp"
android:layout_marginTop="8dp"
android:gravity="center"
app:layout_constraintBottom_toTopOf="@+id/txtTakePicture"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toBottomOf="@+id/btnProcessNext"
app:layout_constraintVertical_bias="0.0" />
<TextView
android:id="@+id/txtTakePicture"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginTop="8dp"
android:gravity="center"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toBottomOf="@+id/btnTakePicture" />
</android.support.constraint.ConstraintLayout>
</ScrollView>
我们定义了两个 ImageView，TextView 和 Button。一个用于循环遍历示例图片并显示结果。另一个用于从相机捕获图片。下面给出了 MainActivity.java 文件的代码。
package com.Olivia.facedetectionapi;
import android.Manifest;
import android.content.Context;
import android.content.Intent;
import android.content.pm.PackageManager;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.net.Uri;
import android.os.Environment;
import android.provider.MediaStore;
import android.support.annotation.NonNull;
import android.support.v4.app.ActivityCompat;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.SparseArray;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;
import android.widget.Toast;
import com.google.android.gms.vision.Frame;
import com.google.android.gms.vision.face.Face;
import com.google.android.gms.vision.face.FaceDetector;
import com.google.android.gms.vision.face.Landmark;
import java.io.File;
import java.io.FileNotFoundException;
public class MainActivity extends AppCompatActivity implements View.OnClickListener {
ImageView imageView, imgTakePicture;
Button btnProcessNext, btnTakePicture;
TextView txtSampleDesc, txtTakenPicDesc;
private FaceDetector detector;
Bitmap editedBitmap;
int currentIndex = 0;
int[] imageArray;
private Uri imageUri;
private static final int REQUEST_WRITE_PERMISSION = 200;
private static final int CAMERA_REQUEST = 101;
private static final String SAVED_INSTANCE_URI = "uri";
private static final String SAVED_INSTANCE_BITMAP = "bitmap";
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
imageArray = new int[]{R.drawable.sample_1, R.drawable.sample_2, R.drawable.sample_3};
detector = new FaceDetector.Builder(getApplicationContext())
.setTrackingEnabled(false)
.setLandmarkType(FaceDetector.ALL_CLASSIFICATIONS)
.setClassificationType(FaceDetector.ALL_CLASSIFICATIONS)
.build();
initViews();
}
private void initViews() {
imageView = (ImageView) findViewById(R.id.imageView);
imgTakePicture = (ImageView) findViewById(R.id.imgTakePic);
btnProcessNext = (Button) findViewById(R.id.btnProcessNext);
btnTakePicture = (Button) findViewById(R.id.btnTakePicture);
txtSampleDesc = (TextView) findViewById(R.id.txtSampleDescription);
txtTakenPicDesc = (TextView) findViewById(R.id.txtTakePicture);
processImage(imageArray[currentIndex]);
currentIndex++;
btnProcessNext.setOnClickListener(this);
btnTakePicture.setOnClickListener(this);
imgTakePicture.setOnClickListener(this);
}
@Override
public void onClick(View v) {
switch (v.getId()) {
case R.id.btnProcessNext:
imageView.setImageResource(imageArray[currentIndex]);
processImage(imageArray[currentIndex]);
if (currentIndex == imageArray.length - 1)
currentIndex = 0;
else
currentIndex++;
break;
case R.id.btnTakePicture:
ActivityCompat.requestPermissions(MainActivity.this, new
String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE}, REQUEST_WRITE_PERMISSION);
break;
case R.id.imgTakePic:
ActivityCompat.requestPermissions(MainActivity.this, new
String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE}, REQUEST_WRITE_PERMISSION);
break;
}
}
@Override
public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
switch (requestCode) {
case REQUEST_WRITE_PERMISSION:
if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
startCamera();
} else {
Toast.makeText(getApplicationContext(), "Permission Denied!", Toast.LENGTH_SHORT).show();
}
}
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
if (requestCode == CAMERA_REQUEST && resultCode == RESULT_OK) {
launchMediaScanIntent();
try {
processCameraPicture();
} catch (Exception e) {
Toast.makeText(getApplicationContext(), "Failed to load Image", Toast.LENGTH_SHORT).show();
}
}
}
private void launchMediaScanIntent() {
Intent mediaScanIntent = new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE);
mediaScanIntent.setData(imageUri);
this.sendBroadcast(mediaScanIntent);
}
private void startCamera() {
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
File photo = new File(Environment.getExternalStorageDirectory(), "photo.jpg");
imageUri = Uri.fromFile(photo);
intent.putExtra(MediaStore.EXTRA_OUTPUT, imageUri);
startActivityForResult(intent, CAMERA_REQUEST);
}
@Override
protected void onSaveInstanceState(Bundle outState) {
if (imageUri != null) {
outState.putParcelable(SAVED_INSTANCE_BITMAP, editedBitmap);
outState.putString(SAVED_INSTANCE_URI, imageUri.toString());
}
super.onSaveInstanceState(outState);
}
private void processImage(int image) {
Bitmap bitmap = decodeBitmapImage(image);
if (detector.isOperational() && bitmap != null) {
editedBitmap = Bitmap.createBitmap(bitmap.getWidth(), bitmap
.getHeight(), bitmap.getConfig());
float scale = getResources().getDisplayMetrics().density;
Paint paint = new Paint(Paint.ANTI_ALIAS_FLAG);
paint.setColor(Color.GREEN);
paint.setTextSize((int) (16 * scale));
paint.setShadowLayer(1f, 0f, 1f, Color.WHITE);
paint.setStyle(Paint.Style.STROKE);
paint.setStrokeWidth(6f);
Canvas canvas = new Canvas(editedBitmap);
canvas.drawBitmap(bitmap, 0, 0, paint);
Frame frame = new Frame.Builder().setBitmap(editedBitmap).build();
SparseArray<Face> faces = detector.detect(frame);
txtSampleDesc.setText(null);
for (int index = 0; index < faces.size(); ++index) {
Face face = faces.valueAt(index);
canvas.drawRect(
face.getPosition().x,
face.getPosition().y,
face.getPosition().x + face.getWidth(),
face.getPosition().y + face.getHeight(), paint);
canvas.drawText("Face " + (index + 1), face.getPosition().x + face.getWidth(), face.getPosition().y + face.getHeight(), paint);
txtSampleDesc.setText(txtSampleDesc.getText() + "FACE " + (index + 1) + "\n");
txtSampleDesc.setText(txtSampleDesc.getText() + "Smile probability:" + " " + face.getIsSmilingProbability() + "\n");
txtSampleDesc.setText(txtSampleDesc.getText() + "Left Eye Is Open Probability: " + " " + face.getIsLeftEyeOpenProbability() + "\n");
txtSampleDesc.setText(txtSampleDesc.getText() + "Right Eye Is Open Probability: " + " " + face.getIsRightEyeOpenProbability() + "\n\n");
for (Landmark landmark : face.getLandmarks()) {
int cx = (int) (landmark.getPosition().x);
int cy = (int) (landmark.getPosition().y);
canvas.drawCircle(cx, cy, 8, paint);
}
}
if (faces.size() == 0) {
txtSampleDesc.setText("Scan Failed: Found nothing to scan");
} else {
imageView.setImageBitmap(editedBitmap);
txtSampleDesc.setText(txtSampleDesc.getText() + "No of Faces Detected: " + " " + String.valueOf(faces.size()));
}
} else {
txtSampleDesc.setText("Could not set up the detector!");
}
}
private Bitmap decodeBitmapImage(int image) {
int targetW = 300;
int targetH = 300;
BitmapFactory.Options bmOptions = new BitmapFactory.Options();
bmOptions.inJustDecodeBounds = true;
BitmapFactory.decodeResource(getResources(), image,
bmOptions);
int photoW = bmOptions.outWidth;
int photoH = bmOptions.outHeight;
int scaleFactor = Math.min(photoW / targetW, photoH / targetH);
bmOptions.inJustDecodeBounds = false;
bmOptions.inSampleSize = scaleFactor;
return BitmapFactory.decodeResource(getResources(), image,
bmOptions);
}
private void processCameraPicture() throws Exception {
Bitmap bitmap = decodeBitmapUri(this, imageUri);
if (detector.isOperational() && bitmap != null) {
editedBitmap = Bitmap.createBitmap(bitmap.getWidth(), bitmap
.getHeight(), bitmap.getConfig());
float scale = getResources().getDisplayMetrics().density;
Paint paint = new Paint(Paint.ANTI_ALIAS_FLAG);
paint.setColor(Color.GREEN);
paint.setTextSize((int) (16 * scale));
paint.setShadowLayer(1f, 0f, 1f, Color.WHITE);
paint.setStyle(Paint.Style.STROKE);
paint.setStrokeWidth(6f);
Canvas canvas = new Canvas(editedBitmap);
canvas.drawBitmap(bitmap, 0, 0, paint);
Frame frame = new Frame.Builder().setBitmap(editedBitmap).build();
SparseArray<Face> faces = detector.detect(frame);
txtTakenPicDesc.setText(null);
for (int index = 0; index < faces.size(); ++index) {
Face face = faces.valueAt(index);
canvas.drawRect(
face.getPosition().x,
face.getPosition().y,
face.getPosition().x + face.getWidth(),
face.getPosition().y + face.getHeight(), paint);
canvas.drawText("Face " + (index + 1), face.getPosition().x + face.getWidth(), face.getPosition().y + face.getHeight(), paint);
txtTakenPicDesc.setText("FACE " + (index + 1) + "\n");
txtTakenPicDesc.setText(txtTakenPicDesc.getText() + "Smile probability:" + " " + face.getIsSmilingProbability() + "\n");
txtTakenPicDesc.setText(txtTakenPicDesc.getText() + "Left Eye Is Open Probability: " + " " + face.getIsLeftEyeOpenProbability() + "\n");
txtTakenPicDesc.setText(txtTakenPicDesc.getText() + "Right Eye Is Open Probability: " + " " + face.getIsRightEyeOpenProbability() + "\n\n");
for (Landmark landmark : face.getLandmarks()) {
int cx = (int) (landmark.getPosition().x);
int cy = (int) (landmark.getPosition().y);
canvas.drawCircle(cx, cy, 8, paint);
}
}
if (faces.size() == 0) {
txtTakenPicDesc.setText("Scan Failed: Found nothing to scan");
} else {
imgTakePicture.setImageBitmap(editedBitmap);
txtTakenPicDesc.setText(txtTakenPicDesc.getText() + "No of Faces Detected: " + " " + String.valueOf(faces.size()));
}
} else {
txtTakenPicDesc.setText("Could not set up the detector!");
}
}
private Bitmap decodeBitmapUri(Context ctx, Uri uri) throws FileNotFoundException {
int targetW = 300;
int targetH = 300;
BitmapFactory.Options bmOptions = new BitmapFactory.Options();
bmOptions.inJustDecodeBounds = true;
BitmapFactory.decodeStream(ctx.getContentResolver().openInputStream(uri), null, bmOptions);
int photoW = bmOptions.outWidth;
int photoH = bmOptions.outHeight;
int scaleFactor = Math.min(photoW / targetW, photoH / targetH);
bmOptions.inJustDecodeBounds = false;
bmOptions.inSampleSize = scaleFactor;
return BitmapFactory.decodeStream(ctx.getContentResolver()
.openInputStream(uri), null, bmOptions);
}
@Override
protected void onDestroy() {
super.onDestroy();
detector.release();
}
}
从上述代码中得出的一些推论是：
- imageArray holds the sample images that’ll be scanned for faces when the “PROCESS NEXT” button is clicked.
-
The detector is instantiated with the below code snippet:
FaceDetector detector = new FaceDetector.Builder( getContext() )
.setTrackingEnabled(false)
.setLandmarkType(FaceDetector.ALL_LANDMARKS)
.setMode(FaceDetector.FAST_MODE)
.build();Landmarks add up to the computation time, hence they need to be explicitly set. Face Detector can be set to FAST_MODE or ACCURATE_MODE as per our requirements. We’ve set tracking to false in the above code since we’re dealing with still images. It can be set to true for detecting faces in a video.
- processImage() and processCameraPicture() methods contain the code where we actually detect the faces and draw a rectangle over them
- detector.isOperational() is used to check whether the current Google Play Services library in your phone supports the vision API(If it doesn’t Google Play downloads the required native libraries to allow support).
-
The code snippet that actually does the work of face detection is :
Frame frame = new Frame.Builder().setBitmap(editedBitmap).build();
SparseArray faces = detector.detect(frame); - Once detected, we loop through the faces array to find the position and attributes of each face.
- The attributes for each face are appended in the TextView beneath the button.
- This works the same when an image is captured by camera except the fact that we need to ask for the camera permissions at runtime and save the uri, bitmap returned by the camera application.
