A few years back, MIT released a paper describing a ridiculously cool method of measuring a person’s heart rate with only a camera. Every time your heart beats, blood vessels near the surface of the skin on your face dilate and slightly alter the absorption of incident light. The effect is too feignt to see with human eyes, but can be picked up on with a decent camera and signal processing techniques. An Iphone camera works just fine. The above video is a demo of some hopefully easy to use open source python software I wrote to do just that.. You’ll need openCV installed to try it out.

Here’s a general overview of the methods used:
1. Accurately track the user’s face frame-to-frame using a Haar Cascade Classifier
2. Take a spatial average of the RGB values near the center of the user’s face every frame
3. Apply independent component analysis to the RBG time series.
4. Take the FFT of the components and filter for signals within a reasonable range for a heart rate.
5. Pick the most prominent frequency.

Here’s a more detailed explanation, if you’re interested!

1.) Face Tracking
A Haar Cascade Classifier is a computationally fast feauture-based method of object detection, commonly used in computer vision. It slides a small window over an image and calculates a “cascade” of features inside the window with increasing complexity. To search for a face, for example, it might first check whether the upper region of the window is darker than the lower (which is common in faces because of the eyes), then check for things like bilateral symmetry, edge-features and center features (like the the nose). If enough relevant features are observed in the window, it detects the object. Haar Classifiers are trained with a large number of images to teach the algorithm what specific features to look for. I used the OpenCV implementation in this project.

2.) RGB Measurement
With the face located, isolate a region of pixels in the center of the face(near the nose/eyes/lower forehead) and take the mean of Red, Blue and Green values over the region. If you do this every frame frame, and the ambient lighting is good enough, you can get a signal that looks like this:

There’s almost always noise and other artifacts that occur due to movement, lighting changes, obstructions, ect. (see that random dip about two thirds of the way through?), but if you look closely you can see a constant, small periodic signal, particularly in the green channel. Those oscillations are heart beats.

3.) Independent Component Analysis
ICA is a type of blind signal separation. The RBG and green channels all carry information about heart-rate, but the signals may be mixed with noise and other sources in different ways. ICA is an attempt to pull out a clearer underlying heart-beat signal from the three channels. Without using ICA, the final heart-rate prediction was much less accurate and more susceptible to noise.

4 + 5.) Fast Fourier Transform (FFT)
Applying the FFT to the time series signal transforms the signal from the time domain to a frequency domain. All sorts of frequencies are contained within this signal due to the environment or even how the specific camera operates. Hopefully, most of those unwanted signals have frequencies beyond what would be reasonable for a human heart rate. After filtering all of them out, and only leaving behind frequencies from, say, 40BPM to 200BPM, you can see one prominent frequency. Ta-Da! That’s the heart beat. In the video, the “ICA Best Signal” panel is the output of the FFT, and you can usually see one tall, labeled peak.