add the ability to count the beats with your finger through the camera and flash

[Emanuele Antonio Faraone]

add the ability to count the beats with your finger through the camera and flash, as in much
and other applications, the phone using the flash illuminates the finger, which must be attached to the camera and flash, and the camera is able to see the veins and to count their pulse rates in fairly specific

[Michal Predotka (mpredotka)]

Thanks for your bug report. This functionality is on my todo list.

[danjjl (danjjl)]

I was playing around with my Ubuntu Touch phone and python. I wrote a little scripts that calculates the heart rate from a video of my thumb captured on my phone.

For the moment it is a python script but it could probably be used as a base for a C++ equivalent code that could be used in Pulse.

The script and sample data is available at https://github.com/danjjl/pulse

The script can calculate the pulse using two different methods (FFT, spike detection). The script works like this:

 1. Calculate brightness on each frame (mean value over each pixel and RGB channel)
 2. Remove baseline fluctuations using the Asymmetric Least Squares Smoothing

    3a. FFT method
       * calculate FFT of signal
       * find frequency corresponding to maximum FFT coefficient
       * Heart rate is given by maximum FFT frequency * 60

    3b. Spike detector
       * count positive amplitude spikes (and calculate spike occurrence [spikes/minute])
       * count negative amplitude spikes (and calculate spike occurrence [spikes/minute])
       * Heart rate is given by the average of the positive and negative spike occurrence

The script processes the file `thumb.mp4` which is a video **taken without sound** from my phone with the flash on.

The current dependencies are imageio(mp4 processing), scipy, numpy (and matplotlib for figures)

Raw data is quite clean and results seem quite accurate (not validated in any way)

(for the moment I do not have time to convert this script to C++ - but I might find time in August)

[Michal Predotka (mpredotka)]

danjjl, many thanks for that! Unfortunately I have zero knowledge about C++. I was hoping it would be doable with just QML and JavaScript. TBH I haven't look into the problem properly yet. I need to finally find some time for that.

[danjjl (danjjl)]

It should be possible in JavaScript. The performance bottleneck of this problem is probably in converting the video images to a brightness value. It should be possible to speed up this step by taking lower quality images (resolution/bit-rate) or/and by reducing the frame-rate.

- I was using 1280 x 720 (6499 kbps) at 30fps
- 300bpm correspond to 5Hz -> 10fps should be enough
- Running the script on a down-sampled video of 1280 x 720 (958 kbps) at 10fps yields very similar results (should go through more thorough testing and films of finger just after effort (higher bpm))

[Emanuele Antonio Faraone (emanueleant03)]

Have you any idea?