# Measuring Your Heart Rate Using Your Phone’s Camera and Flutter

Published 8 April 2020 by Afonso Raposo.


In this article, I’ll explain how you can develop a simple app with Flutter that measures heart rate variability and displays it in a chart using only the phone’s camera and flash.

![Screenshot of the application](screenshot1.jpeg "Screenshot of the application.")


---


# **Concept**

You’ve probably seen or know of devices that people clip to their fingers in hospitals that measure their heart rate, or smartwatches capable of measuring your heart rate. They all have one thing in common: They measure the heart rate with a technique called photoplethysmography.

> *A photoplethysmogram (PPG) is an optically obtained plethysmogram that can be used to detect blood volume changes in the microvascular bed of tissue. — Wikipedia*

Shining a light into a blood irrigated tissue, we can measure the variability of reflected light and extract the variation of blood flow. As we all know, the blood flow is dependent on the heart rate, so we can calculate the heart rate using the blood flow variation.

![Image of finger over camera and resulting PPG signal](ppg.jpeg "Vandenberk, et. al. (2017). Clinical Validation of Heart Rate Apps: Mixed-Methods Evaluation Study. JMIR Mhealth Uhealth. 5. e129. [10.2196/mhealth.7254](https://doi.org/10.2196/mhealth.7254)")

So, in our application, we’ll shine the camera’s flash and measure the intensity reflected using the phone’s camera. More specifically, we’ll measure the average value of all the pixel’s intensity of the camera image. Then, if we cover the camera and flash with our finger, the intensity measured will vary with the blood flow.


---


# **Code**

## **Dependencies**

First, we need to install the dependencies:

- **charts_flutter** — Material Design data visualization library written natively in Dart.
- **wakelock** — This Flutter plugin allows you to enable and toggle the screen wakelock on Android and iOS, which prevents the screen from turning off automatically.
- **camera** — A Flutter plugin for iOS and Android allowing access to the device cameras.

```yaml
...
dependencies:
  flutter:
    sdk: flutter
  cupertino_icons: ^0.1.2
  charts_flutter: ^0.9.0
  wakelock: ^0.1.4+1
  camera: ^0.7.0+4

...
```


## **Application**

Our application’s interface is divided into three files: `main.dart`, `homePage.dart`, and `chart.dart`.

`main.dart` - Here we only need to set the HomePage widget as our home widget, so it displays when the application runs:

```dart
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'homePage.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'PPG',
      theme: ThemeData(
        brightness: Brightness.light,
      ),
      home: HomePage(),
    );
  }
}
```


`HomePage` - The core of the application is written on the `homePage.dart` file, which is our `HomePage` widget.

First, we need to create a `Scaffold` and, in its body, insert a centered `IconButton`, that will activate or deactivate the camera for the reading process.

```dart
import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:wakelock/wakelock.dart';
import 'chart.dart';

class HomePage extends StatefulWidget {
  @override
  HomePageView createState() {
    return HomePageView();
  }
}

class HomePageView extends State<HomePage> {
  bool _toggled = false;
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      backgroundColor: Colors.white,
      body: SafeArea(
        child: Center(
          child: IconButton(
            icon: Icon(_toggled ? Icons.favorite : Icons.favorite_border),
            color: Colors.red,
            iconSize: 128,
            onPressed: () {
              if (_toggled) {
                _untoggle();
              } else {
                _toggle();
              }
            },
          ),
        ),
      ),
    );
  }
}
```

As one can see, there is a `boolean` `_toggled` which stores the state of the `IconButton`. Now, we only need to define the functions `_toggle` and `_untoggle`.

For now, these functions will only change the value of the `_toggle` variable:

```dart
_toggle() {
  setState(() {
    _toggled = true;
  });
}

_untoggle() {
  setState(() {
    _toggled = false;
  });
}
```
<br>


![Screenshot of the app](untoggled.png "Screenshot of the untoggled and toggled iconButton, respectively.")

The next step, is dividing the screen into three equal parts, using three `Expanded` widgets inside a `Column`:

```dart
@override
Widget build(BuildContext context) {
  return Scaffold(
    backgroundColor: Colors.white,
    body: SafeArea(
      child: Column(
        children: <Widget>[
          Expanded(
            child: Container(
              color: Colors.red,
            ),
          ),
          Expanded(
            child: Center(
              child: IconButton(
                icon: Icon(_toggled ? Icons.favorite : Icons.favorite_border),
                color: Colors.red,
                iconSize: 128,
                onPressed: () {
                  if (_toggled) {
                    _untoggle();
                  } else {
                    _toggle();
                  }
                },
              ),
            ),
          ),
          Expanded(
            child: Container(
              color: Colors.black,
            ),
          ),
        ],
      ),
    ),
  );
}
```


![Screenshot of the app divided into 3 parts](3-parts.png "Screen divided into 3 equal parts.")


On the bottom `Container` we display the real-time chart, where the camera’s data will be displayed. A margin and round corners were also added to this `Container`.


```dart
import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:wakelock/wakelock.dart';
import 'chart.dart';

class HomePage extends StatefulWidget {
  @override
  HomePageView createState() {
    return HomePageView();
  }
}

class HomePageView extends State<HomePage> {
  bool _toggled = false;
  List<SensorValue> _data = [];

  ...

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      backgroundColor: Colors.white,
      body: SafeArea(
        child: Column(
          children: <Widget>[
            Expanded(
              child: Container(
                color: Colors.red,
              ),
            ),
            Expanded(
              child: Center(
                child: IconButton(
                  icon: Icon(_toggled ? Icons.favorite : Icons.favorite_border),
                  color: Colors.red,
                  iconSize: 128,
                  onPressed: () {
                    if (_toggled) {
                      _untoggle();
                    } else {
                      _toggle();
                    }
                  },
                ),
              ),
            ),
            Expanded(
              child: Container(
                margin: EdgeInsets.all(12),
                decoration: BoxDecoration(
                    borderRadius: BorderRadius.all(
                      Radius.circular(18),
                    ),
                    color: Colors.black),
                child: Chart(_data),
              ),
            ),
          ],
        ),
      ),
    );
  }
}
```


![Screenshot of the chart container](chart.png "Screenshot of the chart container.")

For the chart widget, we will use the package `charts_flutter`. The file `chart.dart` contains a `StatelessWidget` that displays in a chart points provided in a list. Each point is constituted by a `DateTime` value, which indicates the `x` value, and a `double` value that represents the `y` value.

```dart
import 'package:charts_flutter/flutter.dart' as charts;
import 'package:flutter/material.dart';

class Chart extends StatelessWidget {
  final List<SensorValue> _data;

  Chart(this._data);

  @override
  Widget build(BuildContext context) {
    return new charts.TimeSeriesChart([
      charts.Series<SensorValue, DateTime>(
        id: 'Values',
        colorFn: (_, __) => charts.MaterialPalette.green.shadeDefault,
        domainFn: (SensorValue values, _) => values.time,
        measureFn: (SensorValue values, _) => values.value,
        data: _data,
      )
    ],
        animate: false,
        primaryMeasureAxis: charts.NumericAxisSpec(
          tickProviderSpec:
              charts.BasicNumericTickProviderSpec(zeroBound: false),
          renderSpec: charts.NoneRenderSpec(),
        ),
        domainAxis: new charts.DateTimeAxisSpec(
            renderSpec: new charts.NoneRenderSpec()));
  }
}

class SensorValue {
  final DateTime time;
  final double value;

  SensorValue(this.time, this.value);
}
```

Regarding the upper `Container`, we wish to divide it into two halves. On the left one, will display the `CameraPreview` and, on the right half a `Text` containing the Beats Per Minute (BPM).

```dart
class HomePageView extends State<HomePage> {
  bool _toggled = false;
  List<SensorValue> _data = [];
  CameraController _controller;

  ...

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      backgroundColor: Colors.white,
      body: SafeArea(
        child: Column(
          children: <Widget>[
            Expanded(
              child: Row(
                children: <Widget>[
                  Expanded(
                    child: Center(
                      child: _controller == null
                          ? Container()
                          : CameraPreview(_controller),
                    ),
                  ),
                  Expanded(
                    child: Center(
                      child: Text("BPM"),
                    ),
                  ),
                ],
              ),
            ),
            Expanded(
              child: Center(
                child: IconButton(
                  icon: Icon(_toggled ? Icons.favorite : Icons.favorite_border),
                  color: Colors.red,
                  iconSize: 128,
                  onPressed: () {
                    if (_toggled) {
                      _untoggle();
                    } else {
                      _toggle();
                    }
                  },
                ),
              ),
            ),
            Expanded(
              child: Container(
                margin: EdgeInsets.all(12),
                decoration: BoxDecoration(
                    borderRadius: BorderRadius.all(
                      Radius.circular(18),
                    ),
                    color: Colors.black),
                child: Chart(_data),
              ),
            ),
          ],
        ),
      ),
    );
  }
}
```


![Screenshot of the app](layout.png "Screenshot of the app.")

The `CameraPreview` widget requires a valid `CameraController`. So we need to initialize the controller once we press the heart button and it’s toggled. Once the button is untoggled, we must also dispose of the controller:

```dart
Future<void> _initController() async {
  try {
    List _cameras = await availableCameras();
    _controller = CameraController(_cameras.first, ResolutionPreset.low);
    await _controller.initialize();
  } catch (Exception) {
    print(Exception);
  }
}

_disposeController() {
  _controller.dispose();
  _controller = null;
}
```

Now, we only need to integrate these functions in the functions that toggle the button.

```dart
_toggle() {
  _initController().then((onValue) {
    setState(() {
      _toggled = true;
    });
  });
}

_untoggle() {
  _disposeController();
  setState(() {
    _toggled = false;
  });
}
```

We should also override the dispose method and dispose of the `CameraController` once the application is disposed.

```dart
@override
void dispose() {
  _disposeController();
  super.dispose();
}
```

We must not forget to activate the camera’s flash and then start the `ImageStream`, which will provide the images that we will process. The function `_scanImage` will take care of it.

```dart
Future<void> _initController() async {
  try {
    List _cameras = await availableCameras();
    _controller = CameraController(_cameras.first, ResolutionPreset.low);
    await _controller.initialize();
    Future.delayed(Duration(milliseconds: 100)).then((onValue) {
        _controller.setFlashMode(FlashMode.torch);
    });
    _controller.startImageStream((CameraImage image) {
      _scanImage(image);
    });
  } catch (Exception) {
    print(Exception);
  }
}
```

The function `_scanImage` calculates the average of the camera image’s red channel and adds the value to the data list, which is displayed on the chart explained above. Notice that we limit the number of points of the data list to 50 values.

```dart
_scanImage(CameraImage image) {
  double _avg =
      image.planes.first.bytes.reduce((value, element) => value + element) /
          image.planes.first.bytes.length;
  if (_data.length >= 50) {
    _data.removeAt(0);
  }
  setState(() {
    _data.add(SensorValue(DateTime.now(), _avg));
  });
}
```

We do not need to process every frame, therefore, we can select a sampling rate. In this example, a sampling rate of 30 samples/seconds is used. For that purpose, we have a boolean, `_processing`, which becomes `true` once the `_scanImage` function is called and stays that way for 1/30 seconds, then returns to false. The `_scanImage` function will only be executed if the boolean _processing is false.

```dart
Future<void> _initController() async {
  try {
    List _cameras = await availableCameras();
    _controller = CameraController(_cameras.first, ResolutionPreset.low);
    await _controller.initialize();
    Future.delayed(Duration(milliseconds: 100)).then((onValue) {
        _controller.setFlashMode(FlashMode.torch);
    });
    _controller.startImageStream((CameraImage image) {
      if (!_processing) {
        setState(() {
          _processing = true;
        });
        _scanImage(image);
      }
    });
  } catch (Exception) {
    print(Exception);
  }
}

_scanImage(CameraImage image) {
  double _avg =
      image.planes.first.bytes.reduce((value, element) => value + element) /
          image.planes.first.bytes.length;
  if (_data.length >= 50) {
    _data.removeAt(0);
  }
  setState(() {
    _data.add(SensorValue(DateTime.now(), _avg));
  });
  Future.delayed(Duration(milliseconds: 1000 ~/ 30)).then((onValue) {
    setState(() {
      _processing = false;
    });
  });
}
```

For good practice, we should also set the `_processing` value to false every time we change the heart button’s toggled state.

```dart
_toggle() {
  _initController().then((onValue) {
    setState(() {
      _toggled = true;
      _processing = false;
    });
  });
}

_untoggle() {
  _disposeController();
  setState(() {
    _toggled = false;
    _processing = false;
  });
}
```

We now have an application that measures the blood flow volume variability and displays it in a chart. Now we only need to calculate the heart rate, which is the frequency of the plotted signal. I used a simple algorithm that measures the average and the max along our window data, sets the threshold to the mean of those values, and detects the peaks above that threshold. It then updates the BPM value with an attenuation coefficient so we don’t have abrupt changes.

```dart
class HomePageView extends State<HomePage> {
  ...

  double _alpha = 0.3;

  _toggle() {
    _initController().then((onValue) {
      setState(() {
        _toggled = true;
        _processing = false;
      });
      _updateBPM();
    });
  }

  ...

  _updateBPM() async {
    List<SensorValue> _values;
    double _avg;
    int _n;
    double _m;
    double _threshold;
    double _bpm;
    int _counter;
    int _previous;
    while (_toggled) {
      _values = List.from(_data);
      _avg = 0;
      _n = _values.length;
      _m = 0;
      _values.forEach((SensorValue value) {
        _avg += value.value / _n;
        if (value.value > _m) _m = value.value;
      });
      _threshold = (_m + _avg) / 2;
      _bpm = 0;
      _counter = 0;
      _previous = 0;
      for (int i = 1; i < _n; i++) {
        if (_values[i - 1].value < _threshold &&
            _values[i].value > _threshold) {
          if (_previous != 0) {
            _counter++;
            _bpm +=
                60000 / (_values[i].time.millisecondsSinceEpoch - _previous);
          }
          _previous = _values[i].time.millisecondsSinceEpoch;
        }
      }
      if (_counter > 0) {
        _bpm = _bpm / _counter;
        setState(() {
          _bpm = (1 - _alpha) * _bpm + _alpha * _bpm;
        });
      }
      await Future.delayed(Duration(milliseconds: (1000 * 50 / 30).round()));
    }
  }

  ...

}
```

The last thing we need to do is avoid the screen from turning off while the :

```dart
_toggle() {
  _initController().then((onValue) {
    Wakelock.enable();
    setState(() {
      _toggled = true;
      _processing = false;
    });
    _updateBPM();
  });
}

_untoggle() {
  _disposeController();
  Wakelock.disable();
  setState(() {
    _toggled = false;
    _processing = false;
  });
}
```

At last, we have an application that measures the blood volume variability and estimates the BPM.

The full code:

```dart
import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:wakelock/wakelock.dart';
import 'chart.dart';

class HomePage extends StatefulWidget {
  @override
  HomePageView createState() {
    return HomePageView();
  }
}

class HomePageView extends State<HomePage> {
  bool _toggled = false;
  bool _processing = false;
  List<SensorValue> _data = [];
  CameraController _controller;
  double _alpha = 0.3;
  int _bpm = 0;

  _toggle() {
    _initController().then((onValue) {
      Wakelock.enable();
      setState(() {
        _toggled = true;
        _processing = false;
      });
      _updateBPM();
    });
  }

  _untoggle() {
    _disposeController();
    Wakelock.disable();
    setState(() {
      _toggled = false;
      _processing = false;
    });
  }

  Future<void> _initController() async {
    try {
      List _cameras = await availableCameras();
      _controller = CameraController(_cameras.first, ResolutionPreset.low);
      await _controller.initialize();
      Future.delayed(Duration(milliseconds: 100)).then((onValue) {
        _controller.setFlashMode(FlashMode.torch);
      });
      _controller.startImageStream((CameraImage image) {
        if (!_processing) {
          setState(() {
            _processing = true;
          });
          _scanImage(image);
        }
      });
    } catch (Exception) {
      print(Exception);
    }
  }

  _updateBPM() async {
    List<SensorValue> _values;
    double _avg;
    int _n;
    double _m;
    double _threshold;
    double _bpm;
    int _counter;
    int _previous;
    while (_toggled) {
      _values = List.from(_data);
      _avg = 0;
      _n = _values.length;
      _m = 0;
      _values.forEach((SensorValue value) {
        _avg += value.value / _n;
        if (value.value > _m) _m = value.value;
      });
      _threshold = (_m + _avg) / 2;
      _bpm = 0;
      _counter = 0;
      _previous = 0;
      for (int i = 1; i < _n; i++) {
        if (_values[i - 1].value < _threshold &&
            _values[i].value > _threshold) {
          if (_previous != 0) {
            _counter++;
            _bpm +=
                60000 / (_values[i].time.millisecondsSinceEpoch - _previous);
          }
          _previous = _values[i].time.millisecondsSinceEpoch;
        }
      }
      if (_counter > 0) {
        _bpm = _bpm / _counter;
        setState(() {
          _bpm = (1 - _alpha) * _bpm + _alpha * _bpm;
        });
      }
      await Future.delayed(Duration(milliseconds: (1000 * 50 / 30).round()));
    }
  }

  _scanImage(CameraImage image) {
    double _avg =
        image.planes.first.bytes.reduce((value, element) => value + element) /
            image.planes.first.bytes.length;
    if (_data.length >= 50) {
      _data.removeAt(0);
    }
    setState(() {
      _data.add(SensorValue(DateTime.now(), _avg));
    });
    Future.delayed(Duration(milliseconds: 1000 ~/ 30)).then((onValue) {
      setState(() {
        _processing = false;
      });
    });
  }

  _disposeController() {
    _controller.dispose();
    _controller = null;
  }

  @override
  void dispose() {
    _disposeController();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      backgroundColor: Colors.white,
      body: SafeArea(
        child: Column(
          children: <Widget>[
            Expanded(
              child: Row(
                children: <Widget>[
                  Expanded(
                    child: Center(
                      child: _controller == null
                          ? Container()
                          : CameraPreview(_controller),
                    ),
                  ),
                  Expanded(
                    child: Center(
                      child: Text(
                        (_bpm > 30 && _bpm < 150 ? _bpm.round().toString() : "--"),
                        style: TextStyle(fontSize: 32, fontWeight: FontWeight.bold),
                      ),
                    ),
                  ),
                ],
              ),
            ),
            Expanded(
              child: Center(
                child: IconButton(
                  icon: Icon(_toggled ? Icons.favorite : Icons.favorite_border),
                  color: Colors.red,
                  iconSize: 128,
                  onPressed: () {
                    if (_toggled) {
                      _untoggle();
                    } else {
                      _toggle();
                    }
                  },
                ),
              ),
            ),
            Expanded(
              child: Container(
                margin: EdgeInsets.all(12),
                decoration: BoxDecoration(
                    borderRadius: BorderRadius.all(
                      Radius.circular(18),
                    ),
                    color: Colors.black),
                child: Chart(_data),
              ),
            ),
          ],
        ),
      ),
    );
  }
}
```

Here’s the working application:

![Gif of application measuing PPG signal](/images/ppg.gif "Application measuring the variation of blood flow on the index finger.")

The GitHub repository for this project can be found [here](https://github.com/Afonsocraposo/ppg)

**On my GitHub repository, you’ll notice that I updated the application to using a `Timer.periodic`, which presented better results. I also added more customization and an animation the heart-shaped button that simulates a beating heart. I didn’t include it in this tutorial since its not necessary.**
<br>