An iOS App for Automatic Calorie Estimation.
A food photo generally includes several kinds of food
dishes. In order to recognize food images including
multiple dishes, we need to detect each dish in food
images. Meanwhile, in recent years, the accuracy of
object detection has improved drastically by the
appearance of Convolutional Neural Network (CNN). In
this demo, we present an automatic calorie estimation
app, DeepCalorieCam, running on iOS. DeepCalorieCam can
estimate the calorie after detecting dishes from the
video stream captured from the built-in camera of an
iPhone. We use YOLOv2 [2] which is the state-of-the-art
object detection system using CNN, as a dish detector to
detect each dish in a food image, and the food calorie
of each detected dish are estimated by image-based food
calorie estimation [1].
We implemented the network proposed as a real-time object detection network called YOLOv2 by Redmon et al. [1] and the multi-task CNN proposed as a simultaneous estimation of food categories and calories calorie by Ege et al. [2].
First, we trained the dish detector by fine-tuning YOLOv2[1] with UECFOOD-100. Next, we trained food calorie estimation CNN by fine-tuning ResNet50 with food calorie estimation with 15 categories dataset[2]. After that, we converted the trained models for CoreML and deployed it to iOS.
[1]J. Redmon and A. Farhadi. YOLO9000: Better, Faster,
Stronger. In The IEEE Conference on Computer Vision
and Pattern Recognition (CVPR), 2017.
[2]T. Ege and K. Yanai. Simultaneous Estimation of
Food Categories and Calories with Multi-task CNN. In
Proc. of ACPR International Conference on Machine
Vision Applications (MVA), 2017.