Set Up ofxFacetracker2 on Arch Linux

Published on 2020-09-21 by Kenneth Flak

Back to Tech Research

Table of Contents

• Installing the Extensions
• Set up dlib
• Set up a Test Project

We have started playing around with the idea of using face tracking for our next performance, and the most immediately available tool for the job seems to be openFrameworks in general, and, more specifically, Jonas Jongejan's ofxFaceTracker2 extension. Originally written for Android and macOS, the extension does not work out of the box on Linux. Luckily Christopher Baker has forked it to work with Ubuntu-based distros, which gets us halfway to something that works on Arch Linux and derivatives.

After spending a day and a half of linking errors and bug hunting, I finally found a way to cover the last half of that stretch. This is a record of what I ended up with, mostly so that I will be able to perform the same procedure again in the future, and maybe somebody else will find it useful too.

ofxFaceTracker2 makes Davis King's machine learning toolkit dlib available for apps written with openFrameworks.

Installing the Extensions

First things first: clone ofxFaceTracker2 and ofxDlib into the openFrameworks addons folder:

cd ~/openframeworks/addons
git clone https://github.com/bakercp/ofxDlib.git
git clone https://github.com/bakercp/ofxFaceTracker2.git

I make the assumption that your oF root directory is ~/openframeworks. Change according to your setup.

Set up dlib

The next step is to pull in the required dependencies. cd to ofxDlib/scripts and delete the line

sudo apt-get -qq install -y cmake libblas-dev liblapack-dev libpng-dev libcairo2-dev libjpeg-dev libgif-dev

Instead, write this:

sudo pacman -S cmake blas lapack libpng cairo libjpeg-turbo libgif-dev

As far as I can tell, these are the Arch Linux equivalents to the necessary Ubuntu packages.

You will now be able to run the bootstrap script from the scripts folder without error:

~/openframeworks/addons/ofxDlib/scripts/bootstrap.sh

The final step of setting up ofxDlib is to edit the addon_config.mk file. On line 50 of that file, add cblas to the ADDON_PKG_CONFIG_LIBRARIES. The final result looks like this:

	ADDON_PKG_CONFIG_LIBRARIES += blas lapack cblas

Depending on your graphics card you probably want to enable cuda (for Nvidia cards) or MKL (Intel cards). My laptop has a modest, integrated Intel GPU, so I run:

sudo pacman -S intel-mkl

and uncomment line 56:

	ADDON_INCLUDES += /opt/intel/mkl/include

Set up a Test Project

That should hopefully be all you need to do to get dlib working. Now, let's test this in an app and see if it works. cd to ~/openframeworks/apps/myApps and run

projectGenerator --addons="ofxDlib, ofxFaceTracker2, ofxCv, ofxOpenCv" faceTrackerTest
cd faceTrackerTest

Edit src/ofApp.h to include the dependencies. Set up a ofxFaceTracker2 and a ofVideoGrabber:

#pragma once

#include "ofMain.h"
#include "ofxCv.h"
#include "ofxDlib.h"
#include "ofxOpenCv.h"
#include "ofxFaceTracker2.h"

class ofApp : public ofBaseApp{

    public:
        void setup();
        void update();
        void draw();

        void keyPressed(int key);
        void keyReleased(int key);
        void mouseMoved(int x, int y );
        void mouseDragged(int x, int y, int button);
        void mousePressed(int x, int y, int button);
        void mouseReleased(int x, int y, int button);
        void mouseEntered(int x, int y);
        void mouseExited(int x, int y);
        void windowResized(int w, int h);
        void dragEvent(ofDragInfo dragInfo);
        void gotMessage(ofMessage msg);

        ofxFaceTracker2 tracker;
        ofVideoGrabber grabber; 
};

Edit the implementation file, src/ofApp.cpp and plug in the appropriate code:

#include "ofApp.h"

//--------------------------------------------------------------
void ofApp::setup(){

    ofSetDataPathRoot(ofFile(__BASE_FILE__).getEnclosingDirectory()+"../bin/data/model/");
    // Setup grabber
    grabber.setup(1280,720);
    
    // Setup tracker
    tracker.setup();
}

//--------------------------------------------------------------
void ofApp::update(){
    grabber.update();
    
    // Update tracker when there are new frames
    if(grabber.isFrameNew()){
        tracker.update(grabber);
    }

}

//--------------------------------------------------------------
void ofApp::draw(){
    // Draw camera image
    grabber.draw(0, 0);
    
    // Draw tracker landmarks
    tracker.drawDebug();
    
    // Draw estimated 3d pose
    tracker.drawDebugPose();
    
    // Draw text UI
    ofDrawBitmapStringHighlight("Framerate : "+ofToString(ofGetFrameRate()), 10, 20);
    ofDrawBitmapStringHighlight("Tracker thread framerate : "+ofToString(tracker.getThreadFps()), 10, 40);
    
// #ifndef __OPTIMIZE__
    ofSetColor(ofColor::red);
    ofDrawBitmapString("Warning! Run this app in release mode to get proper performance!",10,60);
    ofSetColor(ofColor::white);
// #endif

}

And that's it!

Run make -j$(nproc) run from the root directory of your application and hopefully you will see the output of your video camera with some facial features drawn on top of it!