PDA

View Full Version : FaceTracking with just your web-cam



robcap
November 3rd, 2010, 13:40
Hi all,
I had the opportunity to try this cool 22885 stuff:

a FREEWARE headtracking software, that only uses a webcam, and some software that has to be installed.

Please alow me to quote from their site:


The existing programs have proven their power and are used by many people around the world. These programs have also got a few shortcomings: they may cost you quite a bit and they require the users to wear some clip, hat or other device with LED's to make it work...

To make face-tracking available to everyone and eliminate the need for anything but your computer and a (simple) web-cam, we started making FaceTrackNoIR in May 2010 and by now, over 100 games/sims can be played with it.

New in version 1.4 is support for MS Flight Simulator 2002, 2004 and FSX, as well as support for some older games like CFS3.The webcam I use is an old Playstation 2 EYETOY (stole it from the kids:kilroy:), with windows drivers.

Just install the program, version 1.3, then install the update 4 over that, for FS compatibility.

Check their site:
http://facetracknoir.sourceforge.net/home/default.htm

Downloads here:
http://facetracknoir.sourceforge.net/information_links/download.htm

Give it a go, it's extremely easy to use, and, a DUTCH developement and implementation.
Let me know what you think of this!

Cheers, Rob

AndyE1976
November 3rd, 2010, 13:57
Excellent, I like trackir, but hate having to wear a hat for the track clip.

I downloaded the face API to try building something like this, but my c++ is a bit rusty. Will definitely have to try this

James
November 3rd, 2010, 14:13
So it's basically trackir, but for free?

HighGround22
November 3rd, 2010, 14:20
.
Naw, I surmise it's an adjunct to TrackIR, but it tracks your face, rather than the paraphernalia we must currently wear on our heads.

So if I have it right, with this software also running, you'd be able to ditch the clip thingies and just have TIR follow your face.

If !

robcap
November 3rd, 2010, 14:35
.
Naw, I surmise it's an adjunct to TrackIR, but it tracks your face, rather than the paraphernalia we must currently wear on our heads.
So if I have it right, with this software also running, you'd be able to ditch the clip thingies and just have TIR follow your face.
If !

Correct!
But without the Infra Red LED's as well. Just a webcam, the program, your face, ambient light (no need to sit in a darked out room). Easy as pie, 6 DOF (like trackIR) fully custumisable for sensitivity.

Absolutely brilliant.
Mind you, still under constant developement.

R.

anthony31
November 3rd, 2010, 15:02
Thanks for the link Rob.

I've thought about trackir but been put off by the price and the need to have a clip. If I had a webcam though I'd give this a try.

skyblazer3
November 3rd, 2010, 15:32
Will have to try this out.

I use FreeTrack http://www.free-track.net/english/ and am very pleased with it.

This would be interesting though. Has anyone had success with it in FSX?

empeck
November 3rd, 2010, 15:39
I've tried and it works in FSX, but it's very slow when compared to TrackIR 4 (this is the one I own), but maybe it's because my camera is very cheap one.

Bjoern
November 3rd, 2010, 16:30
I've tried and it works in FSX, but it's very slow when compared to TrackIR 4 (this is the one I own), but maybe it's because my camera is very cheap one.

Headtracking with webcams is ver dependent on the camera used. My self-built IR headtracking system is gathering dust becuase I never got around to buy a better cam (= 30 GPS steady) for it.

Wing_Z
November 3rd, 2010, 16:33
Darn good concept...I got it to work quite smoothly, but it disabled my joystick!

MHAircraft
November 3rd, 2010, 18:19
I wrote a little program to interface faceAPI and SimConnect sometime ago. The problem is that face tracking tends to be lower and not so reliable. I've given up and built IR markers to use with FreeTrack instead.

falcon409
November 3rd, 2010, 19:34
I installed it, and after an ippcore-5.3.dll error was corrected went into FSX to see what I had. Everything appears to connect correctly, but I get nothing in the game, no movement at all. The Manual is no help as it doesn't really tell you how to get it up and running by the numbers. . .it seems to just assume it will on it's own I guess. From there it gives you information on fine tuning, but nothing to suggest how to get started and what to do if it doesn't.:salute:

DaveKDEN
November 3rd, 2010, 20:06
Looks interesting. Still think a binocular cam set-up would work very well. Something using a modified Kinect or dual web-cams could vastly increase the accuracy of head tracking without having to wear an IR or reflective device on you head.

navychief8
November 4th, 2010, 16:40
This program is great. I just installed it today and only had to make a couple iif sensitivity tweeks and it seems to be working fine in FSX. I am running it on an HP laptop with Win 7 64 bit and a built in web cam. The only thing i dont like is when you shift to outside view it is still working and disables my hat switch side movements but allows up and down. Other than that it feels very natural when scanning in the virtual cockpit. Hopefully further developements will make this even better than it already is.:salute:

James
November 5th, 2010, 06:21
I just installed it, and on starting, it says : No DLLs were found in the Waterfall procedure-OK-The application was unable to start correctly (0xc0000142). Click OK to close the application...how do I fix this?

robcap
November 5th, 2010, 09:52
@ navychief8. That's exactly how I have it working.

@ James and falcon409, I have no further knowledge of the program. But on their site is a link for support. Try there. I'ms sure they will help. I will point the autor to this thread, maybe he can help.

@wing_z. indeed it disables the hat switch of the joystick. With banking camare, you can use both, so it can probably be fixed.

@anthony31, get a webcam ;-)

Cheers, Rob

cortomalteseit
November 5th, 2010, 10:40
interesting, even if i use track-ir....
what about frame-rate reductions?

gera
November 5th, 2010, 12:50
Sorry to say, this program is a total flop!!!, it has no instructions worth a nickel, the camera lights up shows your ugly face, a mask forms around it and bye bye guy!!!!!....I spent one whole hour fooling with this thing and "junked it" a few seconds ago.......I wonder why some people put out these " snake doctor medicine man" concoctions without really messing with them first....crazy, totally crazy....if it works for ya, I hope you enjoy it a lot........me?, I'll continue to keep my head in its place, secure.....:gameoff::barf:

crazysundog
November 5th, 2010, 13:08
Sorry to say, this program is a total flop!!!, it has no instructions worth a nickel, the camera lights up shows your ugly face, a mask forms around it and bye bye guy!!!!!....I spent one whole hour fooling with this thing and "junked it" a few seconds ago.......I wonder why some people put out these " snake doctor medicine man" concoctions without really messing with them first....crazy, totally crazy....if it works for ya, I hope you enjoy it a lot........me?, I'll continue to keep my head in its place, secure.....:gameoff::barf:

....

orionll
November 5th, 2010, 23:16
Seems like a good concept - getting it running was a breeze, just install and copy the update in place.

On the downside, the program used up quite a bit of my CPU and RAM while running, and the results weren't that spectacular. It tracked slowly, and didn't feel like it was translating my movements properly.

V4Friend
November 8th, 2010, 08:45
I just installed it, and on starting, it says : No DLLs were found in the Waterfall procedure-OK-The application was unable to start correctly (0xc0000142). Click OK to close the application...how do I fix this?Hello James,

Robcap asked me if I could join in, to answer some questions and help solve some 'misunderstandings'.

The error you get is listed in the 'known issues' section of the website. But, I'm happy to explain: The faceAPI uses the Intel IPP-library to do the facetracking math. This library is installed automatically with version 130 of FaceTrackNoIR, but unfortunately the installer (that came with faceAPI) does NOT recognize all CPU-types. If it goes wrong, the 'waterfall' error results.

To solve it, you should do the following:


un-install the IPP-library, via Control Panel/Add, remove software;
download the headtracking demo from SeeingMachines (here: http://www.seeingmachines.com/product/faceapi/downloads/). You may have to register, but no worries: it's free!;
install the demo: it will install the correct IPP-libraries for your CPU;
(optional) un-install the demo, the IPP-library will be left untouched by this;

I'm sorry for this somewhat tedious patch, but I have no other solution (yet). The faceAPI-people won't respond to any e-mails, because I use the non-commercial version.

Hope this helps!

V4Friend
November 8th, 2010, 09:12
I installed it, and after an ippcore-5.3.dll error was corrected went into FSX to see what I had. Everything appears to connect correctly, but I get nothing in the game, no movement at all. The Manual is no help as it doesn't really tell you how to get it up and running by the numbers. . .it seems to just assume it will on it's own I guess. From there it gives you information on fine tuning, but nothing to suggest how to get started and what to do if it doesn't.:salute:Hello falcon409,

OK, I assume you have installed v130 and then applied update 4 (you should see 1.4 in the window-title)? Then you can load the file SimConnect.ini (Menu File/Open) from the (default) Settings folder. If you do that, some default settings for all parameters and curves will be set and the game-protocol Simconnect will be chosen.

If you then press 'Start', the faceAPI-tracker will start and the video-display will show your face, with a yellow outline. You can minimize FaceTrackNoIR and start FSX. On my PC, I did not have to change any settings in FSX. I don't know if it's an issue, but in FS2004 tracking only works in the Virtual Cockpit.

Please let us know if it works and don't hesitate to ask again, if not :kilroy:

jhoopes2
November 9th, 2010, 06:59
Installed it, worked like a charm. Took a little bit to get the sensitivities adjusted, but man, I love the concept and the delivery...To me, works great (8.5 out of 10). nice work

My only issue is that its not allowing up/down, and side to side translational movement. Not completely crucial but it would be kinda nice. Any ideas? (Simply moving my head up, not tilting, but actual positioning of my head higher, does not raise the point of view, nor does moving side to side while keeping my face forward)...using FS9 on Windows 7 64-bit.

Thanks again...love this...and just in time as I got a new web cam the other day!

V4Friend
November 9th, 2010, 07:14
Installed it, worked like a charm. Took a little bit to get the sensitivities adjusted, but man, I love the concept and the delivery...To me, works great (8.5 out of 10). nice work

My only issue is that its not allowing up/down, and side to side translational movement. Not completely crucial but it would be kinda nice. Any ideas? (Simply moving my head up, not tilting, but actual positioning of my head higher, does not raise the point of view, nor does moving side to side while keeping my face forward)...using FS9 on Windows 7 64-bit.

Thanks again...love this...and just in time as I got a new web cam the other day!
Hello jhoopes2,

You are right about the translations (X and Y) not working in FS2004 (should I say FS9?): only the rotations and zoom are supported (now). Unfortunately that seems to be a limitation of FSUIPC, though it should still be possible... I just 'adapted' the way Free-track does it :wiggle:

I'm glad you like the program (which of course is not even 'ready') and thanks for providing such useful feedback!

Dangerousdave26
November 9th, 2010, 08:35
You are right about the translations (X and Y) not working in FS2004 (should I say FS9?): only the rotations and zoom are supported (now). Unfortunately that seems to be a limitation of FSUIPC, though it should still be possible... I just 'adapted' the way Free-track does it :wiggle:



That is the same issue with Free Track. The FSUIPC interface does not support 6DOF. However if you use the TIRviews.dll in free track you do get 6DOF.

I have not tried this software but I am going to give it a go on my laptop with the built in cam to see if it works. It looks very interesting.

V4Friend
November 9th, 2010, 12:16
That is the same issue with Free Track. The FSUIPC interface does not support 6DOF. However if you use the TIRviews.dll in free track you do get 6DOF.
Hello Dangerousdave26,

Well, you are right (I have the Free-track sources: they're a great source of inspiration. I like open-source :applause: ).

You could also use TIRViews with FaceTrackNoIR (copy TIRViews.dll in FaceTrackNoIR folder, select 'fake TrackIR', edit Settings (tick checkbox 'use TIRViews')). I have tried it with FS2004, but my PC can't handle the load. I really only implemented TIRViews for older games like CFS3 (and Over Flanders Fields).

Frankly, when you're sitting in the pilots seat, I don't think you really need X and Y-translations. Or you must like hopping like this one:
:jump:...
Thanks for your comment(s)!

BTW.: FaceTrackNoIR is open-source too...

AndyE1976
November 9th, 2010, 14:11
Finally got around to trying this and it works pretty well, the only thing that stops it being a TrackIR replacement for me is the lag between moving your head and getting a reaction in FSX. It's a bit like trying to fly drunk!

I tried maxing the sample rate and adjusting the curves to give more degrees in game per real movement, but although that helped a bit, there is still a delay before the view changes which makes it a real challenge to use.

I really like not having to wear a hat though.

V4Friend
May 14th, 2011, 04:01
Finally got around to trying this and it works pretty well, the only thing that stops it being a TrackIR replacement for me is the lag between moving your head and getting a reaction in FSX. It's a bit like trying to fly drunk!

I really like not having to wear a hat though.
Hi Andy and others,

Sorry to shake up this thread (so late), but I guess: better late than never? Since your last post, update 5 of FaceTrackNoIR was released in March. It contains a brand new filter-algorithm, which eliminates any 'sluggish' behaviour and still gives a stable view.

Work on update 6 is already in progress and that will have some nice improvements too! Check out the download (http://facetracknoir.sourceforge.net/information_links/download.htm) page and the news (http://facetracknoir.sourceforge.net/news/news.htm) page of the website. :salute:

Daube
May 14th, 2011, 05:32
I confirm the new patch has brought a lot of improvements.
But a good lighting and a decent webcam are still mandatory.
And it still works better when you smile (not kidding!) :icon_lol:

V4Friend
May 14th, 2011, 05:52
I confirm the new patch has brought a lot of improvements.
But a good lighting and a decent webcam are still mandatory.
And it still works better when you smile (not kidding!) :icon_lol:
That's why the whole project began: to make people have fun and smile!
I'm glad you like it...