Verbal commands

[Pepere]

Does anyone use verbal commands with ATC and or aircraft functions? If so what program do you use?

Thanks

David

[Bjoern]

Interesting question. I used to have a Voice Buddy setup for FSX but never really used it. As long as you're willing to spend some time setting everything up, any voice control program can be trained to perform FSX' standard keyboard-controlled aircraft functions, including ATC.

I can't help you with what program will work best as I'm really out of the loop on this, so you'll have to do some research and experimenting yourself. Feeding Google with "FSX voice control", "FSX voice recognition" or similar might be a good start.

[Pepere]

Thanks. I have a couple of programs I'm going to try. They are demos. Thought I may try a CB mic wired through a usb head set. will be interesting to try.

David

[gecko65]

I've used It's Your Plane (IYP) a few years back. It now comes bundled as a co-pilot & verbal command ATC. Back when I used it the ATC could be separated out. It was a bit buggy back then, but I assume the interface has improved since I used it. I know the developer has continued to work on it quite a bit.

[Bjoern]

They are demos.

I wouldn't pay money for these unless it's specifically tailored to FSX like VOXATC.

Here's a list of programs. Try to find out which are free and which aren't and test the free ones.
http://en.wikipedia.org/wiki/List_of_speech_recognition_software#Add-ons_for_Windows_7_speech_recognition




-Edit:
I just had a bit of fun with Windows' default speech recognition function and FSX.
If you don't mind saying "Press F7" instead of "Flaps Down", this will be the easiest solution of them all.

[Bjoern]

Okay, here's a free solution for FSX.

http://library.avsim.net/search.php?SearchTerm=speechbuddy22.zip&CatID=root&Go=Search

It was made for FS9 and works in conjunction with Windows' default speech recognition engine. I'm running it with an unregistered FSUIPC in FSX and most, if not all commands like "Set Parking Brake" seem to work. Fine adjustment of trim, throttle, props etc...doesn't seem to work and needs to be done with the keyboard - unless you want to spend fifteen minutes mumbling "trim up" into a microphone. ;)
I haven't looked at extending functionality beyond the supported commands though.

[Pepere]

Thanks. will give these a try also. Been very busy getting the house ready for winter up here in Vermont......

David

[Bjoern]

You've kicked loose quite a bit of an avalance. I've spent last night testing out various free voice control programs and really was a little hoarse when I went to bed from repeating various commands.
I'm still making scripts for various tools and see how they respond in FSX.

Tools I've tested:

Windows Speech Recognition (WSR)
Ships with Windows and can be programmed with macros on a XML script basis.
The advantage is that it's totally free as it comes in Windows, that making a control script is fairly easy and that it supports repeated keystrokes. So if I say "Prop Pitch Increase 5", it tries to increase prop pitch by five percent in FSX, although any number beyond five or six triggers a much larger than necessary response in FSX. I might be able to tweak this a bit further, but no guarantees for that.
The disadvantage is that the documentation for the macros on the web is sparse at best. There is a WSR macro reference book, but it costs $10. Not gonna do that. Also, as it's the Windows Speech Recognition engine running, so you'll have to pick you commands carefully, otherwise they'll trigger one of the standard commands. RAM usage is a bit on the high side for my taste, with 48 MB for WSR and the macro editor which somehow has to run alongside to enable use of macros.

SpeechBuddy 2.2 (SB)
I've tested this a bit further and this expands a bit on my previous post about it.
SpeechBuddy's biggest adavantages are that it's configured for FSUIPC and accepts a lot of command variations. While all the other tools are entirely dependant on FSX' keyboard layout, this one directly tackles the internal simulator commands. So no matter what key you've mapped the "landing gear" function to in FSX, a "gear up" command will raise the gear of your aircraft. The opposite command will lower your landing gear. If the same function was performed with a WSR macro, saying "gear up" or "gear down" would be performed regardless of gear state as it would not check back for the actual state of the landing gear in FSX and merely do a virtual stroke of the "g" key. The command variations are another thing that are a plus for SB. Instead of "gear up" you can also say "toggle gear" or "raise gear" and SB will process it. Very natural compared to the other tools, but...
The disadvantage is, that all these variations for single commands makes it relatively prone to comprehension errors. Especially during the more dangerous phses of flight (treetop, landing), you don't really want to have SB perform the wrong action in your cockpit. Also, despite working in FSX, the SB configuration files for MSFS are from FS2002 times and only got updated for FS9. I've taken a look at the configuration file for FS9 and they are about 400 times more extensive and complicated than the scripts or macros for the other tools. Also, SB is dependant on FSUIPC, which is another hindrance to further investigation. I don't know if there were any bigger changes between FSUIPC 3 and 4 regarding control axis of if the SB's FSUIPC-internal license partially expired, but in any case, trying to fix up the configuration is likely to be wasted time. As SB runs on top of WSR, RAM usage for both is about 41 MB.

GlovePIE (GP)
GlovePIE is mostly used by people who want to control their PCs with their WiiMotes or Kinects or need to set up their HOTAS systems for FSX or IL-2 or DCS, but it also has pretty nifty voice recognition capabilities. These seem to be qute popular in the ArmA crowd to control their squadmates.
I really like its scripting language as it's quite flexible and really easy to pick up. I haven't found out yet whether or not I can perform stuff like "increase throttle 5" as with WSR with it, but I can tell GP to hold down a key until I say "Stop" or "Release". This means that I can hold the left brake to perform a parking spot turn or that I can smoothly advance the throttle to its TO/GA setting without having to do mental math at the same time. As GP brings its own speech recognition capabilities, it isn't dependant on WSR running in the background and thus only uses about 17 MB of RAM while avoiding any conflicts with standard commands for controlling Windows. So far, it seems like the perfect tool to control FSX aircraft by voice, if it wasn't for...
It's biggest disadvantage: Issues with key combinations and hold-downs like CTRL+F3 for prop pitch increase. That command is recognized and performed in FSX, but only once (unlike F3 for increasing the throttle). I don't know if it's a bug with GP, an inherent limitation between GP and Windows or simply an unfortunate script syntax, but in any case, it really breaks some functionality (think of the trim wheels or the heading bug). I hope that I can still work this out as it would make GP the best free solution for voice control.

Game Commander 2 (CG)
This is the tool I've played with the least.
Its biggest advantage is that it only takes up about 5 MB of RAM and that it just does what it's supposed to - convert voice commands into keystrokes. It can hold down keys until you say a command to release them, creating commands is entirely done within the user interface and you can train it to better recognize specific commands. So far, so good.
There is a wee problem though. The tool is payware from about thirteen years old and the company making it went belly-up a few years ago. So it's hard to find on the web and you need to make up your mind about whether abandonware (http://en.wikipedia.org/wiki/Abandonware) means fair game for everyone or not. Also, finding a keycode to unlock it requires you to tread carefully in the shadier part of the I still might give it a whirl in FSX and see if it does everything better than the other tools I've tried. If it does, well...it's gonna be a bit of a quagmire. Also command detection is a bit more unreliable than with the other tools at times.


In any case, it's clear that there's no perfect solution for the common cheapskate like me. I don't know if the various generic third-party payware voice control tools perform a job any better than the tools I've tested, but so far, it seems that a satisfactory voice control for FSX can be achieved without spending a dime.



- Edit:

HA! I think I've made a significant breakthrough regarding axis control in GlovePIE!

[Bjoern]

Oh David, what did you do to me?
I'm programming that control profile for GlovePIE and so far, I've managed to get all trim, prop, throttle and mixture controls principally working and I really just need to copy, paste and adapt a lot of code to all other relevant basic functions from FSX. I've got a boatload of work ahead of me, but I just...can't...stop.
The cockpit really comes alive if you say "Cowl Flaps Open" and the command gets recognized, executed and read back to you by Microsoft Anna. It's awesome!
(I've also implemented a "Shut up!" function for her for all you married people or die-hard soloists.)
Even the ATC window isn't much of a challenge (you still have to say "Option 1", "Option 2", etc... though, so no natural conversation).
If I get around to implementing Radar Contact functionality, I can maybe get a bit more creative with acceptable phrases.

GlovePIE's language is pretty straightforward and the scripts are relatively easy to read. I'm trying to document my progress and the script as well as I can, because some necessary adaptions will have to be made. (I'm pretty sure that everyone has different key assignments for FSX.)
Also, should I run out of steam on this, someone else can finish it.

Note though that only functions with key assignments will work, so you won't be able to control some custom cockpit switches.

[Pepere]

Bjoern

Sounds like fun. I will give that program a try. It may be a couple of days before I get into it, though.

Thanks for the update.

David

[Bjoern]

There's one thing curbing my enthusiasm a bit and that's memory usage. I haven't even implemented half of the keys from FSX and the script alone already takes up about 10 MB RAM. Ugh!
I might have to do some heavy streamlining here, like throwing out the selective readback feature.

I'm writing a user guide/manual for my script alongside, so if you bear with me a bit, I can at least put in the section where it explains what each part of the code does.

[Bjoern]

The script now incorporates all commands that I consider necessary for basic aircraft operation, including all commands for naval ops.
I've done a quick lap around the boat yesterday and it's quite a bit of fun to be able to to do everything by voice, including saying "Salute Catapult Officer" to launch from the deck and receive a rather badly modulated "Here we go!" as a response. :D

I still need to optimize a bit more though. The script reached over 1000 lines of code even after moving all comments to the manual and RAM use was about 43 MB while running. I then found out that there's a much more efficient way to control simple keystrokes that cuts the number of lines of code for a simple keystroke by 50%. I'm not done cutting down yet, but so far I'm back down to 770 lines and about 40 MB RAM.

If I can get ShadowPlay figured out, I might do a video demonstrating the voice commands in action on a carrier circuit.

Fascinatingly enough, the recognition quality is so good that it can even detect normally spoken commands with the microphone being about a meter away from my mouth at the end of an exhausting coding session.

As for the documentation, I still need to do the crackdown of the scripting language because you absolutely will have to alter some commands or keystrokes as my FSX key layout differs from the standard one in a few aspects (e.g. CTRL+SHIFT+SPACE for launch assist instead of the default SHIFT+I).

Might also be worth testing whether other languages are supported by merely changing the lnaguage of the input commands. As English is my primary computer language and everything is configured for that it'd be interesting to see (or hear) if GlovePIE also reacts to commands in German or French.

[Dutcheeseblend]

This is rather interesting!

So, Bjoern, if I understand, GP 'hears' a word, finds the right keystroke with this word and virtually hits the key combination.

Isn't there something possible that directly links the spoken wordt to an FSX variable? So no need to have the keystroke in between? Something with SimConnect?

[Bjoern]

Isn't there something possible that directly links the spoken wordt to an FSX variable? So no need to have the keystroke in between? Something with SimConnect?

Possible, yes, but you'd still need a tool as a bridge between your voice command and SimConnect. Something that continuously reads an external command list and uses the standard Windows Speech recognition to detect whether this command was spoken and then triggers the appropriate event in SimConnect and then FSX.

But this is not simple and efficient enough for my taste. The only real advantages that a SimConnect-based solution could offer would be capability for direct input "Mixture 50%", "Altimeter 30.09", etc... and being independent of the user's keyboard layout, but on the other hand, being confined to what is available in FSX is a plus for keystrokes. They're just much more understandable for the average user than a list filled with mysterious internal variables for FSX. Also, imagine that you could control every aspect of the aircraft by voice. You'd have quite some overlaps in command syntax and thus throw the voice recognition program into confusion. That's one thing that bugged me about SpeechBuddy, as it accepts different input combinations for a myriad of commands. "Increase Mixture" would be understood as "Mixture Full", "Mixture Full Lean" as "Mixture Decrease" and so on...
As for limited availability of keystrokes, I was a bit bummed out when I found out that the available keystrokes for lights in FSX are confined to all, landing, strobe and cockpit lights (unless I am blind and overlooked something in the list). But then again, it does make sense as the only lights you ever need to change during a flight are the strobe and landing lights (when entering and leaving the active runway and during climb and descent). And since voice control is just meant to assist during high-workload situations (takeoff, climb, approach, landing) and ease controlling a complex aircraft with nothing but a mouse, keyboard and flightstick instead of inner ears, butt and sense of touch, it just does the trick.
(Still, if anyone wants to try a user-configurable, efficient SimConnect-based solution, go ahead.)

[Bjoern]

Update from the voice-control front:
I'm done with optimizing, down from about 1000 to 640 lines of code and down from 43 to about 29 MB RAM use. Still less than WSR with macros, which is excellent.
GlovePIE also recognizes German and French.

Now for that video...

[Bjoern]

Aaaaand there it is.

Notes:
- Occasional use of strong language here and there. If you don't like to wash your ears with soap, don't click play!
- The language test feature will not be in the final script. This was just done to demonstrate input language capabilities
- Sound mixing isn't the best and I didn't want to yell in my apartment in the middle of the night, so I might be a bit too quiet throughout the video
- My carrier flying...well, let's not talk about it.


https://www.youtube.com/watch?v=qaWO7qrk01E

[Pepere]

That's cool. Are you going to make any of this available for us?

Thanks David

[Bjoern]

That's cool. Are you going to make any of this available for us?

Thanks David

Video comment: "Whole package out soon. Or not so soon. Who knows."

[Peter SWE]

I've been using "multi crew experience" for a couple of years. It has become essential to my flightsimming and i never fly without it.

http://www.multicrewxp.com/

[Pepere]

I've been using VoxATC. It's more for ATC than aircraft controls. You can use a co-pilot with the program I've not tried it yet. Just got it set up and did one IFR flight. I did good and it understands what you say a lot better than some of the other voice programs. A big plus is it also controls the AI traffic. I don't know how many times I've had to "Go Around" at JFK and or had aircraft fly right past me on final approach. So that will be the big test for me to fly into Kennedy with a Boeing 737-200 and fit into the air traffic. I'll give some updates if needed. Right now I am only using the demo. Will purchase if the AI traffic keep in line!!!!


David

[Bjoern]

VoxATC is a sensible purchase if you want to talk to ATC more naturally. For ATC itself, I find Radar Contact the better product.

[Pepere]

VoxATC is a sensible purchase if you want to talk to ATC more naturally. For ATC itself, I find Radar Contact the better product.


I cannot seem to find the demo for Radar Contact. I've done some flights in and out of jfk and VoxATC works fine for ATC. Trying "multi crew experience" not having very good luck with it. Would love to try Radar Contact but find the demo to down load. I'm directed to the RC site but cannot find the download button?

Right now VoxATC may be my pick. The down side so far with VoxAtc is the voice is not very clear.

David

[Bjoern]

There is no demo for RC4. It's a make-or-break purchase.

[DickB]

I've been using "multi crew experience" for a couple of years. It has become essential to my flightsimming and i never fly without it.

http://www.multicrewxp.com/

+1 (apparently I need to enter at least 10 characters so here they are)

[Bjoern]

FSXSVC released here:
http://www.sim-outhouse.com/sohforums/showthread.php?90441-FSX-Simple-Voice-Control&p=913729

[strykerpsg]

I've been using "multi crew experience" for a couple of years. It has become essential to my flightsimming and i never fly without it.

http://www.multicrewxp.com/

Thanks for sharing this tool and the link to it. As someone always looking for more immersion, this looks to fit the bill rather nicely.

I also want to see what Bjoern's come up with. I've never really thought much about voice commands, just macro switches linked to a joystick. You just have to love technology and what it can do for your entertainment. Thanks for putting your utility together Bjoern.

[Naismith]

Game (http://youtu.be/dzpd4lbfOo0) Commander 2 - had that years ago (FS98 ish) it worked after a fashion. Don't think it liked Scots/Canadian accent. :scotland::canada:

[Pepere]

VoxATC did not work out for me. It made to many mistakes in giving victors. Turn right to 200 when I was at 220 is not correct. Any rate I'm back to square ONE.

David

[Bjoern]

VoxATC did not work out for me. It made to many mistakes in giving victors. Turn right to 200 when I was at 220 is not correct. Any rate I'm back to square ONE.


Check my script then. Not as flexible as VoxATC, but still better than nothing. And free.