Programming C, C++, Java, PHP, Ruby, Turing, VB
Computer Science Canada 
Programming C, C++, Java, PHP, Ruby, Turing, VB  

Username:   Password: 
 RegisterRegister   
 Voice-controlled personal assistant
Index -> General Discussion
Goto page 1, 2  Next
View previous topic Printable versionDownload TopicSubscribe to this topicPrivate MessagesRefresh page View next topic
Author Message
SNIPERDUDE




PostPosted: Mon Nov 12, 2012 3:21 pm   Post subject: Voice-controlled personal assistant

PRELIMINARY
I know they have been done before, but I don't like the ones out there. As a long-term personal project (to help kill boredom, and create something cool) I started (a while back) a voice command system inspired by (of course) Star Trek. I loved how they were able to talk to the computers (without using memorised commands) and the computers could not only understand them, but also intelligently interprets that information (not redirecting you to google or wikipedia).

I already have the foundation for this project, now it comes down to adding ideas (commands). I am also creating a USB driver to control any appliance I wish to connect (ex: TV remote), so there is also the capability for hardware too.

QUESTION
Any ideas for commands/abilities? Do not limit your ideas on what you believe to be possible, for such a thing is what cripples innovation.
Sponsor
Sponsor
Sponsor
sponsor
Tony




PostPosted: Mon Nov 12, 2012 4:15 pm   Post subject: Re: Voice-controlled personal assistant

I'm a bit confused
SNIPERDUDE @ Mon Nov 12, 2012 3:21 pm wrote:
...(without using memorised commands)...

but
SNIPERDUDE @ Mon Nov 12, 2012 3:21 pm wrote:
...now it comes down to adding ideas (commands)...


Also, how is this different from something like Siri?
Latest from compsci.ca/blog: Tony's programming blog. DWITE - a programming contest.
Insectoid




PostPosted: Mon Nov 12, 2012 4:15 pm   Post subject: RE:Voice-controlled personal assistant

What bothers me most about voice control is the response it gives you. You're constantly reminded that you're talking to a machine. "I'm sorry, I didn't quite catch that" said the perfectly polite computer, every single time. I'd like for it to say "Hey, can you speak a little more clearly?" and even get agitated when it has to keep asking. "Speak up! How many times do I have to tell you?". "Oh, THAT's what you meant! Right away!".
mirhagk




PostPosted: Mon Nov 12, 2012 5:59 pm   Post subject: RE:Voice-controlled personal assistant

I am confused, you said you didn't want to limit it to memorized commands and yet you are doing it by adding new commands?

If you wanted to truly build something that could interpret anything a human said, I'd say apply Jeff Hawkins theories to sentence structure and build a machine that can read and understand text first. That is NOT an easy task, but it'd be required in order to have this not just use memorized commands.

English is way too difficult of a language but lojban has always intrigued me for this very purpose. With lojban you could forget about syntax and ambiguities and just worry about what a sentence means. Wouldn't be an easy task by any means though.
SNIPERDUDE




PostPosted: Mon Nov 12, 2012 10:28 pm   Post subject: Re: Voice-controlled personal assistant

Tony wrote:
Also, how is this different from something like Siri?

1) This project was started just over a year ago, SIRI did not exist (on the mass market at any rate)
2) I don't use any Apple products, so I can't make direct comparisons
3) Since mine will be on a central server in my house, I can alot it much more power (mobile devices can only do so much)
4) It's not being developed for distribution, but making my life easier. Since the only target consumer is myself, I can better make it fit my own needs.

Insectoid wrote:
What bothers me most about voice control is the response it gives you.

Agreed. Though I'm sure everyone hates this. I've been researching how the voice recognition will work, which includes varying dialects (just a handful). And since voice recognition software adapts to the user, I'm not worried since the primary user will be myself.

mirhagk wrote:
I am confused, you said you didn't want to limit it to memorized commands and yet you are doing it by adding new commands?

I could have phrased that better. What I mean is: What abilities (not exact command strings) would be cool/useful.

mirhagk wrote:
If you wanted to truly build something that could interpret anything a human said, I'd say apply Jeff Hawkins theories to sentence structure and build a machine that can read and understand text first. That is NOT an easy task, but it'd be required in order to have this not just use memorized commands.

As I am not trying to do anything of that extremity, it will first get the input (the command sentence) from the user, match it against the commands in the database (the commands listed have synonymous variations, different ways of saying the same thing), then acted upon. If no match is found in the database, it uses matched keywords to figure out the interpretation. Command structures use varying word categorizations to create wiggle room (ex: discarded words like "please", optional words, etc).

It is not to interpret sentences at a degree where even translations could be made, but the stucture I have made works wonderfully for understanding me. An example of this in effect is how many ways (while maintaining comprehension) can you ask for the time? What time is it? What is the time? Time? Do you have the time? What's the time?

Anyway right now I'm looking for what can be done with commands.
Insectoid




PostPosted: Mon Nov 12, 2012 10:50 pm   Post subject: RE:Voice-controlled personal assistant

Quote:
3) Since mine will be on a central server in my house, I can alot it much more power (mobile devices can only do so much)
Siri is run on a server.
mirhagk




PostPosted: Mon Nov 12, 2012 11:01 pm   Post subject: RE:Voice-controlled personal assistant

hmm.... that will be a very difficult task. Most of the best voice control systems can't understand what people are saying 90% of the time. What they do is filter the input and look for a match against a very small set of commands.

You're trying to read in raw speech, convert to text, and then find the closest command for what was interpreted? I see 2 major sections where something can go wrong, the machine could mishear something, and misunderstand something.

I'm assuming you're going to have an active keyword to listen for (like the xbox does), because you need to make sure that active keyword won't come up in normal conversation. Otherwise you'll VERY often have the system hear the keyword, and carry out some command that you didn't even come close to saying.

This may work on a very small scale in a controlled environment, but once commands start becoming similar (tv on, lights on) you'll seem some really bizarre effects.
SNIPERDUDE




PostPosted: Mon Nov 12, 2012 11:17 pm   Post subject: Re: RE:Voice-controlled personal assistant

Insectoid @ November 12th 2012, 10:50 pm wrote:
Quote:
3) Since mine will be on a central server in my house, I can alot it much more power (mobile devices can only do so much)
Siri is run on a server.

1) As I said, I can't make comparisons.
2) What I meant by that statement is since it is all run on a computer (also acting as a server) I have more power available to me on the 'client-side'. Ex: access to better hardware, USB ports (see bottom for example), etc
3) I am not trying to do anything ground-breaking with how it runs, I just want to create that Star Trek type computer for my home. Therefore it must understand what I am saying, trying to say, and have that ubiquitous feel. Since I am making it myself, I can do more than just check my email and add stuff to calendars (etc). The most important aspects I suppose is the ubiquity -- integration into my home, and power -- I shouldn't set a limit in my head of what I can do with it, I should just be able to ask it whatever comes to mind.

Anyway this has deviated quite a bit from my posted question. Though questions may arise in how it functions, that is not the focus. My question boils down to:
what would you want a system like this be able to do for you? Dream big.

examples:
when's the next bus?
who called?
have it turn on the TV and change to a show I like when I walk in (uses a schedule to check what's on, compares it to the list of shows you like, uses the plugged in TV remote to turn to that channel)
^ TV remote plugged in through USB, mounted and fixed toward the TV.
Sponsor
Sponsor
Sponsor
sponsor
QuantumPhysics




PostPosted: Tue Nov 13, 2012 9:47 am   Post subject: RE:Voice-controlled personal assistant

The third example seems impossible with a computer, the only possible way to do it is with a webcam, that being said its still insanely difficult. Do you have a motion detector and good electrical engineering skills?
SNIPERDUDE




PostPosted: Tue Nov 13, 2012 10:44 am   Post subject: Re: RE:Voice-controlled personal assistant

QuantumPhysics @ November 13th 2012, 9:47 am wrote:
The third example seems impossible with a computer, the only possible way to do it is with a webcam, that being said its still insanely difficult. Do you have a motion detector and good electrical engineering skills?

Yes, I originally took computer engineering in college. Anyway it's completely possible: using an old remote that works with my TV, relays will be used for the buttons, connected to a USB cable to the computer. The computer will have a home-made driver to control the relays, pushing the buttons I need. The remote will be mounted and fixed to always point at the TV. The TV schedule will be provided from online, no webcam required.

But as I said, I like to come up with ideas first (no matter how implausible), then see how it can be done.

Oh yeah, forgot about the walk-in part.
I'm working on that already -- thinking motion control cameras or perhaps a device I carry around that gives off a specified frequency.
mirhagk




PostPosted: Tue Nov 13, 2012 10:58 am   Post subject: RE:Voice-controlled personal assistant

To use a webcam and detect if something has moved really wouldn't be too difficult. Simply put a threshold for colour variance (for dynamic changes in light, like the sun) and once it passes the threshold it's counted as moved. If a certain number of pixels move (say 5%) then you count it as movement and you enable whatever is supposed to be enabled then.

I could write a code snippit pretty quickly in C# that would do exactly that. The biggest problem would be processing power, but you really only need to check for changes a couple times a second. (it doesn't need to see movement, just that things have moved, so even a couple second delay might be fine, so long as the person doesn't enter and exit the room before the check happens)

One thing that would be ridiculously hard to implement but I always thought would be amazing in a smart home (and something I want to eventually make) is syncing the dryer to your alarm. Everyone loves wearing clothes right out of the dryer, so why not leave the clothes you want to wear in the dryer overnight, and 10 minutes before your alarm goes off, the dryer starts up so that when you wake up you'll get nice, warm and comfy clothes to wear.

You could do this with similar things as well (like a microwave or something, although doing appliances like that might be dangerous).

A really awesome idea would be for the system to know what food is in the fridge/cupboards (would require a lot of extra work when you get groceries). Then you could ask the system if you could make certain foods with what ingredients you have, or just ask in general what ingredients you have, when you need to go shopping again (could even generate grocery lists automatically based on how much you ate and what you normally use, and could possibly even automatically order them online).

You could also integrate some awesome cooking website and have the system find a new awesome recipe from the site that you have the ingredients to make. Would be hella difficult to set up, but would be completely awesome.
SNIPERDUDE




PostPosted: Tue Nov 13, 2012 1:27 pm   Post subject: Re: RE:Voice-controlled personal assistant

mirhagk wrote:
To use a webcam and detect if something has moved really wouldn't be too difficult. Simply put a threshold for colour variance (for dynamic changes in light, like the sun) and once it passes the threshold it's counted as moved. If a certain number of pixels move (say 5%) then you count it as movement and you enable whatever is supposed to be enabled then.

Yup, was thinking something like this.

mirhagk wrote:
I could write a code snippit pretty quickly in C# that would do exactly that. The biggest problem would be processing power, but you really only need to check for changes a couple times a second. (it doesn't need to see movement, just that things have moved, so even a couple second delay might be fine, so long as the person doesn't enter and exit the room before the check happens)

It can be simplified further than that by grouping pixels (dropping the resolution, but for this purpose not a problem), or just checking every Xth pixel (ex: check every other pixel, or every 5th pixel). Speculations.

mirhagk wrote:
One thing that would be ridiculously hard to implement but I always thought would be amazing in a smart home (and something I want to eventually make) is syncing the dryer to your alarm. Everyone loves wearing clothes right out of the dryer, so why not leave the clothes you want to wear in the dryer overnight, and 10 minutes before your alarm goes off, the dryer starts up so that when you wake up you'll get nice, warm and comfy clothes to wear.

This is a wicked idea. These are the types of things I'm looking for.

mirhagk wrote:
You could do this with similar things as well (like a microwave or something, although doing appliances like that might be dangerous).

I've thought about that, but the danger of such automation of such appliances turned me off from the idea.

mirhagk wrote:
A really awesome idea would be for the system to know what food is in the fridge/cupboards (would require a lot of extra work when you get groceries). Then you could ask the system if you could make certain foods with what ingredients you have, or just ask in general what ingredients you have, when you need to go shopping again (could even generate grocery lists automatically based on how much you ate and what you normally use, and could possibly even automatically order them online).

You could also integrate some awesome cooking website and have the system find a new awesome recipe from the site that you have the ingredients to make. Would be hella difficult to set up, but would be completely awesome.

I have something like this in mind already. I'm thinking as I unpack groceries I can dictate what they are/amount. They are added to the kitchen inventory, and used to crosscheck what I have to recipes. I know an app like that exists (I think), but integrating such a feature would be sweet. Just ask what can I make, and it tells me, and shows on screen. I would of course have to say to the computer I am out of something as I use the last of it.

Since this idea was brought up, do those services that deliver groceries to your house still exist? It would be cool to add automatic ordering.
mirhagk




PostPosted: Tue Nov 13, 2012 3:40 pm   Post subject: RE:Voice-controlled personal assistant

well if the system knows your inventory and the recipes of what you're making, a lot of the stuff running out would be known to it, the only parts that wouldn't would be when you change the recipe, or when the recipe is imprecise (like cheese or salt to taste).

I have no clue. I remember having a hard time finding one that'd deliver to my area a while ago, but who knows.
SNIPERDUDE




PostPosted: Tue Nov 13, 2012 5:37 pm   Post subject: Re: RE:Voice-controlled personal assistant

mirhagk @ November 13th 2012, 3:40 pm wrote:
well if the system knows your inventory and the recipes of what you're making, a lot of the stuff running out would be known to it, the only parts that wouldn't would be when you change the recipe, or when the recipe is imprecise (like cheese or salt to taste).

That's not why I'm too worried about that part, it just comes down to creating a simple script for inventorying, inventory what I do have already, and upload all my recipes (a way to search the web and add them automatically would be ideal).

mirhagk @ November 13th 2012, 3:40 pm wrote:
I have no clue. I remember having a hard time finding one that'd deliver to my area a while ago, but who knows.

I would like to look more into this one. Being in school again means my time to get everything done is difficult, cutting out errands such as picking up groceries would be sweet. But even the ability to order a pizza online by asking the computer would be interesting.
QuantumPhysics




PostPosted: Thu Nov 15, 2012 12:54 am   Post subject: RE:Voice-controlled personal assistant

Control the light switch by saying on or off in the room you are in. That would be pretty cool, but how would you go about doing it?

Possibly even in your entire, say "kitchen light on" and it turns on/off.
If you have an electric door lock, trigger the lock/unlock by saying so.
Display posts from previous:   
   Index -> General Discussion
View previous topic Tell A FriendPrintable versionDownload TopicSubscribe to this topicPrivate MessagesRefresh page View next topic

Page 1 of 2  [ 16 Posts ]
Goto page 1, 2  Next
Jump to:   


Style:  
Search: