Professional Documents
Culture Documents
cranklin.com
Chiba City hacks
I want to create my OWN version of Siri…. except I don’t care for having it on my
phone. I want my entire house to be talking to me… more like Jarvis (from Ironman).
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 1 din 26
:
I believe I have access to all the right resources to create this AI.
It breaks down into three major parts:
1) convert speech to text
2) query database populated with q&a
3) convert text to speech
Speech to Text
Most speech to text engines suck. Siri’s works exceptionally well because the engine
isn’t on your phone… it’s remote. I supposed we can hack Siri by running a MITM
attack on an iphone and faking the SSL cert and intercepting the apple ID…. OR we
can do something much simpler. Google’s Chrome 11 browser includes a voice input
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 2 din 26
:
function (which isn’t yet part of the HTML5 standard) and can convert your speech
into text. This guy discovered that it was happening remotely through an
undocumented API call to google. All we have to do is access this same API and we
got ourselves a free Speech-to-Text engine!
In case you don’t understand Perl, this is how you use the API:
POST params: Content (which should include the contents of a .flac encoding of
your voice recorded in mono 16000hz or 8000hz)
Content_Type (which should read “audio/x-flac; rate=16000” or 8000 depending
on your voice recording. This should also be mirrored in the Content-Type section of
your header.)
{"status":0,"id":"008bd1a95c3c2b04bd754da5e82949f4-
1","hypotheses":[{"utterance":"what day is it
today","confidence":0.91573924}]}
Sweet.
This is probably the most difficult part to obtain. To build it from scratch would
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 3 din 26
:
require tons of data and advanced algorithms to interpret sentences constructed in
various ways. I read somewhere that Siri was using Wolfram Alpha’s database…..
so…. I checked out Wolfram Alpha and they have an engine that answers your
questions. Not only that, they also offer an API service. (If you query less than 2000
times a month, it’s free!). So I signed up for the API service and tested it out. I asked
it some simple questions like “What day is it today?” and “Who is the president of
the United States?”. It returns all answers in a well-formed XML format.
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 4 din 26
:
<plaintext>Friday, January 13, 2012</plaintext>
</subpod>
</pod>
</queryresult>
Again…. sweet.
Text to Speech
This part is easy… and google makes it even easier with yet another undocumented
API! It’s straight-forward. A simple GET request to:
http://translate.google.com/translate_tts?
tl=en&q=speech+to+convert
Just replace the q parameter with any sentence and you can hear google’s female
robot voice say anything you want.
Voice Input
I can either make my program run over a web browser or as a stand-alone app.
Running it over the web browser is cool because I would then be able to run it from
just about any machine. Unfortunately, HTML 5 doesn’t have a means of recording
voice. My options are a) only use google Chrome, b) make a flash app, c) make a
Java applet.
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 5 din 26
:
<?php
$stturl = "https://www.google.com/speech-api/v1/recognize?xjerr=
$wolframurl = "http://api.wolframalpha.com/v2/query?appid=[GET+Y
$ttsurl = "http://translate.google.com/translate_tts?tl=en&q=mas
$filename = "./test1.flac";
$upload = file_get_contents($filename);
$data = array(
"Content_Type" => "audio/x-flac; rate=16000",
"Content" => $upload,
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $stturl);
curl_setopt( $ch, CURLOPT_HTTPHEADER, array("Content-Type: audio
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
ob_start();
curl_exec($ch);
curl_close($ch);
$contents = ob_get_contents();
ob_end_clean();
$textarray = (json_decode($contents,true));
$text = $textarray['hypotheses']['0']['utterance'];
$wolframurl .= urlencode($text);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $wolframurl);
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 6 din 26
:
ob_start();
curl_exec($ch);
curl_close($ch);
$contents = ob_get_contents();
ob_end_clean();
$obj = new SimpleXMLElement($contents);
$answer = $obj->pod->subpod->plaintext;
$ttsurl .= urlencode($answer);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $ttsurl);
ob_start();
curl_exec($ch);
curl_close($ch);
$contents = ob_get_contents();
ob_end_clean();
header('Content-Type: audio/mpeg');
header('Cache-Control: no-cache');
print $contents;
?>
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 7 din 26
:
or IR blaster and respond with a “yes, master”. I’ll post videos when it’s done.
Updated to give you a link to a working demo. This version requires you to use the
Chrome browser (thanks to Shiv Kokroo for generously providing hosting / wolfram
app ID):
Working Demo
Click on the little microphone and try asking her a question like “how many legs does
a spider have?” or “what is 15 + 11?” or “turn off the lights”.
Sponsored Content
« Previous Next »
Leave a Reply
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 8 din 26
:
cranklin on March 21, 2012 at 9:56 am
Hey Shiv! Dude, that would be awesome. Just to keep you updated, I’m
looking into the X10 devices so I can make utilize Jarvis to automate
the home. I have a arduino + ethernet shield programmed to accept
commands from jarvis to trigger responses for TV, lights, garage, etc.
Reply
http://mikepultz.com/2011/03/accessing-google-speech-api-
chrome-11/
On that page you can see how he used perl to access the
google speech api.
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 9 din 26
:
chefwear on May 8, 2012 at 9:24 pm
Reply
Hey chefwear, I wasn’t hosting the demo and didn’t realize it was
down. I’ll try to get a working prototype back up.
Reply
Looks awesome. Demo what nice. I’m very interested in the result.
Reply
Hi Cranklin! What’s the status of this? Did you get the arduino working?
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 10 din 26
:
instructions like “http://[internal IP]:[port]/?dev=tv&cmd=on” to turn
the tv on.
I’m looking into other protocols such as X10, xbee, etc before I finalize
the project.
I’ll post source codes for the arduino webserver and an updated
jarvis/siri in a future post.
Reply
hey man this is so cool , i always kinda wanted to do this can i put this on my
tech blog ? cheers
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 11 din 26
:
cranklin on May 13, 2012 at 4:40 pm
Reply
Could you make a 100% custom server and make it sound like zazu from the
lion king ?
Or even better allow “pst!” to toggle siri and have hyper sensitive sound for
intimate conversations ?
Reply
lol David. That would be comical. I can’t get Zazu’s voice, but I did
notice that when the google TTS api is triggered from a different
country, the accent is different.
Reply
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 12 din 26
:
cranklin on June 12, 2012 at 4:37 pm
Hi Stefanoxr. It’s not dead, but I just haven’t had time to work on it
lately because of my job. Feel free to develop it further. Everything is in
github though I apologize for the lack of organization. I also added a
trueknowledge API version.
Reply
Reply
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 13 din 26
:
and whatnot? I’m just looking for some open resource to learn.
Hey Michael, there’s nothing wrong with that. I’m pretty sure
you’ll grasp everything you need soon enough. If you don’t
mind me asking, can you tell me what technologies and/or
programming languages you are currently comfortable with?
Reply
Hey Michael, I’ll use whatever language is the best fit. For example,
programming the arduino microcontroller requires C (not true C as it
does use objects… but similar enough)… I chose PHP for the backend
of the web interface, but I could have easily used Python or another
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 14 din 26
:
language of choice. Don’t let the language be your focus. If you’re a
good programmer, you’ll be able to learn a new language on the fly.
If I was going to make a recommendation to a new programmer, I’d
recommend python. It’s widely supported, it has many different
applications, it’s fairly easy to learn, and it’s just an over great
language.
Reply
Awesome! Python and C are my next focus’ then. I’ll let you
know how it’s coming along in a while. Thanks for all the
help!
Reply
Rahul, you can disable one of the 2 AI engines. I reckon the double
query will slow things down significantly.
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 15 din 26
:
Tom on September 2, 2012 at 4:24 pm
Reply
Hi Tom, I am sorry about the late response. I have been so busy with
work.
If you haven’t learned any computer languages yet, this may all seem
very overwhelming. I recommend getting your feet wet first. There are
tons of online resources. codecademy.com offers some great classes
that will help get you on your feet. I recommend it.
Reply
Reply
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 16 din 26
:
Hey I came across your jarvis project while I was looking into doing
something similar. Would you possibly be able to email me I have some
questions I would like to ask. Thanks in advance
Reply
Reply
How did you get the male voice vs the google female voice?
Thanks
Reply
Reply
Thanks for the fast response. So you just set your region to
what?
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 17 din 26
:
cranklin on January 24, 2013 at 7:56 am
Reply
I’m in the USA as well and it’s a female. Where did you hear the
male voice? The link that I provided is actually an Indian server.
That might be why. lol
Syn virtual assistant is coming this april i saw its video on youtube
something named like madonna virtual assistant its free and made for
developers. if it can be extend i will definitely be using it because they say
its free
Reply
I had a bunch of trouble attempting to use the google api until somebody
suggested to me that I try http instead of https. I don’t know why https was
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 18 din 26
:
failing for me, but just in case somebody else is having problems, here’s
something to try.
Reply
Hello, do you run the code on an wamp server or similar? because php is
serverside, so i don’t know how you host it, for the arduino to connect,
doesn’t they both have to be on the same network? Or do you forward the
requests to the arduino from a website? Victor
Reply
Hey Cranklin,
I’m looking into getting my feet back into programming, i’ve had basic C++
experience so I have a very general idea of whats going on in your program.
I’m interested in replicating your program here but I want to learn what is
happening at the same time and not just copy the code line for line. Is there
any way you can add a few comments to the file to further explain the
implementation of the APIs in the code?
Regards
Reply
Hi Tiko, sorry for the late reply. I’ve been crazy busy. Yes. Actually, if
you can wait, I’m re-releasing Jarvis with a lot of enhancements and it
will be easier to follow.
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 19 din 26
:
Reply
Hi i have been wondering about your JARVIS and thought that this is really
cool, but i do not know the programs you used so if you can please tell me
them i will be most pleased to make my own JARVIS! P.S. Ive been looking
for this for a while and this seems perfect!
Reply
I’m really new to coding. I’m kinda confused as to what you’re coding this
on, and what language, and if you’d be interested in kinda making a more
step by step kind of post.
Reply
Hi, I have been following your javrvis project, and it’s ridiculously cool! I
would like to recreate your program. I know java and I wanted to ask if this
could be recreated using java or would I have to pick up some python to
recreate the project? Look forward to hearing from you. Awesome Project!
Reply
you can use java (or other language). Just pay attention to the requests
being made to google as well as the AI API. With just a little bit of work,
you can easily port this to Java.
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 20 din 26
:
Reply
Reply
Reply
I’m not exactly sure what you’re asking, but yes. If you look at part 2 of
this blog post, I think I’m doing what you’re asking about. I may be
mistaken.
Reply
Okay will look into it thanks! Do you know if this whole project
can be made using a beagle bone board?
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 21 din 26
:
cranklin on July 13, 2013 at 10:32 am
http://www.codeproject.com/Articles/579471/How-to-Write-Your-Own-Siri-
Application-Mobile-Assi
Reply
Hello cranklin
i am a newbie for programming. You have a code posted there how can i get
it running?
the PHP code you have given in the github files..
Reply
how can we run the above code?? by using which software ?? pls tell me
..anybody
Reply
Hi I’m a UI design Developer so I’m always looking for some cool projects
and i think this is very cool and would love to help you develop this to make
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 22 din 26
:
it a desktop app that people can just down load and just have it running
every where so like jarvis even if you are at work from your phone you can
have your AI complete task at home, stuff like that.
Reply
Hey cranklin on what language is your project based? Please tell me.
Reply
hey brother i pretty much like your work and looking forward for it, but as
you say about the Google’s API..
As i was checking related to JARVIS and came across this website. I am
week in HTML so can you please check this site they are doing same like
google API. and please let me know whether they have used google’s API
Reply
i am very very sorry about it i din’t mentioned the link over hear the link
is http://jarone2.jarviscorp.com/newdemo.html
pleas help me to find the working way of the website
Reply
Good day! I just would like to offer you a big thumbs up for your
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 23 din 26
:
great info you have got here on this post. I’ll be coming back to your site for
more soon.
Reply
Reply
Hi, your site is amazing! Thank to you i have finished my version of the
programm. Now the url is not working becouse it was released a new
version of the api. I solve the problem reading here ->
https://github.com/gillesdemey/google-speech-v2
For windows user: it not necessary to convert the audio to flac! you can use
.wav file!
Reply
I’ve learn some good stuf here. Definitely price bookmarking for
revisiting. I surprise how muh attemt you set to create any such great
informative website.
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 24 din 26
:
Reply
It’s fantastic that you are getting thoughts from this piece of writing as well
as from
our dialogue made at this place.
Reply
Reply
I LIKE THAT
Reply
What if I dont really want it to respond to me with a voice, but with text? But
also understand what im saying. So, I speak in the microphone, and it
responds on the screen with text
Reply
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 25 din 26
:
Reply
There’s an easy and best tutorial on Youtube to get started completely on all
the necessary concepts to Build An Advanced App Like SIRI :
Reply
The Chilean winger and Mesut Ozil are in talks with the Gunners over
lucrative contract extensions.
Reply
Blog at WordPress.com.
https://cranklin.wordpress.com/2012/01/13/building-my-own-siri-jarvis/ 28.10.2023, 22 42
Pagina 26 din 26
: