Hello and welcome to Microsoft Mechanics
Live! Coming up we’re joined by Julia White for a look at Microsoft’s approach to AI. And we’re gonna also make it real by taking a look at some
of the practical applications before breaking down how you can leverage AI for your own apps and services and
online experiences across the areas of knowledge mining, cognitive services
and machine learning. But please join me in welcoming Julia White CVP
of the Cloud Enterprise Group. Thank you so much. It is awesome to be back on Mechanics
after a year or so hiatus. Thank you. So we’re seeing lots of innovation we’re seeing lots of buzz around AI it’s being used a
lot that term I think in the past we’ve covered deep learning that a lot of
custom ml models stuff that data scientists do and you know a lot of us
afraid a scientist but on this show we wanted to actually give everyone a
real-world set of practical ways that they can actually integrate AI into
their own apps right I mean this is really important because we don’t want
AI to just be in the hands of a select few our focus is really on making AI
machine learning easier and more accessible for everyone to use and
that’s really the key focus and for the past decade we’ve been integrating AI
into a lot of our cloud services one of the early implementations actually you
remember was in exchange online of distinguishing what was good mail vs.
junk mail is just a really early example but of course AI is an integral part of
being maps and being search and curse the official and biometric
authentication with Windows hello also using AI and then infused into our
office 365 experience right with PowerPoint designer or word grammar
checker even and then we also use AI pretty pervasively around security right
and using all of our advanced threat protection czar underlying that our
cloud services is using a ion or security technology now we’ve taken all
that technology that we’ve used internally for all of those applications
and turned into Azure AI services that you can use in your own applications and
this include many of the pretty sophisticated pre-trained ml models that
we’ve now turned into Azure cognitive services and we’ve also enabled support
for the most popular AI frameworks so you can train your models using a number
of different frameworks in a very open approach on that front and of course for
those people who need some very highly specialized you can also
collaborate and build your own custom machine learning model using Azure as
well so regardless of whether you using a pre trained model or a custom ML model
you can always of course get the AI compute you need with Azure and then go
ahead and deploy and run that either in the cloud and/or on the edge in that
area right so we’re doing a lot in terms of democratizing AI and really making it
available for everybody to use so why do we get to do this and actually make it
real and show people a couple of demonstrations what we can do I love it
all right so great let’s get started with an example of knowledge mining now
this is about quickly getting insights and all of your existing information
within your organization so for example I’m assuming many of you who have been
around for a long time your businesses have been and you have a lot of data
you’re sitting on but it’s probably sitting in things like file cabinets or
scanned in some archive it could be written it could be pictures it could be
videos so what if you could use AI to add structure to that data and then have
it indexed and easily searched or even more powerfully you can discover
different patterns or relationships within that data by surfacing those
insights so let me give you an example so I’ll make switch here – we have JFK
example so about 25 years ago after Congress mandated that the jf the
documents related to the JFK assassination became Declassified so
last October they released 34,000 pages of documentation 34,000 pages of is a
lot I mean that’s that’s probably how how high would that stack up I’m gonna
say least to the top of that wall right probably so first of all we took that
enormous mistake of data paper and we digitized it and then we put it in Azure
and then we applied as our search with cognitive services where we’re able to
intelligently annotate and index that information to make sense of it find
relationships and surface new insights then we built a simple web app on top of
it so you can literally interact with this data and it’s a site you can all go
to as well but let me show you how it works so I’m sitting out at the the web
experience again on top of that data repository that’s been searched index
I’m gonna go ahead and search on Lee Harvey Oswald hopefully everyone knows
his relationship to the Drake ok assassination
so we hadn’t search on that now you can see over here on the Left adjure is kind
of pulling out different key entities based on the contents of the document so
you see things like the CIA and if JFK of course and Lee Harvey
Oswald it all makes sense over here on the right you can see a variety of
different documents and you can fact see that his name is actually highlighted it
could be like typed information here written in this example I’m scanned and
other things and even if I using optical character recognition or OCR that
cognitive skill we can annotate that text document to make it searchable even
though it was handwritten and now if I scroll down you can see it’s capturing a
picture too so again because if you’re using computer vision the cognitive
skill I can understand the content of a picture as well so if I click on this
you can see it recognizes that’s Lee Harvey Oswald you see the name is
actually highlighted here and you can go down I it’s actually scanning over this
you can see it’s giving me an annotation again using that OCR capability within
the image itself so kind of close that out now the next thing I can show you is
also about the relationships that’s found between the information so if I
click here on the graph site it’s actually pulling together a graphical
representation of the relationships across specific terms that are found
across this whole body of documentation so if I grab Oswald here for example you
see kind of a cluster of information circling around him like Harvey and Lee
of course that makes sense but the interesting thing is you see this
connection right here Sylvia Duren right if I pull that out and a big collection
of things around Duren now if we dig into this document
here our ticket to this here in searching and I can see I’m finding
documents that let me to understand that this is the interrogation of Sylvia
Durand by the Mexican government and we explore the documents you see that she
actually wrote to the Cuban consulate in Mexico and was Lee Harvey Oswald’s
linked to the Russian KGB all just became discovered because this graph can
become advanced out of this it’s pretty amazing
which are able to find with this azor search with the help of those cognitive
services and if you want to try this at home you can go back to this web app
it’s available publicly at AKMs slash JFK files and also this is a public and
available on github two really cool stuff of course you can reverse engineer
some of this work on github so really cool but you’re also able to take
document types that weren’t searchable this is paper like this is handwritten
paper leveraged as your search with cognitive services to really allow us to
index all of these things how do I make them searchable but also structure
how easy is it to harness these types of capabilities well let me show you how
easy it is Jeremy okay oh can I take in that document all that scanned imager
you put it in add your blob storage and I’ve now hooked up my Azure search here
in the imager portal to that storage and all I have to do is literally set up
search I go down here and I can just select the cognitive skills I want to
add to my search and I can just say okay it’s literally that simple but you
remember we were using OCR in that search I did so let me go ahead and
choose OCR and when I do that you’ll see additional skills are generated so I can
also pick additional attributes because of that turning on so and that JFK of
course we use natural language and other things too so all of them literally it’s
that simple okay so we’ve seen a lot of different categories here in terms of
the cognitive services that you mentioned earlier but many of these are
pre-built ai models that came about really as we started developing various
services in Microsoft so how did these come about yeah I mean that’s right it’s
basically where our core engineering teams along with Microsoft Research have
made several breakthroughs across the area of vision language speech and
search and for example we recently received the highest score for our
machine learning capabilities through the Stanford question and answer data
set called squad and it’s a test of AI reading comprehension around hundred
thousand questions under their related answers so pretty impressive what we’re
doing alright so beyond integration with intelligent search how can we use
cognitive services API is in our own applications well I mean basically every
application can benefit from cognitive services in some way but let me give you
a specific example so we’re here in Florida and it’s famous for many things
but it’s including oranges of course so let’s just take the example of an orange
farmer currently many orange growers are not differentiating their produce
they’re sending all of their oranges for juice production but maybe they want to
build a model that creates a sorting facility that uses AI that can decide
what’s great a which and they sell those oranges at a more lucrative retail
channel or what’s the lower grade B and they send those those oranges off
produce production so what if they could use this AI to differentiate oranges by
using computer vision and sharing the oranges that are sorted to grade a go in
one category and go be great B in the other category and using
intelligent edge to do that for them so it have to be a pretty pretty smart
system because bad and good oranges are pretty close looking right you think
right but let me show you how you can actually train a model to do that let’s
try it so I’m out here in a now a within my computer vision area and I have
already pre-loaded a group of oranges into the site and without any trading
it’s already identified that it’s a group of oranges in a pile and has
confidence of eighty-five percent so that’s lutely just loading out the image
it’s publicly available and thanks to everyone participating is getting to me
that confidence I mean it’s like beautiful oranges here as well but the
model hasn’t yet been trained to decide what is a great a orange or a grade B
orange but that of course makes sense because there’s the orange far more
you’d want to decide your your categorization of what’s the threshold
for great a a grade B right so I want to do my own custom training on that so
that me do that now I’ll go over to my custom vision AI and I’ve got a project
I’ve started here on orange classifications so go ahead and click in
here and I’ve got a number of images loaded to help start training my model
so you can see there’s labels on some of these oranges and they are they little
ugly I know dad for that orange but you know yeah and I can’t all be beautiful
on the inside they’re delicious yeah so if I click on great a you’ll see
it’s seeing that these are what I’m defined and categorized as grade a level
oranges and if I I’ve started doing my training on grade B so you can see some
of those but I want to go ahead and add another image to train that model
further so I go ahead and just add images and this is how I started the
model and got it going so I’m going to add a new grade B Orange open it up and
I’m gonna go ahead and tag that as a grade B orange again not simple and I’ll
upload that file uploading it it’s gonna retrain and then with you know within a
few seconds I can go ahead and have that updated to my models relearning so let
me do a quick test to see if that how it’s working so I’ll go to my quick test
area and I just browse from my files I’ll grab I’ll start with an A grade a
orange okay drop that in and I see it’s giving me the probability that’s a 99.5
percent positive that this is a great babies perfect looking orange I think
I’ve ever seen right I mean I guess it’d be hard to get that one wrong right but
let me go try a great b1 here yeah let’s grab a bead
and load that up and see how it does so this case is 97.7 for some Potter’s of
you see it looks a little different if they don’t pick up on that level of
detection now again this is just a great example of how you can get started with
AI for your own uses I didn’t wasn’t doing any coding I just load it up and
train the image and you can try this today on Azure it’s on custom vision AI
where you can get access to this experience and if they move forward
we’re gonna continue to make cognitive services API is even easier to use for a
great example just here at ignite we launched our unified speech service then
now combined speech to text text to speech translation and the ability to
train the model based on unique custom speech mounted patterns all as part of
our Scott custom speech API these are really great ways to leverage the AI
effectively that Microsoft is built and tested and our AI services are even
getting smarter and smarter all the time as we look at more images so what do you
want to what if you wanted to build out your own models and train those to work
with your own data how does that look yeah so for the folks who want the
custom build their own machine learning models and the more sophisticated needs
Ashur provides a complete machine learning platform that you can take
advantage of and I’ll just show you kind of one part of that here okay go into
getting back to my add your portal and I’m in my data bricks notebook and one
of the great things about as your data breaks as it enables both data
scientists and data engineers to collaborate on shared projects within
this interactive workspace so you can easily pull in structured and
unstructured data you can prepare it connects a called data wrangling if you
clean it and get it ready and train for my AI models right within the workspace
so what I’ll show you here is actually example of a shoe retailer and they’ve
built a shoe recommendation engine such that way and a customer comes in
searches they’re shown something it’s highly relevant so it’s to hopefully
drive up sales and results but right now the Shuji realtor wants to then take
that recommendation engine and use AI to make it even more highly predictive do
we do anything then to help with the types of algorithms and data models that
you might use to make these types of recommendations well yeah actually one
of the things we also against this this week at ignite is a capability called
automated machine learning and that’s basically using AI to train AI in a
sense if you think about it that way but essentially we test all of the different
models using AI to see which algorithm performs the best has the most act
results and then automatically recommends that to you so we’d even
expedite that process of it you can also use you know simple sequel like querying
to usually explore the data within spark here in Azure data bricks and also get a
sense of the current recommendations engine and sakura see okay let’s look at
that alright so I’m down here now first of all start and this is the clickstream
results so right well let me show you that what this means essentially this is
the frequency of clicks over the period of time now you see all the clicks are
happening kind of at the end which means that’s when the shoe is going on sale
but I get a clearance out of inventory so that’s probably when it’s the
cheapest price that we’re gonna sell it rate which isn’t great for my sales
right when I’d really wanted to be lots of clicks in the beginning when this she
was new and hot so this gives me a clue that something’s not right
so I’m gonna go ahead and bring the data scientist into the notebook didn’t see
what you know she’s and so she can confirm what people are seeing in the
different searches so as I scroll down I can see that this is the current results
she’s providing it like here’s the target shoe and then here’s the related
results these woolly slippers for open-toed shoes okay so I thought that
we would actually to make the recommendation look more like that first
shoe that’s surprising that they’re recommending bad I’m not getting a lot
of clothes look comfortable but I don’t think you want to wear those in public
all right so what I’m gonna do is I’m going to scroll down here I’m going to
optimize this model by bringing in a deep learning framework in this case
tensorflow package that can help provide recommendations it’s called deep image
feature Iser and effectively breaks down the different attributes or the feature
of every shoe and the Claddagh log so things like open-toed heel color size
type more so that i can really get a granular assessment of that now once i
feed those attributes back into my machine learning model i’m gonna prefer
on the same search again and see what kind of results we get if i scroll down
here you see there again is my target shoe and now you see I’m getting very
similar recommendations around that target and even just to make sure it
works on different types other than these open-toed in different colors of
the same shoe you can see even it’s very close on the other open-toed models yeah
so you can see the accuracy the results around again I have now loaded up a
different Salado heel I can see that I’m getting very similar results so I feel
much more confident about the recommendations
making so then once I see that I’ve got a recommendation engine that’s working
well I can then finally take that and distribute it in Azure cosmos database
and I can integrate that into my machine learning model that’s running the
operational database of my web and mobile applications so this
recommendation is now going to be showing up across all of my customer
experiences it’s a really powerful stuff three awesome ways to start to apply AI
in your own day-to-day workflows and experiences but where should people go
to learn more well luckily you can check all everything that I showed to you out
today it’s all public so the JFK demo they add your cognitive services the
vision API and you can also sign up for a free 14-day trial of azure data bricks
you can see that workbook and do that data wrangling and AI prep as well and
the power of all this is that the more people who use the open frameworks to
participate in this the better the AI innovation gets it really is a kind of
open community approach and we’re all going to benefit from where we can
harness Thank You Julie emotion we keep tracking this on Microsoft Mechanics
having it hit subscribe hit subscribe now follow us on Twitter thanks for
watching and we’ll see you all next time

3 thoughts on “AI and the top 3 ways to adopt it in your Apps using Azure | Ignite 18”

  1. Great presentation, easy to follow for everyone. I specially like how they customized the training by simply adding more training data (around minute 10, oranges example).

  2. Like wow.
    Well then..to anyone who missed getting out ahead of the curve on the dot-com craze 20 years ago or the social media bandwagon 10 years ago — here's your chance to get in on the ground floor yet again: Machine Learning and AI.

Leave a Reply

Your email address will not be published. Required fields are marked *