What are Cognitive Services?

Cognitive Services are pre-built AI APIs that you can use in any application, any programming language and any platform. They based on Deep Learning Research (Microsoft does all data preparation) and are easy to help to develop an intelligent application without any machine learning knowledge.

In other words – there are Algorithms as a Service implemented via APIs into your application.

What technology Cognitive Services are based?

Cognitive Services are part of Microsoft AI platform, and it uses Machine Learning and Deep Learning Algorithms.

What exactly means those terms?

Artificial Intelligence (AI)

]

The Artificial Intelligence term is not new, but it’s definition changes since the first use of it.

Alan Turing created the first paper about AI in 1950: “Computer Machinery and Intelligence”, where he tried to answer the question “Can machines think?” – and based on that Turing Test term was well known as a method for determining if a machine is intelligent.

From that time, there was a lot of changes, where people played around the algorithms and writing different programs with intelligent behaviour, such as playing chess or automatic translation. New programming languages were developed to create more clever programs such as PROLOG (1972) – the first logic programming language or MATLAB (1984), software to accelerating engineers and science.

Based on Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig, there are four categories of Artificial Intelligence, such as:

• Systems that think like humans
  • “The exciting new effort to make computers think… machine with minds, in the full and literal sense.” (Haugeland, 1985)
  • “[The automation of] activities that we associate with human thinking, activities such as decision-making, problem solving, learning…” (Bellman, 1978)
• Systems that think rationally
  • “The study of mental faculties through the use of computational models.” (Charniak and McDernott, 1985)
  • “The study of the computations that make it possible to perceive, reason and act.” (Winston, 1992)
• Systems that act like human
  • “The act of creating machines that perform functions that require intelligence when performed by people” (Kurzweil, 1990)
    *“The study of how to make computers do things at which, at the moment, people are better” (Rich and Knight, 1991)
• Systems that act rationally
  • “Computational Intelligence is the study of the design of intelligent agents” (Poole et al., 1998)
  • “AI… is concerned with intelligent behaviour in artefacts” (Nilsson, 1998)

Artificial Intelligence is a system that can learn from data and based on that, decide to achieve a specific goal or task using Machine Learning. Natural Language Processing, Speech, Vision, Robotics etc.

AI is an entire field dedicated to making machines smart. AI is an umbrella for all intelligent algorithms.

Machine Learning (ML)

Machine Learning is a division of Artificial Intelligence. ML analyses, understand and identify a pattern in the data, such as recognise text, recognise voice etc.

There are two main types of Machine Learning:

• Supervised Learning – getting dataset with labelled training data and training examples – it’s based on input-output pairs
• Unsupervised Learning – predict outcomes on the recognising patterns in input data.

Deep Learning (DP)

Deep Learning is a subset of Machine Learning and Artificial Intelligence. It’s using Artificial Neural Networks that was created based on biological neural networks in human. It’s using multiple layers to extract higher-level features from the raw input progressively.

In other words

Artificial Intelligence, Machine Learning and Deep Learning are techniques to help you create an intelligent program.

Do you need to know all of these fields in details?

The quick answer is no. You don’t need to know Machine Learning or Deep Learning Algorithms to use AI in your program. It’s nice to know the basics about Artificial Intelligence. Still, big companies like Microsoft, Google, AWS or IBM created a bunch of APIs that you can use in your application to give them some kind of intelligence. All of these APIs are trained models that were developed based on a large amount of data, prepared by data scientists. And this what you need to do – just add them to your application.

Why Microsoft Cognitive Services?

Microsoft Cognitive Services are pre-built AI API’s. There are easy to use and requires only a few lines of code in your application to get quick results. There are as well flexible – you can use your own, preferred programming language, such as C#, Python, Java, JavaScript, Go. And the last – Cognitive Services are tested and built by experts from Microsoft Research, provided with quality documentation, sample code and as well as community support.

Cognitive Services APIs

Several categories will be changed/updated since this post will come to life. So this list could be incomplete soon and the full list of actual services you can find on Azure Cognitive Services website.

But for now (June 2020) there are five groups of Cognitive Services (some still on preview – that means you can play with them, create a proof of concept, but it’s not recommended to implement those APIs on production as they could disappear for further research)

Here you can find a list of those services with a summary of what they can do for you:

Vision APIs

Computer Vision

• Image classification
• Scene and activity recognition in images
• Celebrity and landmark recognition in images
• Optical character recognition (OCR) in images
• Handwriting recognition

Ink Recognizer (Preview)

• Recognise digital ink and handwriting
• Recognise and pinpoint common shapes
• Enable productivity scenarios

Custom Vision

• Customisable image recognition
• Custom site: CustomVision.ai

Face

• Face detection in images
• Person identification in images
• Emotion recognition in images
• Similar face recognition and grouping in images

Video Indexer

• Face detection in video
• Object, scene, and activity and keyframe extraction and analysis
• Custom site: https://www.videoindexer.ai/

Form Recognizer (Preview)

• Extract text, key-value pairs, and tables from documents
• Customised to your form, without manual labelling
• Deploy anywhere, from the cloud to the edge

Speech APIs

Speech Services (Speech to Text, Text to Speech, Speech Translation)

• Automatic speech-to-text transcription with customisable models
• Natural text-to-speech with custom voice fonts
• Real-time speech translation

Speaker Recognition (Preview)

• Speaker identification
• Speaker verification

Language APIs

Text Analytics

• Named Entity Recognition
• Key phrase extraction
• Text sentimental analysis

Immersive Reader (Preview)

• Support readers of all abilities
• Translated to 60+ languages

Translator

• Automatic language detection
• Automated text translation
• Customisable translation

QnA Maker

• QnA extraction from unstructured text
• Knowledgebase creation from collections of Q&As
• Semantic marching for knowledge bases
• Custom site: https://www.qnamaker.ai/

Language Understanding

• Contextual language understanding
• Custom site: https://www.luis.ai/

Web Search

Bing Web Search

• Ad-free web search
• Safe web search
• Location-aware web search

Bing Custom Search

• Custom search engine creation
• Ad-free custom search results

Bing Video Search

• Ad-free video search
• Video topic and trend identification

Bing Image Search

• Ad-free image search

Bing Visual Search

• Identification of similar images and products
• Knowledge acquisition from images
• Web source identification

Bing Entity Search

• Named entity recognition and classification
• Knowledge acquisition for named entities

Bing News Search

• Ad-free news search results
• Trending topic identification

Bing Autosuggest

• Search query autocomplete

Bing Spell Check

• Web-scale, multi-lingual spell checking
• Contextual spell checking

Decision APIs

Anomaly Detector (Preview)

• Identify potential problems early on
• Detect anomalies as they occur in real-time, throughout your data set as a batch.
• Get additional information about your data and adjust anomaly detection boundaries.

Content Moderator

*Detected potential offensive and unwanted images

• Filter possible profanity and undesirable text
• Moderate adult and racy content in videos
• Use built-in review tool for best result

Presonalizer

• Create a rich, personalised experience for every user.
• Deliver rich, personalised experiences in your apps
• Deploy anywhere, from the cloud to the edge
• Understand and efficiently manage the reinforcement learning loop

Cognitive Services Labs

Microsoft is also working on new APIs that probably will be available in feature to Cognitive Services family.

All of them could you can find on Microsoft Research Site, and there is an excellent opportunity to have a real impact into those APIs by giving feedback to Microsoft with of this project will be helpful and how they should evolve.

Project Academic Knowledge

• Uses Microsoft Academic Graph
• Getting access to academic knowledge

Project Knowledge Exploration

• Interactive search experience for academic publication
• Contains 1000 academic papers published by Microsoft Research

Project Conversation Learner

• Learn from interaction examples
• Reduce manual coding of dialogue control logic

Project Personality Chat

• Small talk with a bot
• Personality to choose:
  • Profesional
  • Enthusiastic
  • Caring
  • Witty
  • Friendly
• Nearly 100 chit-chats scenarios for each personality
• Can be added during create a QnA bot (QnA Maker)

Project Answer Search

• AI that understand user queries
• Targeted and faster results
• Relevant results from Bing’s industry-leading web index and knowledge graph

Project Event Tracking

• Finding events associated with Wikipedia entities
• List of related events organised by time
• Help to characterise events happening in future

Project URL Preview

• Quickly provides a URL’s page title, brief description, and the most relevant image
• Instantly enhance any social scenarios such as chat apps to let users know what they have been sent and improve their URL sharing experience.
• Leverage Project URL Preview to Suppress inappropriate or adult content, block URLs on protected networks, and get users to the right content faster.

Project Gesture

• Add custom hand gesture to your app
  

Project Anomaly Finder

• Monitoring data over time and detecting anomalies
• It’s now part of Anomaly Detector (preview)

Project Custom Decision

• Understand the context from the information user provide
• Ranks the options and makes a decision
• Automatically optimises based on user feedback

Project Local Insights

• Provides insight into business and entities within a given area that can be reached by driving, walking or public transit within a given time or distance
• It’s now part of Bing Map Local Insights API

Project Ink Analysis

• Understanding digital ink content
• Understanding:
  • Words written by user
  • Structure of content
  • Shapes
    

Container support

Cognitive Services can also be used with containers.

The benefits of using containers with Cognitive Services:

• Control over data – allows where data are processing (it’s suitable for companies that cannot send data to the cloud)
• Control over model updates
• Portable architecture – the application can be created on Azure, on-premises and at the edge
• High throughput / Low latency – the endpoint is strictly used for billing only, no customer data ever flows that way

Services supported by containers

• Anomaly Detector
• Computer Vision
• Face
• Form Recogniser
• Language Understanding (LUIS)
• Speech Service API
• Text Analytics

How to start with Cognitive Services

Try Cognitive Services

You can try Cognitive Services for free (without Azure account).

Go to site: https://azure.microsoft.com/en-us/try/cognitive-services/my-apis/ and signup/login with your email.

For seven days, you will be able to test API’s. You will get endpoints and keys that couldn’t be changed:

You will have access to services:

• Computer Vision
• Face
• Speech services
• Speaker Recognition
• Bing Spell Check API v7
• Bing Search APIs v7
• Text Analytics
• Bing Autosuggest API v7
• Bing Custom Search
• Bing Entity Search
• Ink Recognizer
• Anomaly Detector
• Personalizer

Start for free

If you don’t have Azure account you can sign in to get £150 credit for 30 days, 12 months to popular services + 25+ always free services: you can sign-in here: Azure Free Account.

Most of Cognitive Services you can use for free for development (as those services have restriction of the number of calls per second and number of transaction per month)

Azure

There are two ways to have an endpoint/key for Cognitive Services: get a bunch of services under one endpoint/key or create a new resource for each service.

Cognitive Services

Create Cognitive Services if you want to use more than one service in your application.

This one endpoint/key pair (multi-service) support:

• Computer Vision
• Face
• Content Moderator
• Language Understanding (LUIS)
• Translator
• Text Analytics
• Bing Search v7 (Web, Image, News, Video, Visual)
• Bing Custom Search
• Bing Entity Search
• Bing Autosuggest
• Bing Spell Check

The Cognitive Services resource is available in price S0 (so, it’s not a free resource) and not with all regions.

1. Create Cognitive Services
   

Select (Unique) Name, Subscription, Location (Region), Pricing tier (for now it’s only S0), select Resource Group ) and accept terms.

And after a few seconds, you can enjoy your API

Now, you can grab your key and endpoint and test them in your application.

Specific Cognitive Services

The creation of specific Cognitive Services is almost the same.
You going to choose the service you would like to use:

The difference here is that you can choose F0 tier – free, limited tier.
For example, Personalizer has a limit to 50K transactions per month):

Test your API

If you don’t have your application written yet, but you would like to test and see what kind of result will back, you can use APIs services: https://westcentralus.dev.cognitive.microsoft.com/docs/services/

For example: Computer Vision V3, Describe Image https://westeurope.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-ga/

Tested image:

And result:

Application examples

• Intelligent Kiosk
• Seeing AI App
• PowerPoint Presentation Translator
• Bing Visual Search
• Anomaly Detector Interactive Demo

Resources

• Microsoft AI MVP Book
• Microsoft Docs
• AI School
• AI Platform
• Microsoft AI Research

References

• Haugeland, J. (1985). Artificial Intelligence: The Very Idea. Cambridge, Massachusetts: Bradford.
• Bellman, R. (1978). An introduction to artificial intelligence: Can computers think?. Thomson Course Technology
• Charniak, E.A. and McDermott, D.V. (1985) Introduction to Artificial Intelligence. Addison-Wesley Pub.Company
• Kurzweil, R. (1990). The Age of Intelligent Machines. MIT Press, Cambridge, Massachusetts.
• Rich, E. and Knight, K. (1991). Artificial Intelligence, McGraw Hill, New York.
• Poole, D. L., Mackworth, A. K., & Goebel, R. (1998). Computational intelligence: a logical approach (Vol. 1). New York: Oxford University Press.
• Nilsson, N. J. (1998). Artificial intelligence: A new synthesis. San Francisco, CA: Morgan Kaufmann
• Russell, S., Norivig, P. (2003), Artificial Intelligence. A Modern Approach. International Edition. Pearson Education Inc., New Jersey