What is natural language processing? What is it used for? And why is it such a necessity in today’s modern tech? Most people would largely associate it with automatic language translation, and while that’s a big part of it, that’s only a fraction of the picture. We figure the best way to demonstrate the importance of NLP is to highlight 4 everyday applications that you may not realise rely on this field of artificial intelligence. Let’s dive right in.
The fact that the word itself has made it to the dictionary makes it a pretty common daily action. A Google search allows for access to near endless information at the blink of an eye. But what is actually going on when you hit that search button? At the core of any search engine is a process more technically referred to as “information retrieval”, where your query is passed through a classification algorithm that typically returns millions of matches. In order to narrow it down for you, a ranking system takes place, displaying the documents in order of relevance. And if it does the job well, you find everything you need on your first page of search results, making for a successful search engine!
2. Spell checking
We may take this one for granted but its impact is significant. Built into most word processors, any user can avoid rudimentary spelling mistakes and demonstrate high grammar and spelling proficiency in just a few clicks. This technology has come a long way from detecting misspelled words, and now scans for grammatical errors and is progressing towards being able to finish sentences on your behalf. Spell checking algorithms are very complex to build, but most make use of a calculation called the “Levenshtein distance”. This compares the strings of characters entered with existing sequences and calculates the linguistic difference, which is essentially down to insertions, deletions and substitutions.
It is worth mentioning that spell checking is also an important part of the search engine technology too, to make sure what you query is valid in the first place, as well as semantically “disambiguated”, which is another big deal in NLP.
3. Voice assistants
If you use Alexa, Siri, Cortana or any another voice activated virtual assistant, you’re making use of natural language processing every time you interact with it. For the virtual assistants to recognise, process and reply to a simple question such as “What is the weather forecast for the day?” requires a combination of advanced language technology such as automatic speech recognition (ASR), language understanding, dialogue systems, text to speech (TTS), language generation, and information retrieval. Breakthroughs in automatic speech recognition have given life to virtual assistants such as Alexa, Siri and Cortana. This requires a great deal of training, which includes the recording of speech parallel to written (often phonetically-transcribed) text, by which the machine can learn phonetic patterns relating to each word. This learning curve is similar to the cognitive process we humans are subject to as we learn to comprehend and produce language.
4. Language translation
If you’ve ever wondered why translation apps are great for looking up individual words, but return mixed results when translating full sentences, the reason is context. Consider polysemous words, words which change meaning depending on the context in which they’re used. “Glass” for example, is used to describe a material and refers to a container used to drink from. If these words were rare it may not be such a challenge but in the English language it is estimated around 40% of all words are polysemous.
And what about figurative language, cultural references, unparalleled grammar, morphological rules, and untranslatable words? You could list every rule of a language (rule-based approach), you could take a statistical approach to recognise patterns over large corpora, or you could apply deep learning and neural techniques. Each method is time-consuming, and none are able to consistently translate with 100% accuracy. So, the next time you find your translation isn’t fully accurate, you’ve seen the complex technologies going on behind the curtain!
What’s next for NLP?
Language technologies are becoming ever more present as our world becomes increasingly digital and we can expect a lot more from this cutting edge technology. NLP is progressing quickly and is able to infer context and meaning from language with ever improving accuracy. As a consumer technology, NLP makes things ever more convenient for us. Applied into a business environment, NLP has significant potential to transform operations and processes. Sentiment analysis / risk profiles, predictive triage and support, and decision assistance are all growing fields. What’s most exciting is that this is only the beginning for NLP in the business environment and its impact is sure to grow in the coming years.
If you would like to learn more about how Natural Language Processing could help your organisation, get in touch with us at firstname.lastname@example.org or call 0121 638 0760 and we can arrange a free discovery session.