Flex, Regular Expressions, and Lexical Analysis

 What Makes a Programming Language?

There is an alphabet, words, grammar, statements, semantics, and various ways to organize the previous in order to create a computer program in a programming language. Flex helps developers create a tool called a lexical analyzer which identifies the words of a program during the compilation/interpretation process.

Continue reading

The Power of WordNet and How to Use It in Python

In this post, I am going to talk about the relations in WordNet (https://wordnet.princeton.edu) and how you can use these in a Python project. WordNet is a database of English words with different relations between the words.

Take a look at the next four sentences.

  1.  “She went home and had pasta.”
  2. “Then she cleaned the kitchen and sat on the sofa.”
  3. “A little while later, she got up from the couch.”
  4. “She walked to her bed and in a few minutes she was snoring loudly.”

In Natural Language Processing, we try to use computer programs to find the meaning of sentences. In the above four sentences, with the help of WordNet, a computer program will be able to identify the following –

  1. “pasta” is a type of dish.
  2. “kitchen” is a part of “home”.
  3. “sofa” is the same thing as “couch”.
  4. “snoring” implies “sleeping”.

Let’s get started with using WordNet in Python. It is included as a part of the NLTK (http://www.nltk.org/) corpus. To use it, we need to import it first.

>>> from nltk.corpus import wordnet as wn

Continue reading

Notes from IIT Kharagpur ACM Summer School on ML and NLP

IIT KGP CAMPUS

Entrance to library and academic area.

[This entry has been edited for clarity. An example given discussing the similarity of words in French and English was incorrect. The following sentence has been removed: “The next question addressed by Bhattacharya was the ambiguity that may arise in languages with similar origins, for example in French ‘magazine’ actually means shop while in English, well it is a magazine.”]

Today is June 14th, so I am 14 days into summer school; 7 more days left, and we are all already feeling saddened by the idea of leaving Kharagpur soon. In India, an IIT is a dream for 90% of the 12th graders who join IIT coaching classes. The competition is high so not everyone gets in. I’m one of those who didn’t get in. So when I saw there was an ACM Summer School opportunity at the largest and oldest IIT in India, obviously I grabbed it. By sheer luck, I was selected to actually attend the school. Over the course of 21 days, we have been tasked to learn about machine learning and natural language processing. Continue reading

Build your own Natural Language Processing based Intelligent Assistant using Python, It’s easy!

Before we begin, let us talk about how Mike (a fictional character) spends a typical morning. Mike begins his day by searching for breakfast recipes on Google Now (https://en.wikipedia.org/wiki/Google_Now). After a filling breakfast, Mike starts getting ready for work. He asks Siri (http://www.apple.com/in/ios/siri/) to tell him the weather and traffic conditions for his drive to work. Finally, as Mike gets ready to leave the house, he asks Alexa (https://en.wikipedia.org/wiki/Amazon_Alexa) to dim the lights and thermostat. It is not even 10 a.m. yet, but Mike like many of us has already used three intelligent personal assistant applications using Natural Language Processing (NLP). We will unravel the mysteries of building intelligent personal assistants with a simple example to build such an assistant quite easily using NLP.

Continue reading

The Pythonic Way

Python is a very powerful programming language that understands structural, functional and object oriented programming paradigms. New comers to Python from other languages tend to carry with them their mother (programming) tongue culture. Although they achieve the required task, they might have fallen in the trap of using Python the wrong way. In this post, we cover some efficient tricks to achieve tasks in Python; we call it the Pythonic way. Find an IPython Notebook for all tricks here on our GitHub repository.

Lists, Tuples, Dictionaries and Sets

Continue reading