September 2015

Thirty-Nine months ago, I started my PhD.  As I look back through logbooks, half written papers and the accumulated file lint that clutters my desktop, I wonder where that time has gone.  But this is no time for reflection.  From here on out, my eyes are set firmly on September 2015.  They must be, if I hope to finish my PhD.

Over the coming nine months I hope to achieve all of the following goals:

1. Finish off some experiments using machine learning to rank substitutions in different contexts. 
2. Write up and submit this work to the main track at ACL-IJCNLP 2015.
3. Design and implement an experiment looking at the effects of simplification for people with aphasia.
4. Compose my thesis.
5. Find a job.

All in all, I think that this is manageable.  I would really like to go to ACL this year, but my fate will be left to the hands of the reviewers.  I haven't formally started writing my thesis yet, but I do have lots of material written up, so I intend to start with this and work from that point.  The aphasia experiments will be interesting, but I'll write more about them at a later point.  Lots to do, so best get started.

Why Lexical?

In this post I will put forward my arguments for picking lexical simplification as a PhD topic.  I'm certainly going to face that dangerous 'why' question when writing up my thesis, not to mention in the viva. I figure that if I commit my reasoning to page now, then I can come back to it at a later date. To convince myself, if not others.

The question of 'why lexical?' needs addressing at two levels.  Firstly, why choose simplification given the wide scope of natural language processing applications to which I could have committed four years of my life?  Secondly, why lexical simplification given the wide range of simplification research out there?  I'll answer these in turn.

Why Simplification?

Let me start with a confession. I didn't start a PhD to study lexical simplification.  I didn't even come with natural language processing in mind.  About four years ago I was offered an opportunity to study at Manchester on the newly founded Centre for Doctoral Training (CDT).  One of the aspects of the CDT programme which appealed to me was the fact that students were not initially tied to a supervisor or research group. We were in effect free for our first six months to decide upon a group, supervisor and topic.

During the first six months, I took three masters modules.  Two on machine learning and one on digital biology.  Both fascinating fields, and both of which have ample opportunities for a PhD.  I spoke to the relevant people about these opportunities, but I just couldn't find something that inspired me.  Resigned to the fact that I may have to choose a topic and wait for the inspiration to come I started to focus on the machine learning research.

We had weekly seminars to acquaint us with the research in the school.  Each seminar was given by a different research group.  They varied in style and form - from the professor who brought in props to talk us through the history of hard drives to the professor who promised free cake every time their group had a major publication.  One week, it was the turn of the text mining group.  During the presentation the idea that text mining could be used to make difficult documents easier to understand was mentioned.  I took the bait, started reading papers, emailed my prospective supervisor and from there I was away.  The spark of inspiration drove me to develop my first experiments, which led to a literature review and an initial study on complex word identification.

Simplification is a great field to be working in.  I like that the research I'm doing could improve a user's quality of life. Ok, the technology isn't quite there yet, but the point of research is to reach out to the unattained. To do something that hasn't been done before. There are lots of opportunities in simplification. Lots of unexplored avenues.  I'm going to write a long future work section in my thesis (maybe a future post?) because there is a lot to say.

Why Lexical?

When I started looking into simplification back in 2011, there were three main avenues that I could discern from the literature.  Firstly, syntactic simplification - automated methods for splitting sentences, dealing with wh- phrases and the passive voice.  Secondly, lexical simplification - the automatic conversion of complex vocabulary into easy to understand words.  Finally, semantic simplification - taking the meaning into account and doing some processing based on this.  I grouped both lexical elaboration and statistical machine translation under this category, although I would probably now put elaboration under the lexical category.

I felt that my research would be best placed if it fell underneath one of those categories.  Although I could see viable research options in all of these, my background in machine learning and data mining was best suited for lexical simplification.  This was also around the time that the SemEval 2012 lexical simplification task was announced.  This task gave me an intial dataset, my background in AI gave me a set of techniques to apply.  From there I ran some initial experiments, implemented a basic system and started learning the complexities of the discipline.

It's strange to think that I'm writing about a decision I made several years ago.  I've tried to be as honest as I can remember in this post.  The PhD has not been a particularly easy road, but it has been a good one so far.

The Choices of a Simplification System

There are many choices to make when building an LS system. In my experience there are three big decisions to take: the target audience; the source documents; and the mode of presentation. Let's look at each of these in detail.

Target Audience

Firstly, you need a well defined group to aim your simplification at.  This group should have a clear style of language, documents written specifically for the group may be useful here.  They should all require a similar form of simplification, otherwise you will be writing several simplification systems.  The group shouldn't be too narrowly defined (e.g. Deaf children in the age range 8-11 with a below average reading age), as this will make it difficult to find test subjects.  It also shouldn't be too broadly defined, otherwise different simplification needs may be present.

Once you have a group to simplify documents for, you're ready to consider the next step.

Source Material

You must decide what type of text to simplify.  It's easy to assume that text is text and you can just build a general model, but in fact different genres have their own peculiarities and jargon.  Consider the difference between a set of news articles, Wikipedia entries and social media posts.  Each will be significantly different in composition than the last.  Of course, the text genre should be one which the target audience wants you to simplify!  That's why this step comes after selecting the target group.  It's also important at this point to check that nobody else has tried to simplify the type of documents that you're working with.

Mode of Presentation

There are, roughly speaking, three ways of presenting the simplifications to the end user.  Which one you choose depends upon factors such as the text itself, the requirements of the user and the reason behind the simplification. Each has advantages and disadvantages as outlined below:

Fully Automated

Substitutions are applied directly to the text where possible.  The user never sees the simplifications being made and so does not need to know that they are reading a simplified document.

Advantages:

• User is presented with a simple document
• Requires minimum work from author / user
• Can be performed on the fly - e.g. whilst browsing the web / reading e-books / etc.

Disadvantages:

• Errors in the simplification process cannot be recovered
• Simplification may alter the meaning away from what the original author intended
• Some words may not be simplified - leaving difficult terms to be seen by the user

Author Aid

The simplifications are presented to the author of the document, who chooses when and how to apply the simplifications.  In this case, the simplification acts similarly to a spell checker.

Advantages:

• Author can make simplifications which retain their original intention
• No chance of grammar mistakes - as the author has the final say of which words to use

Disadvantages:

• Work must be simplified before being published
• No guarantee that the author will apply the suggested simplifications

On Demand

The user has the option to look up simplifications if they find a word too difficult.  These simplifications are typically presented somewhere nearby on the screen.  For example, if a word is clicked on, the simplification may appear in a pop up box or in a dedicated area outside of the body of text.

Advantages:

• User gets to see the original text
• Helps language learning / recovery as the user can decide when they require simplification

Disadvantages:

•  User may struggle if a text or portion of a text has a high density of difficult words
• The user may be distracted by the simplifications, which divert their attention away from the text