Why Lexical?

In this post I will put forward my arguments for picking lexical simplification as a PhD topic.  I'm certainly going to face that dangerous 'why' question when writing up my thesis, not to mention in the viva. I figure that if I commit my reasoning to page now, then I can come back to it at a later date. To convince myself, if not others.

The question of 'why lexical?' needs addressing at two levels.  Firstly, why choose simplification given the wide scope of natural language processing applications to which I could have committed four years of my life?  Secondly, why lexical simplification given the wide range of simplification research out there?  I'll answer these in turn.

Why Simplification?

Let me start with a confession. I didn't start a PhD to study lexical simplification.  I didn't even come with natural language processing in mind.  About four years ago I was offered an opportunity to study at Manchester on the newly founded Centre for Doctoral Training (CDT).  One of the aspects of the CDT programme which appealed to me was the fact that students were not initially tied to a supervisor or research group. We were in effect free for our first six months to decide upon a group, supervisor and topic.

During the first six months, I took three masters modules.  Two on machine learning and one on digital biology.  Both fascinating fields, and both of which have ample opportunities for a PhD.  I spoke to the relevant people about these opportunities, but I just couldn't find something that inspired me.  Resigned to the fact that I may have to choose a topic and wait for the inspiration to come I started to focus on the machine learning research.

We had weekly seminars to acquaint us with the research in the school.  Each seminar was given by a different research group.  They varied in style and form - from the professor who brought in props to talk us through the history of hard drives to the professor who promised free cake every time their group had a major publication.  One week, it was the turn of the text mining group.  During the presentation the idea that text mining could be used to make difficult documents easier to understand was mentioned.  I took the bait, started reading papers, emailed my prospective supervisor and from there I was away.  The spark of inspiration drove me to develop my first experiments, which led to a literature review and an initial study on complex word identification.

Simplification is a great field to be working in.  I like that the research I'm doing could improve a user's quality of life. Ok, the technology isn't quite there yet, but the point of research is to reach out to the unattained. To do something that hasn't been done before. There are lots of opportunities in simplification. Lots of unexplored avenues.  I'm going to write a long future work section in my thesis (maybe a future post?) because there is a lot to say.

Why Lexical?

When I started looking into simplification back in 2011, there were three main avenues that I could discern from the literature.  Firstly, syntactic simplification - automated methods for splitting sentences, dealing with wh- phrases and the passive voice.  Secondly, lexical simplification - the automatic conversion of complex vocabulary into easy to understand words.  Finally, semantic simplification - taking the meaning into account and doing some processing based on this.  I grouped both lexical elaboration and statistical machine translation under this category, although I would probably now put elaboration under the lexical category.

I felt that my research would be best placed if it fell underneath one of those categories.  Although I could see viable research options in all of these, my background in machine learning and data mining was best suited for lexical simplification.  This was also around the time that the SemEval 2012 lexical simplification task was announced.  This task gave me an intial dataset, my background in AI gave me a set of techniques to apply.  From there I ran some initial experiments, implemented a basic system and started learning the complexities of the discipline.

It's strange to think that I'm writing about a decision I made several years ago.  I've tried to be as honest as I can remember in this post.  The PhD has not been a particularly easy road, but it has been a good one so far.

3rd Year

On Wednesday (4/9/2013) I successfully completed my end of second year interview.  This means that I am now officially a third year PhD student.  I am now at the dead halfway point of my PhD, having completed 24 months with 24 more remaining.  It has been a long road getting here and there is still a long way to go.  Below is a brief analysis of the achievements in my PhD so far and the goals yet to come.

Completed So Far:

• Literature Review:  This was the first thing I did as a PhD student.  Reading took up most of the first six months of my research.  I consumed, refined and categorised as much of the relevant literature as I could find.  I am attempting to publish this as a survey paper currently, since the only available text simplification survey is a technical report from 2008
• Lexical Simplification Errors: I recently undertook a pilot study looking at the errors thrown up by the lexical simplification pipeline.  I'm looking to publish this in an upcoming conference, so won't say too much about the results here and now.
• Complex Word Identification: This was the first element of the lexical simplification pipeline that I studied.  I built a corpus of sentences, each with one word marked as complex for the purpose of evaluating current methods of identification.  This work was published in 2 separate workshop papers at ACL 2013.
• Substitution Generation: Once we have identified a complex word, we must generate a set of substitutions for it.  However, those words which are complex are also those which are least likely to be found in a thesaurus, complicating the task.  To address this I spent considerable efforts learning simplifications from massive corpora with some success.  This work is also currently being written up for publication.

Still to come:

• Word Sense Disambiguation: The next step in the pipeline is to apply some word sense disambiguation.  This has been done before, so I will be looking at the best ways to apply it and hopefully making a novel contribution here.  I am just starting out on this phase of research and am currently immersed in the WSD literature, trying to get my head round the myriad techniques that already exist there.
• Synonym Ranking: I have looked into the best way to rank synonyms according to their complexity before at the start of my project.  The small amounts of work that I did back then did not discover anything radical, but did help me to better understand the structure of a lexical simplification system.  When I revisit this area it will be with the hope of making some significant contribution.  I was really interested in the work David Kauchak presented at ACL 2013 and will be interested to explore what more can be done in this area.
• User Evaluation: Finally, I will spend some time exploring the effects of each of the modules I have developed on individual users.  It is of paramount importance to evaluate text simplification in the context of the users it is aimed at and to this end I will be focussing my research on a specific user group.  Although which group is as yet undecided.
• Thesis: This will undoubtedly take a significant portion of my final year.  The chapter titles will hopefully be the bullet points you see listed above.

So there you have it.  Although it appears that I have done a lot so far, it still feels like I have a real mountain to climb.  There are significant hurdles and vast amounts of reading, researching and writing ahead.  I look forward to the challenges that the next two years of my PhD will bring.