5 Lessons from a Research Project

Thoughts | Uncategorized By 4 years ago

I recently completed a 3 month R&D project involving computer vision and machine learning. Sadly, the problem area did not have any direct libraries/frameworks and there was a serious lack of ‘do this’ then ‘do that’ guides.

We started with OpenCV, a bunch of academic papers on various algorithms, lots of assorted Stackoverflow posts, a big open ended project and 10 weeks to churn out the first deliverable.

Here is what I learnt — (i) Google is your friend — but only if you know what to ask, (ii) Prototyping and rapid iterations are vital to explore the many unknowns (you know, that agile thing), (iii) Divide then Conquer the problem — but be prepared to change the division strategy, (iv) Data rules — Big Data is King and (v) Some software engineering principles are worth following, even in prototypes. Curious? Read on….

Google is your friend

Or at least it can be if you know what to search.

Our project was in a domain with many terms. Learning these terms and what they meant helped us discover the various algorithms that could help solve the problem.

Libraries also tend to use the language of the domain making them easier to use once the terminology is understood.

That Agile thing

Agile is all the rage these days but does it make sense in a research environment? Turns out it does.

The only difference is iterating on customer feedback vs iterating to test as many solutions/ideas as possible. Open ended problems, by their nature, do not have a single best solution. That is why it is essential to try as many of them as you can as some will be better than others.

Divide then conquer the problem

Oldest strategy in the book, divide and conquer only issue is working out how to divide the problem.

As we started working on the sections of the initial breakdown it soon became clear that we needed to break some items down even further. The small tasks became goals in their own right and we realised that a combination of these small tasks would help solve one aspect of the problem at hand.

Chipping away at the small goals also helped motivation, got us closer to the end goal and helped us show progress to other stake holders.

Data rules

Machine Learning (ML) and Big Data tend to go hand in hand — is this really necessary? If you start with Big Data then you do not need ML but if you want to use ML then you need Big Data.

Large amounts of sample data is needed if you plan to use ML techniques. The more data you train a system on the more that it can recognise.

There are alternatives to ML techniques. A heuristic or rule based approach can sometimes be good enough. The idea being that the developer hard codes various rules into the system to “train” it.

As with all things, it depends on the problem when choosing the correct approach. Whatever you do though make sure you have plenty of data.

Track input/output

Over the course of the project we created up to 40+ magic numbers using a heuristics based approach. In hindsight we should have put all these values in configuration files.

Tweaking becomes easier and it simplifies the process of monitoring the affects on the output. Another benefit is that you can clearly see progress as the output (hopefully) gets closer to what you want it to be.

Well that is it for now. Hope you have learned something that will help you on your next research project.