Being a developer in Tryolabs involves playing with new technologies and pushing them to their limits. Last month I was assigned to work in the development of an interesting vertical search engine that was created by part of our team a while ago. The tool is being used by a couple of our clients to retrieve public data from the internet, but now it was needing some new features and a few adjustments to make it more valuable to its users.

Project Description

Vertical Search, as its name says, is a vertical search engine where you can set a list of keywords to look for, a blacklist of urls not to search, some initial links where you want to start looking and allowed and denied domains among other things. It then returns a set of web-pages that contain any of your keywords sorted by relevance. A cool feature of this project is that you can schedule searches, this way you can set the search to run everyday at 2pm if you wish, and you can look at the results later.

Vertical Search is used a lot for e-commerce related data and mostly for getting information about companies, organizations, places... all kinds of entities. Once VS provides the data you need, you are able to use the power of machine learning and Python to develop analytics and get powerful insights.

The Problem

We love Django just as much as the next guy, but we know it has its limitations. In this project in particular we found ourselves in a tight spot when trying to improve user interface related features or adding complex things to the template. We wanted to keep having the power of Django and Python in the backend, but we needed a more flexible frontend in order to keep the project growing with more interesting interactive features.

In modern projects for other clients we found out that having a Django powered API with an AngularJS frontend is a great compromise of backend and frontend capabilities, so we decided to migrate the Vertical Search project's frontend from its Django based templating to an AngularJS powered one.

Learning AngularJS

Since I'd never worked with AngularJS before, the rest of Tryolabs' team recommended I take the Code School Tutorial or the Official AngularJS Tutorial. I've spent the last three years working with Django so I was really looking forward to learn this new framework. I decided to take the Code School tutorial because it is fast and easy, it has videos, slides and some exercises to practice what you've learned. It's a good tutorial that gives you an idea of all the power of AngularJS. Still, after finishing it, I wasn't feeling ready to start the project. So I found a couple of pages with some good practices on AngularJS that help me start coding, such as: Angular Best Practice for App Structure, Opinionated AngularJS styleguide for teams.

Starting the project

There are a couple of tools that help start an AngularJS application, I'm talking about Yeoman, Gulp and Bower. Yeoman is a plugin that generates all the project structure, Gulp is a task/build runner and Bower a package manager. To start my application I used this Yeoman generator for AngularJS with Gulp that you can find on this Github Repo.

It's pretty simple, first you need to install Yeoman, Gulp and Bower, then the generator for AngularJS projects, and when you have everything installed it's time to create your project running the command yo gulp-angular [app-name].

To create the project, the generator will ask us a few questions to customize your project, for example which version of AngularJS you want to use. Some customization I recommend is when it asks which REST resource library to use select ngResource over Restangular, and when it asks for a router select UI Router over ngRoute. I selected ngResource because is part of AngularJS core and it's pretty similar to Restangular.

While ngRoute is part from the AngularJS core, I recommend you use the third-party module UI Router, first of all because it sets a state to every url you define, so later when you want to redirect to a certain url or on a template create a link you can forget about the url and just do this by using the name of the state, this is pretty similar to what you are used to on Django, with the reverse() and the url template tag.

There are a couple more features that make you choose UI Router over ngRoute, you can see them on this post: Difference between angular-route and angular-ui-router.

Going back to setting up your project, after selecting all the customizations that you want will have a running app with everything you need to start, including the structure. You can test it running the command gulp serve that launches a browser sync server on  your source files.

Migrating Django to AngularJS

AngularJS is based on data-binding for showing values in its view. So, how do we migrate our Django templates in order to show Angular data?

The first thing you need is an API that manages all the backend that you currently have, luckily for me Vertical Search already has an API developed using Django REST Framework. We had it that way in order to separate frontend and backend from scratch. The idea was to connect easily to a mobile frontend or other platforms using the same amazing Python backend.

You don't have one? Maybe you should start by building one. If you have no clue or want to know more about Django REST Framework you can take a look at our previous entry about it.

After you have your API, or at least the most important calls that in my case are getting the searches from a user, the CRUD of a search and getting the results of a run of a specific search, you can start the migration.

I think that, since we are new at this, we should migrate one view at a time. On my case I started with the create search page, that is a simple form with validation. To migrate this page we need to migrate the Django view to an AngularJS controller, migrate the template and declare the url state on index.js file.

Let's start by creating the controller addSearchController based on the Django view addSearch, here is where I define on the scope of the page, our form initialized and the submit action of the form, to complete the submit action I had to create the SearchData resource. Here is the code of the factory that does this:

If you haven't read anything about $resource, you should know that with only defining it like it is above it comes by default with get, save, query, remove and delete actions. You can read more about it here.

So now that I have the resource defined I can make the POST API call to save the search, just by creating the SearchDataresource object with the form data and save it.

After having our controller we need to migrate our template, here what I did was take the Django template and replace the Django template tags to AngularJS tags, for example:

now is:

You can notice how I replace the for template tag with a ng-repeat attribute and the url template tag with the ui-sref attribute.

Last you’ll need to define the url on the index.js file, since we’re using UI Router, to define the url you need to create a state, setting the url, the template and the controller that is going to use. It should be something like this:

Having all our urls as states will help us to redirect or create links to a certain page. For example when we need to change url template tag, you just that simple as setting the ui-sref attribute to the state.

So, What do we have?

At this state we manage our Django models through the API, we migrated our Django views to AngularJS controllers, we migrated the templates and defined all of our urls as states on the index.js file.

If you’ve done this for all your Django views. Congrats! You have migrated your Django app to a AngularJS app.

Final results.

So, as a result after a few days of going over the django views and migrating to AngularJS we ended up with a much more flexible frontend and we were able to do very interesting things like adding js controllers super easy and having much more flexible forms. AngularJS has plenty of resources to build simple yet very powerful frontend features. Here is a great list of AngularJS resources to check.

Right now we are waiting for Angular2, which seems to be a very big change, not only in architecture and paradigm, but it also will change it’s language! We will see how it goes.