Top 10 Python libraries of 2016
Last year, we did a recap with what we thought were the best Python libraries of 2015, which was widely shared within the Python community (see post in r/Python). A year has gone by, and again it is time to give due credit for the awesome work that has been done by the open source community this year. Again, we try to avoid most established choices such as Django, Flask, etc.
Scalable infrastructure in AWS (Part I)
Let’s imagine you have an API running on a single node and you need to implement a new feature that requires some heavy task processing. Obviously you can’t simultaneously process the HTTP request and the task without blocking the web server. Although in Python we have a few alternatives like Celery (check our post on this) or Asyncio (when the heavy task processing is IO bound, future post 😃) to handle this situation, this time we’ll explore a new approach: take advantage of Amazon Web Services (AWS).
Building our site: From Django & Wordpress to a static generator (Part I)
We recently announced the redesign of our website and blog. So far, it has been a great success. The site is a lot faster, SEO is better than ever (signaled by the growth of organic traffic), user bounce rates are down, the amount of visited pages went up as did the duration of the sessions. Hurray! :-) However, the change was much deeper than just a visual revamp. We decided to revise the tech stack that has been powering the site for more than 6 years.
Raspberry Pi + Slack: Our humble contribution to the office's laziness
If there is one issue that permeates every culture, is not wanting to answer the door. We used to have countless hours of discussion about who would do it and led to a myriad of problems: relationships were broken, projects were lost, accidents happened on the way to answer the door phone! (Well, not really. Nothing of this happened, but IT COULD HAVE). This clearly had to be fixed. Thinking a bit, we realized we could ease the annoyance of this task if we could avoid having to do the enormous, unbearable exercise of standing up and walking to the phone.
Top 10 Python libraries of 2015
Update (12/20/2016): here you can find the 2016 edition of this post. As the new year approaches, we often sit back and think about what we have accomplished in 2015. Many of our projects would not have been as successful if it were not for the great work done by the open source community, providing some solid, bullet-proof libraries. Everyone and their grandma seems to be writing top 10 lists, so we couldn’t be less and compiled our own.
How to tame a frenzy of tasks with Celery
Using Celery in our projects is not a constant, but from time to time we need to deal with some kind of asynchronous tasks for example, sending emails, calling APIs, and such. But this time we faced another kind of challenge, we needed to implement a processing intensive pipeline to download tweets, un-short URLs, get the sentiment using MonkeyLearn among other tasks. Celery looks like a perfect tool but, as in other aspects of life itself, it all depends on how you use the tool.
Building a Docker image with templating and initialization support
In my last post about Docker, I outlined a technique I used for having a single container configurable via environment variables, even if the app we are trying to run does not support it by default. I used nginx as an example use case. Today we are going one step further: what if we want to be kind and release said image to the public? Clearly, we have to generalize some steps of the process so we can save users from inheriting the Dockerfile.
Configurable Docker containers for multiple environments
You already know about it: the latest hype is all about application containers and Docker. At Tryolabs we try to keep up to date with latest developments, so we have been using Docker extensively for quite some time; both for development as well as production systems. The idea is that we have all our applications containerized so they and their dependencies can coexist without affecting each other. For example, say your application A depends on version 1.
Elasticsearch Demo: SF crime analysis using significant terms
Significant terms aggregation is one of our favorite features in Elasticsearch. Initially included in the 1.1.0 release (March 2014), it is still an experimental feature. In contrast with other types of Elasticsearch aggregations, which generally do simple counts, sums, and other simple math operations, significant terms tries to find statistical anomalies in your data, or as they call it, finding the uncommonly common. There are plenty of excellent resources on what significant term aggregations does, and some practical examples on how to use it.
Optimizing Elasticsearch (part 1): Rescoring queries with multi-type fields
Sometimes you want to run complex real-time queries on hundreds of millions of documents, and you’ll quickly find out the limits of what your hardware can do. Some complex queries may involve indexing fields in multiples ways, by applying different analyzers. For example, you may want to index a field applying stemming with one analyzer, using the shingle token filter with another and indexing with the standard token filter. Luckily there is an easy way to do this in Elasticsearch using multi-type fields.