Django is a popular Python web framework. I’ve been asked a couple of times how Python based applications can be automated so here I’ll give an example.
We’ll be using Chef. Before we get started let’s discuss most of the items that make Chef’s ecosystem.
- Recipe - a set of steps to make something happen. Just like a recipe for cake. In our case think of it as the recipe of how to build our application or install some dependencies.
- Cookbook - well.. a collection of recipes. Plus few more things we’ll be playing with shortly.
There are different commands like
knife but we won’t discuss them in much detail. Now we’ll go over one cookbook used to deploy the following Django app:
It’s a real world example of a web application that I found not long ago. It has most of the things you would see in such app - database, frontend, some backend.
All Chef code we are going to use to deploy this application is here:
- Chef DK or Chef Workstation
The Chef Cookbook Structure - recipes
The most important files we’ll be looking at are located at the recipes folder.
Here’s what recipes we have in their order of execution:
- install_packages - this one installs some package dependencies using apt-get
We’ll be deploying to Ubuntu. It’s of course an option to support multiple Linux families like Arch, RedHat and Debian in the same cookbook but that’s a more advanced setup we won’t cover here.
- create_user - creates django user and group
- pyenv - setups python virtual environment
- app - deploy the application
Here’s one example of a recipe:
Here we execute commands but you would often employ Chef’s functions. Like in here:
Notice that we’re using a resource called
git which comes with Chef’s namespace and handles git related work. If you want to learn more about the git resource go read the docs here:
Every time you need to write something you can check for pre-defined resource you can use and build things up from there. Of course you can also use Ruby directly. I would prefer to use pre-defined resources because this often saves me a lot of time and I know that those implementations are safe and tested. They are probably much better than what I would come up with now because I’ll be using it in just my cookbook while the pre-defined resources are used by the whole community.
The Chef Cookbook Structure - tests
Just like with software development it’s a good idea to write tests. Tests that cover the functionality of our cookbook are critical for the long-term success of our CI/CD pipelines. It will help us make future changes easier and feel more secure that we won’t break the deployment chain of events and impair our production releases.
Tests are located in their test directory and in our case they cover one to one each recipe.
Here’s how a test looks like:
Notice that the whole syntax is pure Ruby. The interesting part is the
describe method. For the testing part you can start with ChefSpec which is the main facility for writing Chef tests. You can find it here:
There is more to testing. You can mock all kinds of objects that you don’t normally have. Imagine for example that in production you need a mysql database. Or maybe Kafka or Zookeeper. They will provide some service to your application that during testing you don’t need to have. You can safely mock it and make your tests run independently and quickly which is one of the most important thing about tests.
Don’t go and engineer your whole recipes just so you can test them. If you put data in your database during a recipe mock that data up in your test and expect it to be there while you are smoke testing your recipe. That way you’ll know that when you call the real database your smoke will indeed check this data and validate it the way your described.
OK, we have some recipes and tests. How can we really see them run? That’s the fun part. Let’s go and checkout Kitchen.
Testing Cookbooks with Kitchen - configuration
The first thing to do when you want to develop Chef cookbook is install Chef DK or nowadays you can also take advantage of the great Chef Workstation. It has some advantages but it’s still in its early days. Both tools will have the
Here’s our Kitchen definition file called .kitchen.yml:
It’s your door to quick and easy development so let’s see what each section does.
- Driver - that’s the hypervisor you’ll be using when Kitchen runs. In our case we’ll use Virtualbox via a tool called Vagrant.
- Provisioner - that’s our automation tool and while we write Chef Cookbooks we’ll be using chef_zero.
- Verifier - testing facility.
- Platform - what image we will use. As I said we’ll be using Ubuntu image.
- Suites - how many different systems (or VMs in our case) we will build to test with. Here we also put every recipe we want to run.
Testing Cookbooks with Kitchen - execution
kitchen create- creates the VM.
kitchen converge- runs the Chef recipes as defined in the Kitchen configuration file and created in our Cookbook.
kitchen verify- run tests.
kitchen destroy- destroys the VM.
This is one nice order of execution while you’re building things up. If you want to quickly test you can also do:
kitchen test- creates VM and run tests.
If you have a VM created via
kitchen create you can also login to the VM.
kitchen login <name>- logins to a suite. In our case since we have only one we can just say
kitchen loginor if we feel more expressive we can still say
kitchen login default.
In this article you saw how to:
- Create Chef Cookbook recipes
- Create Chef Cookbook tests
- Use Kitchen to aid you while creating or testing your cookbooks