If you have ever tried to follow the Opscode Getting Started Guide for Chef, you’ll quickly be overtaken by Chef jargon, confusing instructions, many assumptions, and no clear direction. Even the most experienced developers had a difficult time following the Opscode Wiki. While it serves as a great reference resource, you pretty much have to know Chef before it is of any use.
In Part 1 of this 2-part series, we will walk through setting up the Chef environment on your local machine. In Part 2, we will demonstrate how to provision your personal laptop using Chef.
What is Chef?
One of the best things about Chef is that it is idempotent. Unlike a simple bash script or complex startup script, Chef can be applied to the same machine hundreds of times. Only the “changes” will be executed on each of the clients. This makes deploying and updating thousands of machines as easy as baking a cake!
Understanding Chef & Jargon
One of the most interesting and confusing parts about Chef is its lingo. Everything relates to food. Be sure to grab a snack before diving in any further.
One of the most dangerous practices when using Chef is applying prior knowledge. In Chef, everything is named relative to Chef. This means, even though your are provisioning a production server, it’s still a Chef client. This can be very confusing at first, but once you start thinking in terms of Chef things will begin to make sense.
(Chef) Server[aka hosted chef] - refers a machines that stores cookbooks, roles, and other Chef configurations
(Chef) Client[aka node] - refers to a machine that connects to and is managed by a Chef Server
(Chef) Workstation- refers to a machine where recipes are developed, altered, deleted, and more
This is the most simplistic picture. It’s possible for a single machine to exist in all three of these roles simultaneously! There are subsets of Chef, such as Chef-Solo, but those will not be discussed here.
A typical scenario begins at the workstation. A developer creates a cookbook, role, or other artifact on the local machine. When finished, that artifact is uploaded to the Chef Server. The Chef Clients receive these (new) instructions and execute them locally.
We’ve also been throwing around some other terms that we should take a second to define:
cookbook- collection of recipe, resources, attributes, templates, metadata, and other files
recipe- a set of instructions written in a Ruby DSL that tells a node what commands to execute
metadata- additional information, such as dependencies, for a given cookbook
resource- cross-platform abstraction of the “thing” you’re configuring such as a package or a user
provider- a platform-specific implementation of a resource
data bag- JSON key-value store for storing data, attributes, and more
environment- provide a mechanism for managing different deployment locations such as production, staging, development, and testing
template- a file (like a config file) to be “rendered” on the server
There are also a few tools:
knife- a command line tool for managing chef and chef recipes
shef- chef console. this is the equivalent of
rails consolefor chef
Before continuing, please make sure you understand that these are very over-simplified definitions and abstractions. Chef is much more powerful that these simplistic definitions allow.
Set up your Opscode Account
For the purposes of this tutorial, we will use Chef Hosted by Opscode (the creators of Chef). The service provides 5 free nodes for use. Go ahead and create your free account by heading over to the Hosted Chef Signup page. You should see something like this:
You’ll need to confirm your email, but once that’s done, head on over to the Opscode Management Console. It really doesn’t matter what you call your organization - I used the same as my username. You should see a screen like this:
You probably only have one organization, but the image should give you a good idea. First thing you’ll want to do is Regenerate validation key and Generate knife config. We will talk about this more in detail, but save these files in a handy place for later. These files should always be kept securely.
Also, if you didn’t get your private key when registering, you should do that now (for some reason, Opscode does not always stream the key). Go to the Opscode Community Site and login (if you aren’t already). Click on your profile and then choose get private key.
At this point you should have the following files:
[your_organization_name]-validator.pem [your_username].pem knife.rb
For example, mine would be:
ci-validator.pem sethvargo.pem knife.rb
Sometimes Opscode doesn’t stream the correct files, so you may need to do some renaming:
_knife_config #=> knife.rb _regenerate_key #=> [your_organization_name]-validator.pem
Install Ruby, Chef, and Git
Chef is written in Ruby. Therefore, you must have Ruby installed. Installing Ruby is beyond the scope of this topic, but here are some quick resources:
- Ruby Installer for Windows (also install the Development Kit)
- Installing Ruby with rvm
- Installing Ruby with rbenv
You should also install
git. Check out the Github tutorial for installing git.
Once you have Ruby and Git installed, you’ll need to install chef:
gem install chef
This command will install the Chef gem as well as some other dependencies.
A this point, you should have a novice understand of Chef and Chef Jargon, have an account on Opscode Hosted Chef, and have a working version of Ruby, Git, and the Chef gem installed. The rest of this guide will assume you have completed all those steps correctly.
Setup your Workstation
Find a working directory on your local machine where you plan to store your Chef cookbooks. For this tutorial, we will use
~/Development. You’ll need a skeleton chef repository. You could make your own or clone the Opscode one:
git clone firstname.lastname@example.org:opscode/chef-repo.git
Take a few minutes to poke around the repository. You definitely won’t understand everything, but look at a few READMEs (in subdirectories).
chef-repo directory (
~/Development/chef-repo), we need to create a hidden folder named
cd ~/Development/chef-repo mkdir .chef
This is where you should put the files we downloaded earlier:
mv [location]/[your_username].pem ~/Development/chef-repo/.chef/ mv [location]/[your_organization_name].pem ~/Development/chef-repo/.chef/ mv [location]/knife.rb ~/Development/chef-repo/.chef/
For my case, it would be:
mv ~/Desktop/sethvargo.pem ~/Development/chef-repo/.chef/ mv ~/Desktop/ci-validator.pem ~/Development/chef-repo/.chef/ mv ~/Desktop/knife.rp ~/Development/chef-repo/.chef/
To confirm everything is working, try and list all the clients (it should be empty):
knife client list
If this command succeeds without error, everything is set up correctly!
This concludes setting up your workstation. Now we are ready to create our first cookbook.
Create your first cookbook
Wash your hands, put on your Chef’s Hat, and get ready to bake! We are going to create your very first cookbook. The cookbook will be very simple and is more for demonstrating working with
Our cookbook will be named “hello world”. Let’s start by using
knife to create our cookbook skeleton:
cd ~/Development/chef-repo knife cookbook create hello_world
You should see the following output:
** Creating cookbook hello_world ** Creating README for cookbook: hello_world ** Creating metadata for cookbook: hello_world
Open up the project in your favorite text editor and look inside the
cookbooks directory. You should see the following:
This cookbook will create a file
~/hello_world.txt that says “Hello World!”.
Let’s first make the template. A template is like the “view” of MVC. It has access to instance variables and uses embedded ruby. Inside the
templates/default directory, create a new file named
hello-world.txt.erb and add some content:
1 2 3 4 5 6
You can see we are referencing a
node variable. This refers to the current client that Chef is running on.
Now we need to create the actual recipe with instructions. You can think of the recipe as the “controller” of MVC. Open up the
recipes/default.rb file and add the following:
1 2 3 4 5
template keyword is a resource like we defined before. It tells the recipe to render the given template source to the given file. We also specify the file permissions using
That’s it! Our cookbook is done and ready to be uploaded to Hosted Chef. We will use the
knife command to do this:
knife cookbook upload hello_world
You should see output like this:
Uploading hello_world [0.0.1] Uploaded 1 cookbook.
If you get a message like:
WARNING: No knife configuration file found ERROR: Your private key could not be loaded from /etc/chef/client.pem Check your configuration file and ensure that your private key is readable
it means that your key is invalid. Regenerate your personal and/or organization keys and ensure everything is placed in the correct directories.
Our cookbook is now on Hosted Chef and ready to be distributed to our nodes. Chef doesn’t automatically tell nodes to update. You can do this with a cron job, running
chef-client as a service, or by running
chef-client manually on any node.
For simplicity, let’s just run this on our local workstation. Let’s set up the local workstation as a client. Run the following command from the inside the repository:
sudo knife configure client /etc/chef
If you’re using
rvm, use the
rvmsudo command prefix:
rvmsudo knife configure client /etc/chef
You should see output like:
Creating client configuration Writing client.rb Writing validation.pem
That’s it! Your workstation is now a client. Let’s provision this server by running
Or with RVM:
You should see output like this:
INFO: *** Chef 0.10.10 *** INFO: Client key /etc/chef/client.pem is not present - registering INFO: Run List is  INFO: Run List expands to  INFO: Starting Chef Run for NODE INFO: Running start handlers INFO: Start handlers complete. INFO: Loading cookbooks  WARN: Node NODE has an empty run list. INFO: Chef Run complete in 0.988165 seconds INFO: Running report handlers INFO: Report handlers complete
Take a look inside your
$HOME directory and see if the
hello-word.txt file was created…
It doesn’t appear the file was created. We must have done something wrong! Actually, we did everything correctly. We just forgot one step - we never told our node to execute the recipe we just wrote. By default, Chef does not execute any of your recipes. You must explicitly require them. There are a variety of ways to do this. We will use a
First, we need to figure out what our node is named. Run
knife node list and you should now see two nodes:
Mine looks like:
We obviously want
NODE. In my case, it’s
seth. Now we can edit the
run_list for that node:
knife node run_list add NODE hello_world
For me, that command would look like:
knife node run_list add seth hello_world
You should see the following output:
Let’s inspect this node using the
knife node show NODE
You should now see
hello_world in the
run_list. Run the
chef-client command again (remember you might need to use
sudo) and you should get output like the following:
INFO: *** Chef 0.10.10 *** INFO: Run List is [recipe[hello_world]] INFO: Run List expands to [hello_world] INFO: Starting Chef Run for seth INFO: Running start handlers INFO: Start handlers complete. INFO: Loading cookbooks [hello_world] INFO: Storing updated cookbooks/hello_world/recipes/default.rb in the cache. INFO: Processing template[/Users/seth/hello-world.txt] action create (hello_world::default line 10) INFO: template[/Users/seth/hello-world.txt] mode changed to 644 INFO: template[/Users/seth/hello-world.txt] updated content INFO: Chef Run complete in 1.188536 seconds INFO: Running report handlers INFO: Report handlers complete
Open up your
$HOME directory and you should see a file named
hello-world.txt. Look inside and you’ll see the those node variables were translated into plain text. Awesome!
You can delete that file or leave it around as a reminder of how awesome Chef is. We have one last thing to do before we are done with this tutorial.
Let’s remove that recipe from the
knife node run_list remove NODE hello_world
Again, mine would be:
knife node run_list remove seth hello_world
You should see output like the following:
Well, this concludes this (rather lengthy) tutorial on installing Chef, registering for Hosted Chef, creating your first cookbook, and provisioning your first machine.
Part 2 of this series will cover more recipes and full provisioning and customization of your personal laptop using Chef.