• LOGIN
  • No products in the cart.

Using Python Fabric to AGNU/Linux Server Configuration Tasks

Fabric (http://www.fabfile.org) is a Python library and command-line tool for automating tasks of application deployment and system administration via SSH. It provides tools for executing local and remote shell commands, and for transferring files through SSH and SFTP, respectively. With these tools, it is possible to write application deployment or system administration scripts, which allows performing these tasks by the execution of a single command.

To work with Fabric, you should have Python and the Fabric Library installed on your local computer, and we will consider using a Debian-based distribution on the examples within this article (such as Ubuntu, Linux Mint, and others).

What you will learn…

In this article, we will run through the basics of using Fabric to run local and remote commands and transfer files between servers. Through some examples, we will try to show some of its capabilities and use cases.

What you should know…

You are not required to have any prior experience to follow this article. However, if you are used to performing SSH system administration or application deployment tasks, it will be easier to realize some practical use cases.

As Python is shipped by default on most of the GNU/Linux distributions, you probably won’t need to install it. Regarding the Fabric Library, you may use pip to install it. Pip is a command-line tool for installing and managing Python packages. On Debian-based distributions, it can be installed with apt-get via the python-pip package:

 $ sudo apt-get install python-pip

After installing it, you may update it to the latest version using pip itself:

 $ sudo pip install pip --upgrade

After that, you may use pip to install Fabric:

 $ sudo pip install fabric

To work with Fabric, you must have SSH installed and properly configured with the necessary user’s permissions on the remote servers you want to work on. In the examples, we will consider a Debian system with an IP address 192.168.250.150 and a user named “administrator” with sudo powers, which is required only for performing actions that require superuser rights.

One way to use Fabric is to create a file called fabfile.py containing one or more functions that represent the tasks we want to execute. For example, take a look at Listing 1.

Listing 1: A basic fabfile. File: fabfile.py

# -*- coding: utf-8 -*-

from fabric.api import *

env.hosts = ['192.168.250.150']

env.user  = 'administrator'

def remote_info():

   run('uname -a')

def local_info():

   local('uname -a')

In this example, we have defined two tasks called “remote_info” and “local_info”. They are used to retrieve local and remote systems information through the command “uname -a”. Also, we have defined the host user and address we would like to use to connect to the remote server using a special dictionary called “env”.

Having defined this, it is possible to execute one of the tasks using the shell command fab. For example, to execute the task “local_info”, from within the directory where fabfile.py is located, you may call:

 $ fab local_info

This gives the output shown on Listing 2.

Listing 2: output of fab local_info

[192.168.250.150] Executing task 'local_info'

[localhost] local: uname -a

Linux renato-laptop 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Similarly, you could execute the task called “remote_info”, by calling:

 $ fab remote_info

In this case, Fabric will ask for the password of the user “administrator”, as it is connecting to the server via SSH, as shown in Listing 3.

Listing 3: output of fab remote_info

[192.168.250.150] Executing task 'remote_info'

[192.168.250.150] run: uname -a

[192.168.250.150] Login password for 'administrator':

[192.168.250.150] out: Linux debian-vm 2.6.32-5-686 #1 SMP Sun May 6 04:01:19 UTC 2012 i686 GNU/Linux

[192.168.250.150] out:

Done.

Disconnecting from 192.168.250.150... done.

 

There are lots of parameters that can be used with the fab command. To obtain a list with a brief description of them, you can run fab –help. For example, by running fab -l, it is possible to check the Fabric tasks available on the fabfile.py file. Considering we have the fabfile.py file shown on Listing 1, we obtain the output of Listing 4 when running fab -l.

Listing 4: output of fab -l

Available commands:

   local_info

   remote_info

As was in the previous example, on the file fabfile.py, the function “run()” may be used to run a shell command on a remote server and the function “local()” may be used to run a shell command on the local computer. Besides these, there are some other possible functions to use on fabfile.py:

  • sudo(‘shell command’): To run a shell command on the remote server using sudo.
  • put(‘local path’, ‘remote path’): To send a file from a local path on the local computer to the remote path on the remote server.
  • get(‘remote path’, ‘local path’): To get a file from a remote path on the remote server to the local path on the local computer.

Likewise, it is possible to set many other details about the remote connection with the dictionary “env”. To see a full list of “env” vars that can be set, visit http://docs.fabfile.org/en/1.6/usage/env.html#full-list-of-env-vars. Among the possible settings, it’s worth to spend some time commenting on some of them:

  • user: defines which user will be used to connect to the remote server.
  • hosts: a Python list with the addresses of the hosts that Fabric will connect to perform the tasks. There may be more than one host, e.g.

env.hosts = [‘192.168.250.150′,’192.168.250.151’]

  • host_string: with this setting, it is possible to configure a user and a host at once, e.g.    env.host_string = “[email protected]

As it was noticed from the previous example, Fabric will ask for the user’s password to connect to the remote server. However, for automated tasks, it is interesting to be able to make Fabric run the tasks without prompting for any user input. To avoid the need of typing the user’s password, it is possible to use the env.password setting, which permits to specify the password to be used by Fabric, e.g.

env.password = ‘mysupersecureadministratorpassword’

If the server uses SSH keys instead of passwords to authenticate users (actually, this is a good practice concerning the server’s security), it is possible to use the setting env.key_filename to specify the SSH key to be used. Considering that the public key ~/.ssh/id_rsa.pub is installed on the remote server, you just need to add the following line to fabfile.py:

 env.key_filename = '~/.ssh/id_rsa'

It is also a good security practice to forbid root users from logging in remotely on the servers and allow the necessary users to execute superuser tasks using the sudo command. On a Debian system, to allow the “administrator” user to perform superuser tasks using sudo, first you have to install the package sudo, using:

 # apt-get install sudo

Thereafter, add the “administrator” user to the group “sudo”, which can be done with this command:

 # adduser administrator sudo

Having done this, you could use the sudo() function on Fabric scripts to run commands with sudo powers. For example, to create a mydir directory within /home, you may use the fabfile.py file shown in Listing 5.

Listing 5: Script to create a directory. File: fabfile.py

# -*- coding: utf-8 -*-

from fabric.api import *

env.hosts = ['192.168.250.150']

env.user  = 'administrator'

env.key_filename = '~/.ssh/id_rsa'

def create_dir():

   sudo('mkdir /home/mydir')

and call

 $ fab create_dir

This will ask for the password of the user “administrator” to perform the sudo tasks, as shown in Listing 6.

Listing 6: output of fab create_dir

[192.168.250.150] Executing task 'create_dir'

[192.168.250.150] sudo: mkdir /home/mydir

[192.168.250.150] out:

[192.168.250.150] out: We trust you have received the usual lecture from the local System

[192.168.250.150] out: Administrator. It usually boils down to these three things:

[192.168.250.150] out:

[192.168.250.150] out:     #1) Respect the privacy of others.

[192.168.250.150] out:     #2) Think before you type.

[192.168.250.150] out:     #3) With great power comes great responsibility.

[192.168.250.150] out:

[192.168.250.150] out: sudo password:

[192.168.250.150] out:

Done.

Disconnecting from 192.168.250.150... done.

When using SSH keys to log in to the server, you can use the env.password setting to specify the sudo password to avoid having to type it when you call the Fabric script. In the previous example, by adding,

 env.password = 'mysupersecureadministratorpassword'

it would be enough to make the script run without the need of user intervention.

However, some SSH keys are created using a passphrase, required to log in to the server. Fabric treats these passphrases and passwords similarly, which can sometimes cause confusion. To illustrate Fabric’s behavior, consider the user named “administrator” can log in to a remote server only by using his/her key named ~/.ssh/id_rsa2.pub, created using a passphrase, and the Fabric file shown in Listing 7.

Listing 7: Example fabfile using an SSH key with a passphrase. File: fabfile.py

# -*- coding: utf-8 -*-

from fabric.api import *

env.hosts = ['192.168.250.150']

env.user  = 'administrator'

env.key_filename = '~/.ssh/id_rsa2'

def remote_info():

   run('uname -a')

def create_dir():

   sudo('mkdir /home/mydir')

In this case, calling

fab remote_info

makes Fabric ask for a “Login password”. However, as you shall notice, this “Login password” refers to the necessary passphrase to log in using the SSH key, as shown in Listing 8.

Listing 8: Output of fab remote_info

[192.168.250.150] Executing task 'remote_info'

[192.168.250.150] run: uname -a

[192.168.250.150] Login password for 'administrator':

[192.168.250.150] out: Linux debian-vm 2.6.32-5-686 #1 SMP Sun May 6 04:01:19 UTC 2012 i686 GNU/Linux

[192.168.250.116] out:

Done.

Disconnecting from 192.168.250.150... done.

In this case, if you specify the env.password setting, it will be used as the SSH passphrase and, when running the create_dir script, Fabric will ask for the password of the user “administrator”. To avoid typing any of these passwords, you may define env.password as the SSH passphrase and, within the function that uses sudo(), redefine it as the user’s password, as shown in Listing 9.

Listing 9: Example fabfile using an SSH key with a passphrase. Improved to avoid the need of user intervention. File: fabfile.py

# -*- coding: utf-8 -*-

from fabric.api import *

env.hosts = ['192.168.250.150']

env.user  = 'administrator'

env.key_filename = '~/.ssh/id_rsa2'

env.password = 'sshpassphrase'

def remote_info():

   run('uname -a')

def create_dir():

   env.password = 'mysupersecureadministratorpassword'

   sudo('mkdir /home/mydir')

Alternatively, you could specify the authentication settings from within the task function, as shown in Listing 10.

Listing 10: Another example fabfile using an SSH key with a passphrase. Improved to avoid the need for user intervention. File: fabfile.py

# -*- coding: utf-8 -*-

from fabric.api import *

env.hosts = ['192.168.250.150']

def create_dir():

   env.user  = 'administrator'

   env.key_filename = '~/.ssh/id_rsa2'

   env.password = 'sshpassphrase'

   run(':')

   env.password = 'mysupersecureadministratorpassword'

   sudo('mkdir /home/mydir')    

On this example, the command “:” does nothing. It only serves as a trick to enable setting env.password twice: first for the SSH passphrase, required for login and then to the user’s password, required for performing sudo tasks.

If necessary, it is possible to use Python’s with statement (learn about it on http://www.python.org/dev/peps/pep-0343/), to specify the env settings. A compatible create_dir() task using the with statement is shown in Listing 11.

Listing 11: Example using Python’s with statement. File: fabfile.py

# -*- coding: utf-8 -*-

from fabric.api import *

env.hosts = ['192.168.250.150']

def create_dir():

   with settings(user  = 'administrator',

                 key_filename = '~/.ssh/id_rsa2',

                 password = 'sshpassphrase'):

       run(':')

       env.password = 'mysupersecureadministratorpassword'

       sudo('mkdir /home/mydir')

The fab command is useful for performing system administration and application deployment tasks from a shell console. However, sometimes you may want to execute tasks from within your Python scripts. To do this, you may simply call the Fabric functions from your Python code. To build a script that runs a specific task automatically, such as create_dir() shown previously, you’ll need to create a Python script as shown in Listing 12.

Listing 12: Python Script using Fabric. File: mypythonscript.py

#! /usr/bin/env python

# -*- coding: utf-8 -*-

from fabric.api import *

def create_dir():

   with settings(host_string  = '[email protected]',

                 key_filename = '~/.ssh/id_rsa2',

                 password = 'sshpassphrase'):

       run(':')

       env.password = 'mysupersecureadministratorpassword'

       sudo('mkdir /home/mydir')  

if __name__ == '__main__':

   create_dir()

As we have seen, with Fabric, it is possible to automate the execution of tasks that can be done by executing shell commands locally, and remotely, using SSH. It is also possible to use Fabric’s features on other Python scripts and perform dynamic tasks, enabling the developer to automate virtually anything that can be automated. The main goal of this article was to show Fabric’s basic features and try to show a solution in different scenarios of remote connections, regarding different types of authentication. From this point, you may customize your Fabric tasks to your needs using the functions local(), run(), and sudo() to run shell commands and put() and get() to transfer files.

In conclusion, we show a more practical example of a Python script that uses Fabric to deploy a very basic HTML application on a server. The script shown on Listing 13 creates a tarball from the local HTML files at ~/website, sends it to the server, expands the tarball, moves the files to the proper directory (/var/www/website) and restarts the server. Hope this article helped you learna bit about Fabric to automate some of your tasks!

Listing 13: A very basic deploy example. File: deployhtml.py

#! /usr/bin/env python

# -*- coding: utf-8 -*-

from fabric.api import *

def deploy_html():

   with settings(host_string  = '[email protected]',

                 key_filename = '~/.ssh/id_rsa2',

                 password = 'sshpassphrase'):

       run(':')

       env.password = 'mysupersecureadministratorpassword'

       local('cd ~; tar -czvf website.tar.gz ./website/*')

       put('~/website.tar.gz', '~')

       run('tar -xzvf ~/website.tar.gz')

       sudo('mv /home/administrator/website /var/www')

       sudo('chown -R www-data:www-data /var/www/website')

       sudo('/etc/init.d/apache2 restart')

       local('rm ~/website.tar.gz')

if __name__ == '__main__':

   deploy_html()

 

About the author

renato_142_400x400Renato Candido is a free (as in freedom) {software, hardware, and culture} enthusiast, who works as a technology consultant at Liria Technology, Brazil. He tries to solve  peoples’ (technical) problems using these sorts of tools (he thinks the world would be a little better if all resources were free, as in freedom). He is an electronics engineer and enjoys learning things related to signal processing and computer science (and he thinks that there will be self-driving cars and speaking robots designed exclusively with free resources). Contact the author on http://www.renatocandido.org

September 15, 2017

Leave a Reply

Be the First to Comment!

Notify of
avatar
wpDiscuz
© HAKIN9 MEDIA SP. Z O.O. SP. K. 2013