[ad_1]
You’ve been automating your repetitive sysadmin duties with Ansible for some time now. You’ve bounced troublesome providers, deployed objects to Kubernetes, up to date methods, and carried out rolling restarts. Possibly you might have even deployed and configured providers utilizing roles you discovered on Ansible Galaxy. Nevertheless, it’s lastly occurred. You possibly can’t discover a position on Ansible Galaxy that does what you need it to (you checked, proper?), or you might have been tasked with writing a job on your group’s app. Regardless of the cause, it’s time so that you can write your individual position. How do you try this in a means the place you might be assured your position works as supposed? This publish will reply that query by offering steering on the best way to greatest start growing Ansible roles.
Infrastructure as Code and DevOps
Earlier than we dive into specifics, it could be useful to outline some terminology. Ansible falls underneath the infrastructure as code (IaC) device umbrella. IaC is the method of managing and provisioning IT assets by machine readable definition information. This method permits the storing of infrastructure configuration in git, gaining all the advantages of doing so, together with branching, historical past, overview and approval insurance policies, and so forth.
DevOps is the set of practices that mixes software program growth (dev) and IT operations (ops) to shorten the event lifecycle and supply steady supply of software program. IaC instruments are one piece of the puzzle that permit organizations to undertake a DevOps method. The DevOps course of is usually depicted as follows:
Determine 1: The DevOps Infinity Loop.
It’s useful to have this course of in thoughts whereas growing your Ansible roles, as a result of as software program they will additionally profit from a DevOps method.
Ansible Roles
In Ansible, roles are a technique of routinely loading sure variables, duties, information, templates, and handlers based mostly on a identified file construction. Grouping content material by roles permits for simple sharing and reuse. The Ansible documentation on roles outlines the file construction and different issues.
When growing roles, you will have to take care of numerous issues, together with what working system(s) and model(s) you may be supporting and whether or not you solely want a single node or if you need to goal a cluster of machines. Additionally it is usually vital to start out from a recent state each time you rerun your position whereas growing it to make sure that (1) your roles full efficiently on their first run, and (2) modifications carried out on earlier runs aren’t affecting the result. You must also confirm that your position is idempotent to make sure that regardless of what number of instances it’s executed, you obtain the identical end result. Additionally it is vital to confirm that issues ought to solely be modified in the event that they should be modified. For instance, a service ought to solely be restarted if configuration modifications warrant a restart.
The ultimate factor to contemplate is the best way to examine that your position has achieved what you propose it to do. Logging in to a goal node and manually checking is actually one technique to do it. Nevertheless, it’s higher to write down assessments that may be routinely run after your Ansible position to confirm the precise state.
To automate these kinds of assessments, you’ll want a goal host (or set of hosts) and also you’ll have to destroy and recreate that host (or set of hosts) continuously through the growth course of. Additionally, you will be accountable for managing connections to the host (or hosts) in one of many Ansible-supported strategies. As well as, you will need to hand off to your chosen testing device and deal with its connection to the node(s). Managing this growth and testing infrastructure might be tedious and can eat up plenty of the time that could possibly be higher spent on position options.
Enter Molecule
Molecule is a mission for facilitating the event and testing of Ansible roles by dealing with the beforehand outlined set of issues and streamlining your complete position growth course of. Pairing Molecule with Docker because it’s provisioner allows you to shortly and simply develop your roles towards any variety of freshly deployed working methods and variations concurrently. Molecule additionally has built-in idempotence checking and help for quite a lot of verification testing strategies.
To get began, first set up Docker. Then assuming you might have pip3, set up Molecule and supporting dependencies:
pip3 set up yamllint ansible molecule[docker] docker pytest-testinfraWith Molecule and its dependencies put in, it’s time to start growing an Ansible position. Molecule can help right here by constructing out your position’s file construction and Molecule’s personal required config information for you:
$ molecule init position maheckathorn.instance -d docker
$ tree
.
├── README.md
├── defaults
│ └── major.yml
├── information
├── handlers
│ └── major.yml
├── meta
│ └── major.yml
├── molecule
│ └── default
│ ├── converge.yml
│ ├── molecule.yml
│ └── confirm.yml
├── duties
│ └── major.yml
├── templates
├── assessments
│ ├── stock
│ └── take a look at.yml
└── vars
└── major.yml
10 directories, 11 informationInside the Molecule folder, the next information have particular functions:
converge.ymlis the playbook file that comprises the decision on your position. Molecule will invoke this playbook withansible-playbookand run it towards an occasion created by the driving force, which is Docker in our state of affairs.molecule.ymlis the central configuration entrypoint for Molecule. With this file, you may configure every device that Molecule will make use of when testing your position.confirm.ymlis the Ansible file used for testing as Ansible is the default verifier, which lets you write particular assessments towards the state of the container after your position has completed executing. Different verifier instruments can be found (Be aware that TestInfra was the default verifier previous to Molecule model 3).
The molecule.yml file comprises totally different sections for configuring how molecule elements behave:
- The dependency supervisor—Molecule makes use of Galaxy by default to resolve your position dependencies.
- The driver supplier. Molecule makes use of Docker by default. Molecule makes use of the driving force to delegate the duty of making situations.
- The lint command—Molecule can name exterior instructions to make sure that greatest practices are inspired. Be aware: Ansible-lint shouldn’t be included with molecule or molecule[lint].
- The platforms definitions—Molecule depends on this to know which situations to create and identify and to establish which group every occasion belongs in. If that you must take a look at your position towards a number of common distributions (CentOS, Fedora, Debian), you may specify that on this part.
- The provisioner—Molecule solely gives an Ansible provisioner. Ansible manages the lifecycle of the occasion based mostly on this configuration.
- The state of affairs definition—Molecule depends on this configuration to manage the state of affairs sequence order.
- The verifier framework—Molecule makes use of Ansible by default to supply a technique to write particular state checking assessments (equivalent to deployment smoke assessments) on the goal occasion.
There are numerous choices for configuring these sections to satisfy your wants. Nevertheless, sticking to a standard configuration file throughout initiatives helps to set commonplace expectations. The next is predicated on Jeff Geerling’s widespread molecule.yml file:
https://github.com/cmu-sei/ansible-role-silk/blob/grasp/molecule/default/molecule.yml
---
dependency:
identify: galaxy
driver:
identify: docker
platforms:
- identify: occasion
picture: "geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:newest"
command: ${MOLECULE_DOCKER_COMMAND:-""}
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
env:
http_proxy: "${http_proxy}"
https_proxy: "${https_proxy}"
no_proxy: "${no_proxy} "
provisioner:
identify: ansible
playbooks:
converge: ${MOLECULE_PLAYBOOK:-converge.yml}
verifier:
identify: testinfra
choices:
v: 1The important thing factor to notice on this file is the utilization of Jeff Geerling’s Docker container(s) because the picture supply, the configuration of Testinfra because the verifier, and the MOLECULE_DISTRO atmosphere variable with centos7 because the default. The customized premade Docker photos already include Python, Ansible, and systemd. They assist to hurry up take a look at runs by not needing Molecule to do something to make use of the picture aside from fetch it. The MOLECULE_DISTRO atmosphere variable lets you simply take a look at towards different OS varieties and variations by:
$ MOLECULE_DISTRO=ubuntu1804 molecule take a look atDifferent prebuilt photos are listed right here.
Lastly, the Testinfra verifier configures Molecule to make use of Testinfra for verification testing, which was the default previous to molecule model 3. If you happen to’ve been growing roles with verification assessments for some time, it’s helpful to have the ability to configure this setting, which suggests the assessments listing that molecule creates is unneeded. You may as well edit your converge.yml file to appear to be the next:
https://github.com/cmu-sei/ansible-role-silk/blob/grasp/molecule/default/converge.yml
---
- identify: Converge
hosts: all
roles:
- position: "{ basename }"
atmosphere:
http_proxy: "{{ lookup('env', 'http_proxy') }}"
https_proxy: "{{ lookup('env', 'https_proxy') }}"
no_proxy: "{{ lookup('env', 'no_proxy') }}"This configuration helps to keep away from points on steady integration/steady deployment (CI/CD) methods and likewise offers with Ansible mission namespace points.
With Molecule configured, we are able to run by the entire default Molecule state of affairs, which is a take a look at suite on your new position:
$ molecule take a look at
INFO default state of affairs take a look at matrix: dependency, lint, cleanup, destroy, syntax, create, put together, converge, idempotence, side_effect, confirm, cleanup, destroy
INFO Performing prerun...
INFO Set ANSIBLE_LIBRARY=/Customers/maheckathorn/.cache/ansible-compat/50d858/modules:/Customers/maheckathorn/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO Set ANSIBLE_COLLECTIONS_PATH=/Customers/maheckathorn/.cache/ansible-compat/50d858/collections:/Customers/maheckathorn/.ansible/collections:/usr/share/ansible/collections
INFO Set ANSIBLE_ROLES_PATH=/Customers/maheckathorn/.cache/ansible-compat/50d858/roles:/Customers/maheckathorn/.ansible/roles:/usr/share/ansible/roles:/and so forth/ansible/roles
INFO Utilizing /Customers/maheckathorn/.cache/ansible-compat/50d858/roles/maheckathorn.instance symlink to present repository to be able to allow Ansible to seek out the position utilizing its anticipated full identify.
INFO Working default > dependency
WARNING Skipping, lacking the necessities file.
WARNING Skipping, lacking the necessities file.
INFO Working default > lint
INFO Lint is disabled.
INFO Working default > cleanup
WARNING Skipping, cleanup playbook not configured.
INFO Working default > destroy
INFO Sanity checks: 'docker'
PLAY [Destroy] *****************************************************************
TASK [Destroy molecule instance(s)] ********************************************
modified: [localhost] => (merchandise=occasion)
TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for example(s) deletion to finish (300 retries left).
okay: [localhost] => (merchandise=occasion)
TASK [Delete docker networks(s)] ***********************************************
PLAY RECAP *********************************************************************
localhost : okay=2 modified=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
INFO Working default > syntax
playbook: /Customers/maheckathorn/take a look at/instance/molecule/default/converge.yml
INFO Working default > create
PLAY [Create] ******************************************************************
TASK [Log into a Docker registry] **********************************************
skipping: [localhost] => (merchandise=None)
skipping: [localhost]
TASK [Check presence of custom Dockerfiles] ************************************
okay: [localhost] => (merchandise={'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'identify': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})
TASK [Create Dockerfiles from image names] *************************************
skipping: [localhost] => (merchandise={'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'identify': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})
TASK [Discover local Docker images] ********************************************
okay: [localhost] => (merchandise={'modified': False, 'skipped': True, 'skip_reason': 'Conditional end result was False', 'merchandise': {'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'identify': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'merchandise', 'i': 0, 'ansible_index_var': 'i'})
TASK [Build an Ansible compatible image (new)] *********************************
skipping: [localhost] => (merchandise=molecule_local/geerlingguy/docker-centos7-ansible:newest)
TASK [Create docker network(s)] ************************************************
TASK [Determine the CMD directives] ********************************************
okay: [localhost] => (merchandise={'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'identify': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})
TASK [Create molecule instance(s)] *********************************************
modified: [localhost] => (merchandise=occasion)
TASK [Wait for instance(s) creation to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for example(s) creation to finish (300 retries left).
modified: [localhost] => (merchandise={'failed': 0, 'began': 1, 'completed': 0, 'ansible_job_id': '429858788464.21737', 'results_file': '/Customers/maheckathorn/.ansible_async/429858788464.21737', 'modified': True, 'merchandise': {'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'identify': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'merchandise'})
PLAY RECAP *********************************************************************
localhost : okay=5 modified=2 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0
INFO Working default > put together
WARNING Skipping, put together playbook not configured.
INFO Working default > converge
PLAY [Converge] ****************************************************************
TASK [Gathering Facts] *********************************************************
okay: [instance]
PLAY RECAP *********************************************************************
occasion : okay=1 modified=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
INFO Working default > idempotence
PLAY [Converge] ****************************************************************
TASK [Gathering Facts] *********************************************************
okay: [instance]
PLAY RECAP *********************************************************************
occasion : okay=1 modified=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
INFO Idempotence accomplished efficiently.
INFO Working default > side_effect
WARNING Skipping, facet impact playbook not configured.
INFO Working default > confirm
WARNING Skipping, no assessments discovered.
INFO Working default > cleanup
WARNING Skipping, cleanup playbook not configured.
INFO Working default > destroy
PLAY [Destroy] *****************************************************************
TASK [Destroy molecule instance(s)] ********************************************
modified: [localhost] => (merchandise=occasion)
TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for example(s) deletion to finish (300 retries left).
modified: [localhost] => (merchandise=occasion)
TASK [Delete docker networks(s)] ***********************************************
PLAY RECAP *********************************************************************
localhost : okay=2 modified=2 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
INFO Pruning further information from state of affairs ephemeral listingSince we confirmed plenty of output above let’s breakdown what fascinating issues are taking place right here. Instantly after operating the Molecule take a look at, we get some output telling us what steps within the take a look at course of are going to be run:
INFO default state of affairs take a look at matrix: dependency, lint, cleanup, destroy, syntax, create, put together, converge, idempotence, side_effect, confirm, cleanup, destroyAs this output exhibits, by default Molecule runs these steps as a part of the take a look at matrix within the order proven. Any time we see a
INFO Working default >line within the output, we’re a unique step within the matrix being run. If you happen to dig by the output, you’ll see that most of the steps are literally skipped by default. For instance:
INFO Working default > dependency
WARNING Skipping, lacking the necessities file.In our instance, step one within the matrix the place Molecule really does one thing is the destroy step. At this step, Molecule interacts with the configured driver, in our case Docker, and makes an attempt to destroy any earlier take a look at environments to make sure a brand new clear testing atmosphere is used. Molecule interacts with the Docker daemon and destroys any operating container with our outlined identify from our molecule.yml file:
platforms:
- identify: occasionIf a operating container with that identify doesn’t at the moment exist, as in our case, it merely strikes on. The following step at which Molecule really does one thing is the create step. At this level within the course of, molecule interacts with the driving force and makes an attempt to create a take a look at atmosphere utilizing the driving force we advised it to and configured within the method outlined within the platform part of molecule.yml:
driver:
identify: docker
platforms:
- identify: occasion
picture: "geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:newest"
command: ${MOLECULE_DOCKER_COMMAND:-""}
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
env:
http_proxy: "${http_proxy}"
https_proxy: "${https_proxy}"
no_proxy: "${no_proxy} "Since we’re utilizing Docker, Molecule handles pulling the container picture we outlined, setting any docker runtime choices, and runs the container within the background. The next strains exhibits the profitable creation of our desired take a look at atmosphere:
TASK [Wait for instance(s) creation to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for example(s) creation to finish (300 retries left).
modified: [localhost] => (merchandise={'failed': 0, 'began': 1, 'completed': 0, 'ansible_job_id': '429858788464.21737', 'results_file': '/Customers/maheckathorn/.ansible_async/429858788464.21737', 'modified': True, 'merchandise': {'command': '', 'env': '' '}, 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'identify': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'merchandise'})With our take a look at atmosphere now in place, Molecule strikes on to operating the converge step of the method, which conveniently runs the playbook named converge.yml that we outlined earlier. This playbook runs our position. As of proper now, our position does nothing, as this output exhibits:
INFO Working default > converge
PLAY [Converge] ****************************************************************
TASK [Gathering Facts] *********************************************************
okay: [instance]
PLAY RECAP *********************************************************************
occasion : okay=1 modified=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0What this does present, nevertheless, is that Molecule was in a position to efficiently hook up with our take a look at atmosphere. Instantly after a profitable converge step, Molecule routinely assessments our position for idempotence. This consists of rerunning our converge.yml playbook and ensuring nothing was modified.
If we had outlined Testinfra assessments, the confirm step would have run them:
INFO Working default > confirm
WARNING Skipping, no assessments discovered.An instance of some easy Testinfra code might be seen within the Ansible SiLK github repository:
import os
import testinfra.utils.ansible_runner
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')
def test_silk_version(host):
model = "3.19.2"
command = """/usr/native/bin/silk_config --silk-version"""
cmd = host.run(command)
assert model in cmd.stdoutOn this case, Molecule would run this Python code on the Docker host and Testinfra would deal with connecting to the take a look at atmosphere, operating the assessments, and offering output again to molecule. The take a look at we’re operating merely checks if the model of SiLK put in within the testing atmosphere is 3.19.2. If our assessments cross, Molecule considers the confirm step a hit and strikes on. The very last thing Molecule does throughout a take a look at run is to scrub up after itself by rerunning the destroy step.
Automating Confidence
As highlighted by our instance above, Molecule is able to streamlining your Ansible position growth course of. It stands up and tears down configurable take a look at environments shortly and simply and handles idempotence and verification testing. This weblog posting solely scratches the floor of what Molecule is able to, nevertheless.
For instance, dealing with a clustered take a look at atmosphere is a breeze by merely including one other named occasion to the platform part of the molecule.yml file. Testing towards totally different working methods and totally different OS variations can be a easy command tweak away. Including preparations to the take a look at nodes that, for one cause or one other, have to happen exterior the position is straightforward by inclusion of the put together step (by including a put together.yml playbook). Additionally it is straightforward so as to add position dependencies by making a necessities.yml file within the Molecule listing.
Lastly, the entire course of is straightforward to maneuver to a CI/CD system. The ci.yml within the Ansible SiLK irepository exhibits how to do that with Github actions, however the course of is moveable sufficient to simply be recreated utilizing your CI/CD platform of selection. In case you are not utilizing Molecule for growing your Ansible roles, it’s possible you’ll be severely slowing down your growth cadence and decreasing the standard of your Ansible code. Writing Ansible roles with the assistance of Molecule makes it extremely probably that you would be able to be assured your position does what you need it to, main to raised high quality code and lowered consumer frustration.
[ad_2]
