I was going through few of DEVOPS tools for automation work out of personal interest and was comparing them with respect to our inhouse environments and while comparing the Puppet Vs Chef Vs Ansible Vs SaltStack I see Salt is a very good open source tool and have the capability to manage cloud environments without buying the enterprise edition and can easily manage 1000+ servers with single master itself plus its capable of handling Docker images also.
One better thing is its marginable faster than all the configuration management tools when it comes to large deployments of 1000+ slaves connected and managed by one master. It will be really helpful for curbing out lot of man hours of manual work specially in hosted environments including the inhouse environments.
As an immediate demo in this article, we will try to show how we can eliminate pain points like:
- Windows Security fixes (KB) patching to 100+ servers takes huge time and weekends time to login manually to each server and do the patching.
- Monitoring Service status and ensuring services are running fine, network status, etc. eats out time from daily work hours.
- Env’s configurations states are not in control and reverting back and tracking a single change is not possible. Due to which many issues gets raised by users.
To quickly summarize the article:
As an initial step towards salts capabilities, I have setup Saltstack(Master & Minion[slave]) doing basic windows security patching successfully. We can easily add any windows, linux, solaris boxes as minions to the same salt master for any kind of patching and configuration management tasks.
We can centralize the salt master to make all kind of patching and maintenance activities risk free and automated saving lot of hours of manual activities. Salt is well capable to manage data centers as well as cloud environment’s and amazon VM’s including our local VM environments.
Patching windows security fixes to all slaves from a single master server:
Objectives of example:
– Installation of salt master and saltstack minion (slave). I have taken my laptop as a minion or slave for testing, we can add 1k+ minions to one salt master also.
– Sample steps to change configuration in all minions automatically from one master.
– Sample steps to patch the security fixes to all the minions (Example: cloud servers)
Setup of Salt (Master):
Note: I have used a RHEL 5.9 server to setup the salt master.
- Run below command to download the script to install the master.
curl -L https://bootstrap.saltstack.com -o install_salt.sh
- Run below command to start the install of salt master
[[email protected] opt]# sudo sh install_salt.sh -M * INFO: sh install_salt.sh -- Version 2015.05.07 * INFO: System Information: * INFO: CPU: GenuineIntel * INFO: CPU Arch: x86_64 * INFO: OS Name: Linux * INFO: OS Version: 2.6.18-348.el5 * INFO: Distribution: Red Hat Enterprise Server 5.9 * INFO: Installing minion * INFO: Installing master * INFO: Found function install_red_hat_enterprise_server_stable_deps * INFO: Found function install_red_hat_enterprise_server_stable * INFO: Found function install_red_hat_enterprise_server_stable_post * INFO: Found function install_red_hat_enterprise_server_restart_daemons * INFO: Found function daemons_running * INFO: Running install_red_hat_enterprise_server_stable_deps() * INFO: Adding SaltStack's COPR repository Loaded plugins: product-id, security, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Setting up Install Process Package python26-PyYAML-3.08-4.el5.x86_64 already installed and latest version Package python26-m2crypto-0.21.1-5.el5.x86_64 already installed and latest version Package python26-2.6.8-2.el5.x86_64 already installed and latest version Package python26-requests-1.1.0-5.el5.noarch already installed and latest version Package python26-crypto-2.3-6.el5.x86_64 already installed and latest version Package python26-msgpack-0.4.5-1.el5.x86_64 already installed and latest version Package python26-zmq-14.5.0-1.x86_64 already installed and latest version Package python26-jinja2-2.5.5-6.el5.noarch already installed and latest version Nothing to do * INFO: Running install_red_hat_enterprise_server_stable() Loaded plugins: product-id, security, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Setting up Install Process Package salt-minion-2014.7.5-2.noarch already installed and latest version Resolving Dependencies --> Running transaction check ---> Package salt-master.noarch 0:2014.7.5-2 set to be updated --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: salt-master noarch 2014.7.5-2 saltstack-salt-el5 765 k Transaction Summary ================================================================================ Install 1 Package(s) Upgrade 0 Package(s) Total download size: 765 k Downloading Packages: Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Installing : salt-master 1/1 Installed: salt-master.noarch 0:2014.7.5-2 Complete! * INFO: Running install_red_hat_enterprise_server_stable_post() * INFO: Running install_red_hat_enterprise_server_restart_daemons() Starting salt-minion daemon: [ OK ] Starting salt-master daemon: [ OK ] * INFO: Running daemons_running() -------------------------------------------------------
- Once installation completed run below command to check for the versions installed.
- Run below command to restart the salt master service.
service salt-master restart
- Now download the saltstack windows minion (slave) software for the servers to manage from below url:
- Install the saltstack windows minion in client node with default settings and update below details while installing.
IP address of master and FQDN of client node.
- Make sure the saltstack windows salt-minion service is up and running on client node which you want to manage.
- Now go back to master server and run below command to see the authentication certificate sent by all minions.
[[email protected] opt]# sudo salt-key Accepted Keys: Unaccepted Keys: SaltMinion-WindowsLaptop1 Rejected Keys:
- Now accept the key to authenticate the saltstack windows minion to master.
[[email protected] opt]# salt-key -a SaltMinion-WindowsLaptop1 The following keys are going to be accepted: Unaccepted Keys: SaltMinion-WindowsLaptop1 Proceed? [n/Y] Y Key for minion SaltMinion-WindowsLaptop1 accepted. [[email protected] opt]# sudo salt-key Accepted Keys: SaltMinion-WindowsLaptop1 Unaccepted Keys: Rejected Keys:
- Run below command to check the status of all saltstack windows minions added to salt master. (Here we have added only one minion – my laptop)
Note : you can add 1k+ minions to a single salt master.
salt '*' test.ping
- We can check if the salt master is able to control the saltstack windows minions by running below command to change the minion computers descriptions.
11. a. Check the computers description before running below command to change.
11.b. Now run the below command to change the description of the saltstack windows minion/slave computer from master.
salt '*' system.set_computer_desc 'Office Laptop Of Ramakanta'
- We are now sure that master is able to control the saltstack windows minion so we will go ahead with one more example of auto downloading and installing security fixes automatically.
Steps to Auto patch windows KB3011780 security update from master to all the salt minion servers.
- Before starting the installation, check the latest KB installed in the minion to keep note of it for future verifications of if the KB got installed successfully or not.
Latest KB = KB3010788
- Run below command from salt master to download the KB from Microsoft website in all the saltstack windows minion servers.
It will download and store the file in C:\TEMP as mentioned in below command:
salt '*' cp.get_url 'http://download.microsoft.com/download/C/C/3/CC36FA0C-974B-444A-B2C6-8E368250E37F/Windows6.1-KB3011780-x64.msu' 'C:\TEMP\Windows6.1-KB3011780-x64.msu' -l debug
Note: Here “*” is used to let salt order all registered saltstack windows minions to download the files. It can be customized further to suit your requirement of patching few servers or servers starting with some special patterns etc.
On The saltstack windows minion C:\TEMP the file got downloaded automatically.
- To install the downloaded KB from C:\TEMP in all saltstack windows minions run below command.
salt -t 900 '*' cmd.run 'wusa.exe C:\TEMP\Windows6.1-KB3011780-x64.msu /quiet /norestart' -l debug
Now to check whether the KB got installed successfully run below command:
Below output shows the 226 number KB got installed successfully in my laptop (minion/slave)
salt -t 200 '*' cmd.run 'systeminfo | find "KB3011780"'
You can see the security fix KB3011780 is now installed successfully on my laptop (saltstack windows minion) without any manual intervention.
So if we implement the SaltStack we can manage any configurations, patching and upgrades in any of hosted or local environment’s easily and automatically saving a lot of man hours of manual work and a single person will be able to handle many environments easily from a master.