After using Puppet with an external node classifier for a while one starts questioning what other information could be generated by this instead of just YAML to feed the puppetmaster. When supervisor was being rolled out there was a need to a large number of near identical config files to be generated, however any special information about the configs really had no place in Puppet. So the solution to this was to have the Django app generate the config files and then have puppet pull them down with a custom parser.
In /var/lib/puppet/lib/puppet/parser/functions lives the file webcontent.rb which has the following contents:
require 'open-uri' module Puppet::Parser::Functions newfunction(:webcontent, :type => :rvalue) do |args| server = args[0] configpath = args[1] config = "" begin⋅ open( "http://#{server}/#{configpath}/" ) do |f| f.each_line do |line| config = "#{config}#{line}" end end⋅ rescue OpenURI::HTTPError => e raise Puppet::ParseError, "404 for http://#{server}/#{configpath}/" rescue Exception => e raise Puppet::ParseError, "content string is http://#{server}/#{configpath}/ #{e}" end return config end end
Using the Ruby module open-uri content is grabbed by the puppetmaster and placed into the catalog. Using the following Django model, view and template a config file is easily generated and passed along to Puppet
class SupervisorProgram(models.Model): name = models.CharField(max_length=128) command = models.CharField(max_length=512) autostart = models.BooleanField(default=True) autorestart = models.CharField(max_length=32,choices=(('false','false'),('true','true'),('unexpected','unexpected'))) startsecs = models.IntegerField(default=10) startretries = models.IntegerField(default=3) exitcodes = models.CharField(max_length=64,default="0,2") stopsignal = models.CharField(max_length=5,choices=(('TERM','TERM'),('HUP','HUP'),('INT','INT'),('QUIT','QUIT'),('KILL','KILL'),('USR1','USR1'),('USR2','USR2')),default="TERM") stopwaitsecs = models.IntegerField(default=10) user = models.CharField(max_length=16 ,default="nagios") redirect_stderr = models.BooleanField(default=False) stdout_logfile = models.CharField(max_length=256,default="AUTO") stdout_logfile_maxbytes = models.CharField( max_length = 8,default="50MB") stdout_logfile_backups = models.IntegerField(default=10) stderr_logfile = models.CharField(max_length=256,default="AUTO") stderr_logfile_maxbytes = models.CharField( max_length = 8,default="50MB") stderr_logfile_backups = models.IntegerField(default=10) environment = models.CharField( max_length=512,blank=True,null=True) directory = models.CharField(max_length=128,default="/") umask = models.IntegerField(blank=True,null=True) priority = models.IntegerField(default=999) def __unicode__(self): return self.name class Meta: ordering = ('name',) class SupervisorProgramAdmin(admin.ModelAdmin): list_display = ('name','command','autorestart','stopsignal','exitcodes','user','stdout_logfile','stderr_logfile')
The following is the view used:
def getSupervisorConfig(request,service): print "getSupervisorConfig has been called for %s" % service service = get_object_or_404(SupervisorProgram,name=service) directives = {} directives["command"] = str(service.command) directives["process_name"] = str(service.name) directives["priority"] = int(service.priority) directives["autostart" ] = service.autostart directives["autorestart"] = service.autorestart directives["startsecs"] = int(service.startsecs) directives["startretries"] = int(service.startretries) directives["exitcodes"] = str(service.exitcodes) directives["stopsignal"] = str(service.stopsignal) directives["stopwaitsecs"] = int(service.stopwaitsecs) directives["user"] = str(service.user) directives["redirect_stderr"] = service.redirect_stderr directives["stdout_logfile"] = str(service.stdout_logfile) directives["stdout_logfile_maxbytes"] = str(service.stdout_logfile_maxbytes) directives["stdout_logfile_backups"] = int(service.stdout_logfile_backups) directives["stderr_logfile"] = str(service.stderr_logfile) directives["stderr_logfile_maxbytes"] = str(service.stderr_logfile_maxbytes) directives["stderr_logfile_backups"] = int(service.stderr_logfile_backups) directives["directory"] = str(service.directory) if service.environment: directives["environment"] = str(service.environment) return render_to_response("sock/supervisor.conf",directives)
With the 20 configuration options per supervisord controlled process there are far too many options that should be sanely passed to puppetmaster from the external node classifier.
Here is the Django template:
#generated config
[program:{{ process_name }}]
command={{ command }}
process_name=%(program_name)s
priority={{ priority }}
autostart={{ autostart }}
autorestart={{ autorestart }}
startsecs={{ startsecs }}
startretries={{ startretries }}
exitcodes={{ exitcodes }}
stopsignal={{ stopsignal }}
stopwaitsecs={{ stopwaitsecs }}
user={{ user }}
redirect_stderr={{ redirect_stderr }}
stdout_logfile={{ stdout_logfile }}
stdout_logfile_maxbytes={{ stdout_logfile_maxbytes }}
stdout_logfile_backups={{ stdout_logfile_backups }}
stderr_logfile={{ stderr_logfile }}
stderr_logfile_maxbytes={{ stderr_logfile_maxbytes }}
stderr_logfile_backups={{ stderr_logfile_backups }}
{% if environment %}
environment={{ environment }}
{% endif %}Finally all of this can be referenced with a custom define as follows:
define supervisorconfig(
$program,
$server = "${rserver}"
) {
file {
"${name}.conf":
owner => root,
group => root,
mode => 0644,
path => "/etc/supervisord.d/${name}.conf",
content => webcontent( $server, "dpuppet/sock3/getsupervisorconfig/$program")
}
}
