Versions Compared

    Old Version 29

    changes.mady.by.user John Burk

    Saved on

    compared with

    New Version 30

    changes.mady.by.user John Burk

    Saved on

    Key

    • This line was added.
    • This line was removed.
    • Formatting was changed.
    Comment: Published by Scroll Versions from this space and version 6.9-0

    ...

    Expand
    titleTemplate Inheritance

    The advantage of the macro file format is the use of template inheritance and value replacement. A section is allowed to "inherit" another section's key/value pairs. This can be accomplished by adding a colon and a space-delimited list of templates.

    Icon

    The qbwrk.conf file is read from top to bottom. A template can only inherit from an already-defined template that appears before it in the file.

    Example

    The section:

    [qube]
    key9 = value00

    [section]
    key = value
    key1 = value1

    [section2] : section
    key = value3

    [section3] : section2 qube

    evaluates to:

    [qube]
    key9 = value00

    [section]
    key = value
    key1 = value1

    [section2]
    key = value3
    key1 = value1

    [section3]
    key = value3
    key1 = value1
    key9 = value00

    (New in 6.9-1)

    A section [global_config] may be defined in the qbwrk.conf file to set up global qbwrk.conf configurations.

    Currently, it supports the following two parameters, used primarily to optimize the loading time of qbwrk.conf:

    templates: explicitly list all template names in the qbwrk.conf file

    non_existent: explicitly list all non_existent hostnames that are listed in qbwrk.conf

     

    During the loading of the qbwrk.conf file, if a section name like [name] is encountered, the "name" is assumed to be a valid hostname of a worker that is online and accessible, and the supervisor tries to look up the IP address from the name.
    In certain network setups, if the "name" is actually a qbwrk.conf template name or a hostname of a non-existent (offline or inaccessible) worker, this can slow things down quite a bit since each such lookup needs to time out. In those situations, the above two parameters, "templates" and "non_existent" in the [global_config] section can be properly set up to optimize the loading time for qbwrk.conf, which in turn speeds up the execution of `qbadmin w -reconfig` and also the supervisor boot process.

    Example:

    [global_config]

    templates = centos,ubuntu
    non_existent = render[071-100],render123,render155
     
    [centos]
    worker_cluster = /projects/foobar
     
    [ubuntu]
    worker_cluster = /projects/secret
     
    [render[001-200]]
    worker_groups = "dedicated"
    In the example above, the section names "centos" and "ubuntu" are listed in the global_config's "templates" parameter, and also the
    machines "render[071-100],render123,render155" are listed as "non_existent". The supervisor, when loading the qbwrk.conf file,
    skips the IP address lookup for these listed names, thereby speeding up the process.
    Note that in the example, it is assumed that the "non_existent" machines are not online, perhaps taken down for maintenance, and therefore
    the site administrator has listed them there.
    Also note that numerical range expansions are allowed in the value of the "non_existent" parameter (as in "render[071-100]"), to conveniently specify a contiguous chunk of hostnames.
    Expand
    titleThe [global_config] Section
    Expand
    titleWorking Example

    As an example, let's say you need all machines to define proxy_account and proxy_password, you would then:

    No Format
    [default] 
proxy_account = render 
proxy_password = cbda878cd5ad5dcdab8967dbc86bc786a857ada57bc # < - generated by qbhash

     

    Then let's say you want to define a worker path map, but it will be different based on the OS, you would then:

    No Format
    [osx] 
worker_path_map = {
 "\\" : "/",
 "X:/project" : "/Volumes/xsan/project",
 "H:/" : "/home", 
} 
 
[winnt] 
worker_path_map = {
 "/Volumes/xsan/project" : "X:/project",
 "/home" : "H:/",
 "/" : "\\", 
}

     

    Now let's say you want hosts01 - host05 to be in a group called "groupA" and a cluster called "/foo"; and host06 - host10 to be in a group called "groupB" and a cluster called "/bar", and then host11-15 need to be in group "groupB" and cluster "/bar" and define a worker_restriction of "/bar/+":

    No Format
    [groupa_rule] 
worker_groups = groupA 
worker_clsuter = /foo 
 
[groupb_rule] 
worker_groups = groupB 
worker_cluster = /bar 
 
[host[01-05]] : groupa_rule 
 
[host[06-10]] : groupb_rule 
 
[host[11-15]] : groupb_rule 
worker_restriction = /bar/+

     

    Now let's say that host07 is special and needs to be a member of both groupB and nvidia, but not lose its worker_cluster. Just redefine it afterwards (the file is read top to bottom):

    No Format
    [host07] 
worker_groups = groupB,nvidia

     

    Here's the whole thing together:

    No Format
    [default] 
proxy_account = render 
proxy_password = cbda878cd5ad5dcdab8967dbc86bc786a857ada57bc
 
[osx] 
worker_path_map = {
 "\\" : "/",
 "X:/project" : "/Volumes/xsan/project",
 "H:/" : "/home", 
} 
 
[winnt] 
worker_path_map = {
 "/Volumes/xsan/project" : "X:/project",
 "/home" : "H:/",
 "/" : "\\", 
} 
 
[groupa_rule] 
worker_groups = groupA 
worker_clsuter = /foo 
 
[groupb_rule] 
worker_groups = groupB 
worker_cluster = /bar 
 
[host[01-05]] : groupa_rule 
 
[host[06-10]] : groupb_rule 
 
[host[11-15]] : groupb_rule 
worker_restriction = /bar/+ 
 
[host07] 
worker_groups = groupB,nvidia

     

    To pull it all together, if host12 were a windows machine, it would get the following config:

    No Format
    proxy_account = render # from default 
proxy_password = cbda878cd5ad5dcdab8967dbc86bc786a857ada57bc # from default 
worker_path_map = {
 "/Volumes/xsan/project" : "X:/project",
 "/home" : "H:/",
 "/" : "\\", 
} # from [winnt] 
worker_groups = groupB # from [host12] ( [host[11-15]] ) which inherited from [groupb_rule] 
worker_cluster = /bar # from [host12] ( [host[11-15]] ) which inherited from [groupb_rule] 
worker_restriction = /bar/+ # from [host12] ( [host[11-15]] )
    Info

    • We never had to inherit from [osx] or [winnt], as those were already applied to all OSX and Windows machines before we even got into [host*] sections.

    • There are no machines called "group*_rule" - those are just placeholders, if you will.

    • You can do all of this through WranglerView on the supervisor (you must be running it on the supervisor for this to work). Simply multi-select workers on the worker layout, right-click > "Configure on supervisor", then change all the parameters you choose and click OK. This is somewhat less flexible, but many people prefer GUIs.

    ...