##############################################################################
@RELEASE: 6.4-5
##############################################################################
==== CL 11108 ====
@FIX: supe's built-in perl library's C++ host object to Perl hash conversion routine to properly include "properties", "stats", "reason", "locks", "flags", "flagsstring", "groups", "description", "jobtypes", "address", "macaddress", "lastupdate"
@FIX: typo in Perl API's _qb_host_hash() routine when converting the "description" field.
==== CL 11093 ====
@NEW: add various helper functions to qb.utils; addToSysPath(), getModulePath(), pyVerAsFloat(), formatExc()
==== CL 11067 ====
@FIX: timing issue where a subjob of an agenda-based job can be incorrectly left in the "blocked" or "pending" state even though there are no more agenda items to be processed.
@INTERNAL: Checker code was added to the statusJob() routine to force the status to "complete" of such jobs.
ZD: 9190
==== CL 11066 ====
@INTEG:main>rel-6.4,rel-6.3,CL11024, CL11056, CL11057
----
This is a partial integration of CL11024,11056,andCL11057. Namely, the "const"-ness fix in the
QbDatabase* classes are being integrated into rel-6.3 and rel-6.4 so they will compile cleanly.
Also, the change in the logging behavior (so that MySQL logs are timestamped) is integrated.
==== CL 11062 ====
@FIX: fixed unreliable "modify" behavior. Multiple modifies (for example, up then down) were behaving oddly.
@CHANGE: added code to automatically retire pending/blocked/running jobs when "modify" reduces the "cpus" ("instances") count.
ZD: 9205
@FIX: fixed a subtle off-by-one error in auto-retire code in assignJob()
==== CL 11058 ====
@FIX: patched a timing issue where the requestWork() handler can sometimes put a running subjob back to "pending" (because it's marked to be passively preempted) even if there are no more agenda items left to process.
ZD: 9132
==== CL 11054 ====
@CHANGE: made all error messages from the QbDatabaseMySQL class prints with a timestamp.
==== CL 11016 ====
@FIX: fixed return data type of qb.submit() to be a list of job objects
ZD: 9314
==== CL 11008 ====
@FIX: issue where modifying a job to reduce the number of instances can sometimes incorrectly retire ALL instances.
==== CL 10984 ====
@FIX: control-characters in C4D Windows paths can break path translation, get evaluated as tabs/newLines/etc. This is due to C4D needing to be run via "start" instead of "cmd.exe /C"
==== CL 10963 ====
@FIX: random worker crash issue on Windows
ZD: 8620
==== CL 10934 ====
@FIX: suppress printing of "Malformed env in parsing" and environment listing when environment values are other than simple strings and "Query SQL" is enabled in the WranglerView prefs
##############################################################################
@RELEASE: 6.4-4
##############################################################################
==== CL 10894 ====
@FIX: Updated qb.conf.wintemp (Windows template file for qb.conf) to be in sync with the Unix template.
Also added proper data paths for Windows Vista 7 and up, in the commented-out default values.
JIRA: QUBE-74
==== CL 10875 ====
@FIX: issue with stdout/err logs getting truncated and duplicate status being logged to .hst file when qbreportjob() is used to report intermediate status by a running instance.
JIRA: QUBE-46
==== CL 10867 ====
@FIX: data warehouse database creation failing on CentOS 6.2; mysql client is installed in /usr/bin instead of /bin, and must provide full paths to bash "." source statements
==== CL 10858 ====
@INTERNAL: qblock/qbunlock source consolidation on Windows
==== CL 10857 ====
@INTERNAL: consolidating qblock and qbunlock source files.
==== CL 10856 ====
@FIX: minor regex issue with previous check-in (CL10855)
==== CL 10855 ====
@FIX: qbhash command (on Windows only) allowed additional options when run as "qbhash.exe"
This was due to it sharing the same code as qblogin, and an internal regex not considering the .exe extension.
==== CL 10841 ====
@FIX: fixed issue where agenda item commands such as "retrywork" would incorrectly be applied to unspecified/undesired agenda items, if the list of agenda items contains items from more than 1 parent job (e.g. "1234:1 1235:1")
For example, "qbretry 1234:1 1235:1" would retry every work item in job
1235, despite the specification being just item 1.
JIRA: QUBE-61
==== CL 10839 ====
@FIX: minor logging fix: "resetting start/complete time of work" now prints the work 'name' instead of 'id' for consistency and readability.
==== CL 10833 ====
@FIX: supervisor msi installation fails during InnoDB cleanup operation, aborts the supervisor installation
==== CL 10830 ====
@FIX: python api qb.submit() fails silently when a job label is not unique within the pgrp
Now raises a ValueError exception, with an error msg when qb.submit() fails.
JIRA: QUBE-32
==== CL 10802 ====
@FIX: suppress warning about missing stderr job logs if stderr merged into stdout in job submission
==== CL 10798 ====
@CHANGE: pyCmd* jobtypes now properly mimic cmdline/cmdrange behavior: apply path conversion to entire cmdline string if convert_path flag is set, otherwise only apply path conversion to strings enclosed in a QB_CONVERT_PATH token block
@CHANGE: also support mixed use of QB_CONVERT_PATH tokens and convert_path job flag, apply translation to tokens first, then the rest of the cmdline
==== CL 10784 ====
@FIX: issue where grid jobs are doubly booted on the allocated nodes in certaing cases.
ZD: 8686
==== CL 10744 ====
@NEW: add support for per-OS environment variables, allows for different envVar values depending on run-time OS. Currently only supported by pyCmdline, pyCmdrange, and appFinder jobtypes. Passed in as job['package']['env_runTimeOS'][ 'Windows' | 'Linux' | 'Darwin' ], keyed off platform.system()
==== CL 10724 ====
@FIX: possible crashes due to timing issue (between queue.listReady() and queue.getById()) in startResources()
ZD: 8566
==== CL 10723 ====
@FIX: memory leak in startHost().
@FIX: possible crashes due to timing issue (between queue.listReady() and queue.getById())
ZD: 8566
==== CL 10715 ====
@CHANGE: decrease default log rotation size from 256MB to 100MB on Windows and OS X, can be overridden by providing the '-s <size>' argument to logrotate.py in the Windows scheduled task or the OS X LaunchDaemon plist
==== CL 10705 ====
@FIX: fixed issue with -p_agenda option incorrectly picking frames.
==== CL 10695 ====
@FIX: Fixed issue with instances being dispatched to multiple workers when jobs were qbmodify-ed their "cpus" down and then up.
When the "cpus" parameter, i.e. the instance count, was qbmodify-ed down
and then up, some instances would end up being dispatched and running on
multiple workers at the same time. This was due to the fact that
Until now, when a job's cpus count is reduced, instances of higher ID
numbers were always chosen to retire (i.e., if a 5-instance job was reduce
to 3, then instances 3 and 4 were retired). Now, instead, the first
instances that request a "requestwork" are retired.
Also, when a job's cpus count is increased, the supe will first revitalize
any instances that are already in the "done" state, and then add more
instances to the job if necessary. For example, say a 5-instance job was
reduced to 3 instances, and instance 1 and 2 were retired in response
(0,3,4 are running). If, later, the job was modified again to increase the
instance count to 7, instances 1 and 2 are revitalized (i.e. moved back to
"pending") AND 2 new instances, 5 and 6, are generated.
ZD: 8542
==== CL 10692 ====
@CHANGE: added more useful msg to print in workerVerifyAssignment()
==== CL 10685 ====
@FIX:fixed examples/cpp/ .sln and .vcproj files to build for x64 and under VS 2005
==== CL 10682 ====
@CHANGE: added a supervisor_preempt_policy of "mixed", to support mixed-mode preemption with custom algorithms (and potentially with built-in algorithms too, in the future).
Setting the preemption mode to "mixed" allows custom algorithms to
aggressively preempt a job that's already been marked to be passively
preempted.
ZD: 8556
==== CL 10642 ====
@NEW: move AppFinder jobs to their own jobtype
==== CL 10641 ====
@NEW: add QB_CONVERT_PATH() tokens to paths in simpleCmds to support runtime path conversion using the conventional qb.convertpath()
@NEW: imports new qb.utils module
@FIX: pattern matches in logs (output paths, highlights, etc) being stored multiple times
==== CL 10640 ====
@FIX: characters in application path string are being interpreted as escaped ctrl-characters
==== CL 10633 ====
@NEW: add QbTableVersion31
==== CL 10608 ====
@INTERNAL: changed MySQL MEMORY table creations to read "ENGINE=MEMORY" instead of "TYPE=MEMORY" which is obsolete as of MySQL 5.5
==== CL 10604 ====
@CHANGE: all obsolete "HEAP" type MySQL tables to the new "MEMORY" type, to conform to MySQL spec change as of version 4.1 (HEAP backward compatibility removed in 5.5)
@INTERNAL: added QbTableVersion31.cpp
@INTERNAL: upped QbVersion version to 6.4.3
BUGZID: 63769
==== CL 10594 ====
@FIX: issue where the automount flag was always set for jobs if client_job_flags was set to the empty string in qb.conf
==== CL 10589 ====
@FIX: job list not updating when switching supervisors, always show jobs from the default supervisor.
==== CL 9730 ====
@TWEAK: modified so that worker name and IP print when job is accepted by worker, in assignJob()
##############################################################################
@RELEASE: 6.4-3
##############################################################################
(version skipped - never released)
##############################################################################
@RELEASE: 6.4-2a
##############################################################################
==== CL 10591 ====
@FIX: fixed issue where the worker rejects jobs with the auto_mount flag turned on when run in desktop user mode and worker_cpus != 1 (which automatically turns of auto_mount in worker_flags)
The auto_mount settings of the job/worker should be irrelevant for workers running in desktop user mode.
##############################################################################
@RELEASE: 6.4.2
##############################################################################
==== CL 10543 ====
@FIX: issue with worker_path_map not working when defined in qbwrk.conf and containing backslashes.
==== CL 10537 ====
@FIX: issue where qbconvertpath() can return an empty string when worker_path_map is undefined.
##############################################################################
@RELEASE: 6.4.1
##############################################################################
==== CL 10514 ====
@FIX: another patch for out-of-order issue. Fixed unexpected short-circuit evaluation that was happening in the startResources() routine
==== CL 10513 ====
@FIX: another patch for out-of-order issue. Fixed unexpected short-circuit evaluation that was happening in the startHost() routine
==== CL 10512 ====
@INTERNAL: QbJob object's _subjobswaiting data was not being initialized or copied correctly, causing some job comparisons based on subjobs waiting counts to unexpectedly fail.
==== CL 10504 ====
@INTERNAL: added more log output for debugging builds, added more comments while working on out-of-order issue.
ZD: 8198
==== CL 10477 ====
@FIX: Another out-of-order fix. Jobs at the same numerical and cluster priority should dispatch in the correct FIFO order now.
The FIFO enforcing should work most of the time, but there still will be
occasional out-of-order behavior, due to the multi-threaded nature of the
supervisor. ("qbshove"-ing the older job should correct it, when it's seen)
ZD: 8198
==== CL 10462 ====
@FIX: yet yet another fix for out-of-order dispatch behavior-- eliminate race-condition that would allow lower priority jobs that were just preempted to get workers before higher-priority jobs.
See also CL10440 10452
ZD: 8198
==== CL 10461 ====
@CHANGE: modified/compacted the multi-line "found a duty to replace" logging to be a single line.
==== CL 10452 ====
@FIX: yet another fix for out-of-order dispatch behavior-- eliminate race-condition that would allow lower priority jobs that were just preempted to get workers before higher-priority jobs.
See also CL10440
ZD: 8198
==== CL 10441 ====
@FIX: killing an already finished (complete, failed, killed) job leaves the job in the "dying" state.
==== CL 10440 ====
@FIX: another fix for out-of-order dispatch behavior-- eliminate race-condition that would allow lower priority jobs that were just preempted to get workers before higher-priority jobs.
ZD: 8198
==== CL 10429 ====
@FIX: out-of-order job dispatching issue with jobs using the "+" sign with the "host.processors" reservations.
ZD: 8198 8261 8229 8233 8228
==== CL 10389 ====
@NEW: add new appFinder submission for C4D
==== CL 10323 ====
@NEW: add support to pyCmd* jobtypes for new "auto-pathing" feature; can now send jobs to a mixed set of workers and find the 3rd-party executable on all OS's, not pre-defined in the job's package
==== CL 10271 ====
@CHANGE: desktop user mode worker to only allow automount when "worker_cpus = 1" is set explicitly.
==== CL 10264 ====
@NEW: add automount support for desktop user mode on Windows
@CHANGE: db table change (additional column to the assignment table) required-- adding QbTableVersion7 definition.
@FIX: unmounting of "subst" style local mounts was broken
@INTERNAL: added a bunch of comments, and renamed some methods in the QbMission class, for readability.
==== CL 10254 ====
@NEW: pyCmdline and pyCmdrange do run-time path translation
==== CL 10233 ====
@FIX: added qb::workerconfig() that was missing to the Perl API
==== CL 10228 ====
@FIX: missing "bin/qbhash" command on Linux
==== CL 10223 ====
@FIX: examples in the code to reflect previous change to the command line options/arg
==== CL 10216 ====
@NEW:Job cleanup script in utils directory. This script is designed to be run by a user or by a user-created scheduled task.
==== CL 10191 ====
@FIX: removed unneeded "install_worker" and "uninstall_worker" scripts from being installed on Mac OSX
==== CL 10189 ====
@FIX: timing issue where some worker resources (host.xyz) would disappear after the worker received a remote config.
@FIX: issue where supervisor tries to dispatch a subjob to a worker with
insufficient resources (reduced the likeliness of that from happening)
@FIX: the above 2 fixes combined should now prevent some of the
out-of-priority-order dispatch issues, especially in environments where
worker resources are deployed.
ZD: 7885
==== CL 10149 ====
@CHANGE: modified so the worker_path_map mapping definition order is preserved when it is applied to paths via convertpath()
==== CL 10144 ====
@FIX: bug with handling lone backslash in the worker_path_map
@CHANGE: modifying QbConfig class to maintain order of option (config parameter) addition
==== CL 10125 ====
@NEW: add automatic runtime path conversion to cmdline and cmdrange jobtypes
@NEW: jobs may have the "convert_path" flag set to tell the jobtype to do runtime path conversion.
@NEW: qbsub now has a "-convertpath" option to set the flag.
@NEW: qubegui simpleCmd interface has a new "convert path" checkbox
==== CL 10118 ====
@FIX: fixed issue where agenda timeouts don't work properly on the first agenda item processed by a subjob, on Unix (Linux/OSX) workers
==== CL 10117 ====
@FIX: fixed issue where agenda items that fail because of timeout don't get automatically retried via retrywork
ZD: 7763
==== CL 10097 ====
@NEW: add Mac OS X 10.8 Mountain Lion support
==== CL 10095 ====
@FIX: fixed newly introduced issue with errors reading licenses in dev/main branch supe
==== CL 10074 ====
@INTEG: main -> rel-6.4
-----
@FIX: data warehouse installation/upgrade scripts on linux/OSX now search /etc/qb.conf for database_user/_password/_port/_host values in order to support non-default values for these parameters
==== CL 10072 ====
@NEW: add activeperl 5.16 support for Windows
==== CL 10068 ====
@NEW: Add doc on QB_CONVERT_PATH(srcpath) in Use.doc and qbsub's online help
==== CL 10067 ====
@NEW: Add documentation on worker_path_map config parameter and the qbconvertpath() API routine.
==== CL 10062 ====
@FIX: fixed parsing code in QbConfigFile.cpp so that the "name" part of a name-value pair can contain special chars if double-quoted.
==== CL 10049 ====
@INTEG: main -> rel-6.4
-----
@FIX: reduce the number of times qb.supervisorconfig() and qb.getusers() are called during GUI startup and normal operation, pre-populate the qbCache with this data at startup
==== CL 10048 ====
@FIX: reduce the number of times qb.supervisorconfig() and qb.getusers() are called during GUI startup and normal operation, pre-populate the qbCache with this data at startup
==== CL 10025 ====
@FIX: data warehouse installation/upgrade scripts on linux/OSX now search /etc/qb.conf for database_user/_password/_port/_host values in order to support non-default values for these parameters
==== CL 10022 ====
@FIX: modified the worker to only report to the supe of its host status when subjobs are completely done and removed, and NOT when they are only marked/scheduled for removal.
This was causing jobs to sometimes run out-of-order, especially when there
are many subjobs to each job (such as one subjob per frame), since that
situation tends to increase the chance of the supervisor dispatching the
same subjob to the same worker. The subjob will be dispatched to the same
worker, but rejected since the worker thinks it's a duplicate assignment of
a subjob that's being removed (and consequently a lower priority job will
get the worker's slot, causing out-of-order job execution)
ZD: 7601
##############################################################################
@RELEASE: 6.4.0
##############################################################################
==== CL 9973 ====
@FIX: bug where UNC paths with backslashes won't work in the new worker_path_map
@INTERNAL: Note: Backslashes are now NOT treated as special chars in QbConfigFile's tokenize() routine (called from parse())
==== CL 9966 ====
@NEW: pyCmdline - a python-based implementation of cmdline jobtype
==== CL 9963 ====
@FIX: add launchCondition so that worker and supervisor will not install if core is not present
@NEW: write a registry key upon installation in order to provide dependency checking for core removal (core will not uninstall if worker or supervisor is installed)
==== CL 9959 ====
@NEW: adding back-end run-time path conversion feature, and exposing in perl, python, and C++ APIs (qbconvertpath())
==== CL 9953 ====
@FIX: fixed config file (qb.conf) parsing code so that it properly parses the worker_path_map
Note: old code was corrupting qb.conf when upgrade_config tool was run.
==== CL 9937 ====
@NEW: houdini loadOnce jobtype finds the appropriate houdini installation at runtime, based off HFS and optionally pkg['houdiniVersion'], user no longer has to guess at python path on the remote worker
@NEW: add versionPicker controls to QubeGUI Houdini submission UI
@NEW: new multi-line syntax for application paths in the job.conf file
@NEW: added scanConfForPaths to backend utils module
==== CL 9930 ====
@NEW: added qbworkerpathmap() to the C++ API and qb.workerpathmap() to the python API.
The worker_path_map in qb.conf (or qbwrk.conf) must be defined like:
worker_path_map = {
[direct]
H: = /home
X: = /proj/x
}
Note, in particular, the "[direct]" keyword. That MUST be present.
qb.workerpathmap() called in the python backend will return a nested dict of the format:
{'directmap': {'X:': '/proj/x', 'H:': '/home'}}
@INTERNAL: fixed bugs in the config-file reader code, added a bunch of comments
==== CL 9918 ====
@UPDATE: update Use.doc with table of all job flags and their descriptions, including info on the new migrate_on_frame_retry job flag
==== CL 9915 ====
@NEW: added a new job flag "migrate_on_frame_retry", which, if set, forces a subjob to migrate to another worker if it fails a frame, and the frame is set to automatically retry (via retrywork).
==== CL 9909 ====
@FIX: fixed issue that was causing jobs to NOT be considered for dispatch immediately at submission.
Bug was introduced while attempting to fix a memory leak bug, in CL9592
==== CL 9903 ====
@FIX: better message from worker when it rejects a dispatched subjob because it's a duplicate (being preempted or migrated on the same worker)
==== CL 9893 ====
@NEW: add example qb.conf files for various-sized farms
@NEW: add example qbwrk.conf to the build
==== CL 9891 ====
@FIX: _highest_priority() routine to disregard priorities that are non-positive.
==== CL 9886 ====
@UPDATE: admin doc with info on supervisor_highest_user_priority
==== CL 9882 ====
@FIX: fixed pathmap bug where the object/data wasn't being properly transmitted over the network at all.
@CHANGE: also uncommented the line that prints out the pathmap to the workerlog on worker boot.
==== CL 9865 ====
@NEW: Added support for supervisor_highest_user_priority to the GUI's "Local Configuration" dialog.
@NEW: Added supervisor_highest_user_priority to the qb.conf.template file.
@CHANGE: Also modified the description of supervisor_max_priority in qb.conf.template to avoid confusion.
==== CL 9864 ====
@NEW: added qb.conf setting "supervisor_highest_user_priority", which sets the highest priority (i.e., smallest numerical value) at which an ordinary (non-admin) user can submit/modify jobs.
Users must be qube admin to be able to submit/modify at higher priority than this value.
It's default value is 1.
BUGZID: 63717
==== CL 9838 ====
@CHANGE: upped the default value for supervisor_max_threads to 100, and worker_max_threads to 32
==== CL 9837 ====
@CHANGE: update the qb.conf templates, supervisor_max_threads=96, leave it uncommented until such time as this matches the supervisor's default behavior
==== CL 9788 ====
@TWEAK: improved log message when worker goes into panic because of the lack of sufficient permissions
==== CL 9785 ====
@FIX: worker issue where desktop worker would randomly crash.
ZD: 6778
==== CL 9736 ====
@NEW: add support for MySQL passwords to qb.query.mysqlConnect
==== CL 9711 ====
@NEW: add Admin->Database Check/Repair functionality to the GUI
@TWEAK: add ability to print to logPane in realtime for long-running processes, no need to wait until operation is finished
@FIX: bugfix for Admin->Ping Supervisor raising KeyError when supervisor is down
==== CL 9698 ====
@FIX: fixed false-negative warning message pertaining to "select() in checkpoint()" seen in supelog.
Examples of these messages:
select() in checkpoint(): Operation timed out
select() in checkpoint(): Interrupted system call
==== CL 9694 ====
@FIX: fixed issue with the supe threads getting tied up on "subjob X seems to be already assigned" message.
On a farm with busy workers, the time between the supe dispatching a sub
job to the worker via assignJob() and the worker reporting that the "subjob
is running" can be several seconds to sometimes even several minutes, which
was causing many supe threads to attempt dispatching the same subjob over
and over. All of those threads end up hitting the "subjob X seems to be
already assigned... retrying" message, and get tied up for 3 seconds while
they retry.
BUGZID:
ZD: 6760 7125
==== CL 9689 ====
@FIX: fixed bug in clustering algorithm where it incorrectly gave more
weight to a job when the only difference was the last letter in the cluster
specification.
For example, if:
host cluster: /3D/projA
job1 cluster: /3D/projB
job2 cluster: /3D
job1 was getting more weight than job2, which is incorrect.
BUGZID: 63740
ZD: 7043
==== CL 9687 ====
@INTEG: rel-6.3 -> main CL 9686
-----
@FIX: using deprecated "waitfor" attribute with Python api causes qb.submit() to raise a KeyError
@FIX: properly convert "waitfor" value (jobid integer) to proper "dependency" string of "link-done-job-<id>"
==== CL 9686 ====
@FIX: using deprecated "waitfor" attribute with Python api causes qb.submit() to raise a KeyError
@FIX: properly convert "waitfor" value (jobid integer) to proper "dependency" string of "link-done-job-<id>"
==== CL 9678 ====
@NEW: provide a "Studio Overrides Prefs" in the QubeGUI which will allow mandated studio-wide preferences, will override userPrefs, which already override the "Studio Defualts Prefs". Added support for --studioprefs cmdline option and QUBEGUI_STUDIOPREFS environment variable.
==== CL 9677 ====
@INTEG: rel-6.3->main CL 9676
-----
@FIX: update documentation and GUI help text to show correct "||" syntax for job restrictions list.
==== CL 9676 ====
@FIX: update documentation and GUI help text to show correct "||" syntax for job restrictions list.
==== CL 9664 ====
@CHANGE: specify unix_socket when connecting to MySQL server on localhost on non-Windows platforms
==== CL 9663 ====
@INTEG: rel-6.3 -> main CL 9662
-----
@FIX: supervisor install was failing postflight scripts on OSX Server, expliclty set the mysql socket to /tmp/mysql.sock in /etc/my.cnf and /etc/qb.conf to avoid conflicting with the factory-installed default of /var/lib/mysql/mysql.sock
==== CL 9662 ====
@FIX: supervisor was failing postflight upgrade scripts on OSX Server, expliclty set the mysql socket to /tmp/mysql.sock in /etc/my.cnf and /etc/qb.conf to avoid conflicting with the factory-installed default of /var/lib/mysql/mysql.sock
==== CL 9615 ====
@FIX: Added code to properly log frames (to supelog and job log) when they go back to "pending" after the processing subjob/worker is found dead.
@FIX: Added code in the supervisor to retry a failed worker connection
after a random 5-10 sec sleep/delay, to alleviate network hiccups during
network commands (kill, preempt, etc. of running subjobs).
ZD: 6760
==== CL 9614 ====
@INTERNAL: fixed a small cosmetic bug introduced in CL 9606
==== CL 9607 ====
@INTERNAL: added converseWorkerWithRetries() and also fixed small bug in the retry loop of converseSubSupervisorWithRetries()
==== CL 9592 ====
Fixed code that was causing memory leaks when supervisor threads handled
job submissions.
==== CL 9585 ====
@FIX: issue where some jobs get stuck in the "dying" state when attempted to be killed
ZD: 6616
==== CL 9578 ====
@NEW: add another python example script which shows a 'block until' type of callback; a job can be submitted to run at a certain time of day, if the TOD is in the past, it's assumed to be tomorrow
==== CL 9570 ====
@FIX: improvements to the handling of GET_LOCK (aka"reserveJob()") timeout situations.
ZD: 6617
==== CL 9549 ====
@FIX: qbwrk.conf files that had any commented-lines before the first valid template was encountered would cause an exception to be raised, QubeGUI->worker->RMB->Configure would fail silently
==== CL 9535 ====
@NEW: add submit-agenda-timeout-job.py example python script, to demonstrate submission of a job with frame-level timeouts.
ZD: 6099
==== CL 9530 ====
@FIX:Submitting paths to shotgun no longer depends on the visibility of output paths to the supervisor.
@FIX:Shotgun submission script fails gracefully & logs a reason as to why it can't generate a thumbnail when thumbnail creation fails.
==== CL 9523 ====
@FIX: fixed issue where the supervisor fails to correctly track the host assignment for subjobs.
Symptom for this included seeing in the supelog, messages like "statusJob(): aberrant report from worker...", then followed by "subjob[xxxx] is assinged to worker[] with mac address[00:00:00:00:00:00]".
These subjobs would then be in the "running" state, but not assigned to a worker.
==== CL 9522 ====
@FIX: removed code that skipped code that made local decision on the supe to test for resource reservations, for jobs with host.processors set to > 1, delegating the decision-making to the workers and resulting in more network traffic and latency.
ZD: 6141
==== CL 9507 ====
@FIX: added more robust code that talks to the SMTP server when sending out email,
to support some email servers with non-standard response behavior.
ZD: 6209
==== CL 9504 ====
@FIX: catch case where sg_path_to_frames is part of the Shotgun versionName, but the job has no outputPaths for the first frame; fallback to naming the version "job id: 123 jobName: ..."
==== CL 9500 ====
@FIX: Windows Vista/7/2008-R2 installer - don't error out when installing the worker or supervisor as an Admin-equivalent account during creation of scheduled tasks. Properly remove scheduled tasks during uninstall.
==== CL 9496 ====
@FIX: catch case when inserting in a new cluster into cluster_dim when more than 1 worker exists in the new cluster; occurs during run of regular_slotcount.sql, doesn't prevent new record from being added, just generates line noise and error emails from cron...
==== CL 9494 ====
@CHANGE: make explanation of "+ | *" in job/host restrictions less ambiguous
==== CL 9484 ====
@FIX: calculate cpu-seconds for agenda-based jobs by summing up work times, not subjobs. Better support for resetting of the start times for retried work.
==== CL 9467 ====
@NEW: add a random offset to the startup so that all workers don't report at the same time if they've started up at the same time.
@CHANGE: don't retrieve job name, it's extraneous and not reported; cuts down the query count by one.
@CHANGE: set workname for subjob to job.subid, not subid; easier to detect case where an agenda-based job falsely reports not having an agenda, so subjob id won't conflict with a frame number
==== CL 9463 ====
@FIX: don't report memory usage in the case where MySQL fails to return a valid agenda name, usually caused by timeouts or maxed out connections.
==== CL 9461 ====
@CHANGE: removing from VS solution: qbdeletevariable qbgetvariable qbsetvariable qbworkervar
==== CL 9460 ====
@CHANGE: removing legacy commands from sbin-- qbworkervar, qbdeletevariable, qbgetvariable, qbsetvariable
==== CL 9459 ====
@NEW: added ip address column ("address") to the banned DB table
@NEW: enabled "qbadmin w -unremove <worker>" to work with hostname and IP address, in addition to the mac address.
BUGZID: 63703
==== CL 9458 ====
@NEW: adding QbTableVersion30.cpp to upgrade_supervisor.vcproj
New DB table schema definition file for rel-6.4
See also the previous changelist, CL9451
==== CL 9456 ====
@FIX: moved the location of QbTableVersion29.cpp (rel-6.3) inside the upgrade_supervisor.vcproj file from the incorrect "Resouces Files" folder to the proper "Source Files" folder.
It appeared as though the file was missing from the build.
(probably mostly only cosmetic, but was also was confusing).
==== CL 9455 ====
Back out changelist 9453, 9454
Changes were somehow not effectively made to the vcproj files, so trying again after backing off these CLs.
==== CL 9451 ====
@NEW: adding "name" column to the "banned" table
Note that this involves a DB table schema change. A new table definition, QbTableVersion 30, is added, and will be released with 6.4.0
BUGZID: 63681
ZD: 5271
==== CL 9449 ====
@FIX: fixed issue with removal of workers using the mac address (i.e. "qbadmin -worker remove <macaddr>") not working properly.
BUGZID: 63447
==== CL 9446 ====
@FIX: added "pgrp" modifying support to the supervisor code and the qbmodify() C++ API, qb.modify() Python API, and qb::modify() Perl API routines, and added a "-mpgrp <int>" option to the qbmodify command-line tool.
BUGZID: 63680
==== CL 9443 ====
@FIX: Added missing "qb.hostorder(id=JOBID)" routine to the python API.
==== CL 9442 ====
@FIX: modified to raise exception when parameter "fields" is not of type list.
BUGZID: 63627
ZD: 3998
==== CL 9440 ====
@FIX: variables such as $qb::jobid not working in callbacks on Windows
BUGZID: 63686
ZD: 5240
==== CL 9438 ====
@FIX: minor fix to a perl example, callback3.pl, so that the job cmdline works in Windows too.
==== CL 9427 ====
@FIX: added code to make sure all end-of-line in email data are CRLF (not just LF) in accordance to RFC2822.
This was causing notification emails to not work with some email servers, as they will not responding, and the communicating supe thread would just stall.
ZD: 5752
==== CL 9411 ====
@FIX: added code to chmod and open up the file permission of .out and .err files in the job log folder.
This was causing subjobs to fail on systems with "mounted" job log path, as the supervisor will initially create these files when when a subjob that previouly never started is retried (the supe writes "qube! - retry/requeue on blahblah...") under the "root" user's ownership with mode 644, and the workers who get the subjobs can't write to it.
ZD: 5965
==== CL 9407 ====
@CHANGE: set upper limit for mysql user filehandles to 70,000; 'open-files-limit' setting in my.cnf is only a suggestion, mysql can auto-determine to a larger number, but it's internal max value in 65535. Setting ulimit upper bound larger than 64K should prevent mysql from ever running out of file handles.
==== CL 9402 ====
@FIX: adding "qbhash" command to windows.
==== CL 9395 ====
@FIX: fixed issue causing the supervisor to crash at initialization, right after "finding other supes..." was printed in the supelog.
The fix was in one of the base commuinication library routines QbConnection::receiveUdp().
Sometimes, unknown/malformed data would be received on the UDP socket, and was causing the code to attempt to access beyond the buffer array (index out-of-bounds error).
ZD: 5638
BUGZID: 63305
==== CL 9370 ====
@FIX: recreate the pfx_dw stored procedures and functions on Windows, as the MSI installer wipes them out during an upgrade.
==== CL 9342 ====
@FIX: fixed a supe thread crashing issue, when global_host or license_host resource tracking is used.
ZD: 5749
==== CL 9334 ====
@FIX: add error handler for MySQL error 1146 "Table 'x' doesn't exist" for work and cpu time calculations for job data collector script
@NEW: increment datawarehouse version to 10 to allow for installing this patch into existing databases
==== CL 9318 ====
@FIX: fixed crash bugs that were introduce when the "dying" state was implemented for 6.3.1.
ZD: 5794
==== CL 9312 ====
@FIX: add mail template for auto-wrangling emails to the installers
==== CL 9299 ====
@FIX: add mail template for auto-wrangling emails to the installers
==== CL 9277 ====
@NEW: increase file handle limit for mysql user on Linux installs to 64K
==== CL 9274 ====
@FIX: create global resource tables in data warehouse DB if they don't exist; creation was failing to happen in new DB installations.
==== CL 9265 ====
@FIX: fixed job-level history not being recorded into .hst file.
(Bug was introduced in CL9145, 9146)
ZD: 5609
==== CL 9261 ====
@CHANGE: cut down on the cmdline & cmdrange jobtypes' stdout; don't print 'LOG: ...' lines, make regex summaries much clearer, change printing or regex's to stderr to make it clearer that they're not actual errors, but rather things being searched for in the stderr stream.
==== CL 9252 ====
@FIX: properly find qb.conf on Windows versions Vista and later when unable to contact the supervisor directly.
==== CL 9245 ====
@FIX: GUI changes to be able to handle when supervisor host goes down, and both supervisor and MySQL server are unavailable. Also fix jobList not refreshing on down supervisor.
==== CL 9241 ====
@FIX: fix GUI crashbug in MySQLConnect when supervisor does not answer a qb.ping
==== CL 9239 ====
@FIX: global resource tables were not getting created in new instances of the datawarehouse db, only on upgrades.
==== CL 9232 ====
@FIX: fixed example python code (jobSubmit06.py) to work on Windows too.
==== CL 9211 ====
@FIX: added code to prevent the QbQueue::getSubjobReadyfindReady() routine from returning the same subjob to be dispatched over and over.
This was causing the findSubjobAndReserveJob() and startJob() routines to
hit the "subjob [N] seems to be already assigned" situation, and cause
threads to enter a long, sometimes semi-infinite, sleep-and-retry loop.
Fixed by adding code in the startJob() routine to quickly update the subjob
status when the the assignJob() returns QB_ASSIGN_OK (i.e., worker says it
has accepted the subjob), instead of waiting until the worker later reports
that the subjob is "running" via the STATUS_JOB message, which can take
more than several seconds on a busy farm.
Also reduced the number of maximum retries to 3 (MAX_ATTEMPTS), in the
situations where a subjob "seems to be already assigned" or when a worker
host says it's busy (QB_ASSIGN_BUSY). This prevents the threads to get
stuck for 10 or more seconds in a sleep-retry loop, and allow them to give
up quickly and move on.
ZD: 5449
==== CL 9198 ====
@FIX: fixed issue with non-node-locked licenses ("FF:FF:...") not working (since 6.3.0)
==== CL 9174 ====
@INTEG: rel-6.3->main CL 9173
-----
@FIX: ensure that mail sent by "qbadmin --emailtest" is RFC2822-compliant (no bare LF's, only CRLF)
==== CL 9173 ====
@FIX: ensure that mail sent by qbamdin --emailtest is RFC2822-compliant (no bare LF's, only CRLF)
==== CL 9161 ====
@NEW: add support for new 'dying' state into the GUI
==== CL 9150 ====
@INTERNAL: QbDebug::filename(QbString) took if statement out, so resetting _filename is allowed
==== CL 9145 ====
@FIX: disabled logging to /var/spool/qube/{host,user}, as it was creating large log files and causing sluggish performance.
An option to enable these logs may be made available in the future.
==== CL 9142 ====
@FIX: fixed issue where global resources tracking drift sand more subjobs than can be accomodated by the actual global resource count is dispatched.
ZD: 5074
==== CL 9133 ====
@INTERNAL: CentOS support for "buildpyc" in rpm/quberpm.pm
==== CL 9105 ====
@NEW: A new transitional "dying" state for jobs that have been ordered to be "killed", but still being processed by the system
==== CL 9084 ====
@CHANGE: increase MySQL max_allowed_packet value from default of 1MB to 64MB to decrease frequency of "MySQL server has gone away (2006)" error messages.
==== CL 9083 ====
@CHANGE: increase MySQL wait_timeout value from default of 8 hours to 36 hours to decrease frequency of "MySQL server has gone away (2006)" error messages.
==== CL 9066 ====
@FIX: fixed "cpus" (subjob) count inaccuracy when a job's "cpus" was modifed down and then up.
For example, if a job with initially 10 "cpus" was reduced to 5, then
subsequently increased to 6, the system had inaccurately recomputed the
subjob count to be 10.
==== CL 9058 ====
@FIX: renaming logs during rotation would fail on Windows
==== CL 9037 ====
@FIX: rename the globalResource_fact table to be all lower-case; causes issues stored procedure PFX_CREATE_DATASUBSET_TABLE() which errors out with "ERROR 1050 (42S01) at line 1: Table 'globalresource_fact_12h' already exists" (note lower-cased name)
==== CL 9016 ====
@NEW: adding license agreement for 3rd-party software
@NEW: also adding our own License.rtf to the docs dir.
==== CL 9013 ====
@NEW: added description of supervisor_job_flags in the qb.conf.template file
==== CL 9010 ====
@FIX: fixed memory bloat issue in supervisor threads on start up, on farms with many jobs.
In some cases, it had been reported that each supe thread was taking up 500+ MB.
==== CL 8939 ====
@FIX: fixed another small "hole" that could cause race-conditions to dispatch a single subjob more than once
ZD: 4783
BUGZID: 63657
==== CL 8937 ====
@FIX: supe issue where the same subjob can be dispatched more than once to worker(s).
ZD: 4783
BUGZID: 63657