#################################################
@RELEASE: 6.5-3
#################################################
==== CL 12442 ====
@FIX: Fixed a corner-case MySQL permission problem with OSX/Linux supervisor and the qube_readonly user.
Fixed by adding a "GRANT SELECT" with an explicit hostname (fetched via
"SELECT @@hostname"), as in:
GRANT SELECT ON *.* TO 'qube_readonly'@'mysqlserverhostname'
JIRA: QUBE-438
==== CL 12358 ====
@FIX:Fixed example python scripts so import of qb module will work in most cases.
==== CL 12347 ====
@FIX: pyCmd* jobtypes report all subsequent frames as failing when a 'regex_error' is matched and a frame is marked as failed
==== CL 12339 ====
@FIX: fixed inaccurate worker host memory reporting on Windows platforms
ZD: 11367
==== CL 12333 ====
@FIX: worker shutdown code (QbWorker::hostShutdown() and sendHostReport()) will now give up a lot quicker when being unable to contact the supervisor, instead of retrying for a long time.
==== CL 12322 ====
@FIX: issue where job instances don't terminate properly when very early kill/interrupt orders come in.
Sometimes interrupts and kills can come in before the worker has a chance to properly complete the launching process of the proxy.exe process and its main thread, causing unexpected behavior, such as a never-dying job instance.
ZD: 11409
==== CL 12315 ====
@FIX: bug in initialization code of the QbJob class that messed up comparisons of jobs when sorting, which, among other things, prevented FIFO/FCFS ordering to be compromised. Now FIFO dispatching behavior should be more closely followed by jobs of equal priority (although not 100% strictly, due to the nature of the multithreaded architecture of the supervisor).
ZD: 11259
@INTERNAL TWEAK: added more debugging code to QbSupervisorQueue module.
==== CL 12311 ====
@FIX: adding in ubuntu support: use bash explicitly rather than sh, specify 'awk' in location found on all OS's
==== CL 12306 ====
@FIX: issue where auto-expanded subjobs (instances) don't inherit the "retrysubjob" value set in the parent job, causing them NOT to auto-retry properly on failure.
ZD: 11292
==== CL 12298 ====
@FIX: Python API routines, such as qb.retrywork(), expecting workID as input would behave erroneously (such as retrying ALL agenda items on ALL jobs) when input a subjobID instead. Vice versa for routines expecting subjobIDs, such as qb.retry().
These
ZD: 11372
==== CL 12295 ====
@NEW: add support for new 'exiting' status
==== CL 12272 ====
@FIX: unreliable behavior when frequently modifying "cpus" of jobs up and down.
ZD: 11288
==== CL 12257 ====
@FIX: bug where auto-expand subjobs are incorrectly auto-retired, and in turn caused them NOT to expand any more.
ZD: 11217
==== CL 12255 ====
@FIX: issue where, if some intermediate job processes crash and die unexpectedly, other job processes may be missed by the cleanup code and left behind as zombies.
ZD: 11236
==== CL 12250 ====
@FIX: WorkerConfigFile makes a better effort at finding the worker config file, previously would save to default location when the file is actually in a non-default location as specified by supervisor_worker_configfile.
==== CL 12237 ====
@FIX: avoid inserting duplicated values into the 'outputPaths' for a frame when retried
==== CL 12232 ====
@FIX: "UnboundLocalError: local variable 'qb' referenced before assignment" - issue experienced by single customer on linux, re-importing qb module in main() resolves the issue. ZD# 11218
==== CL 12230 ====
@FIX: additional fixes to remedy "retrywork" issue with maya (and possibly other Perl-API based) jobs. See also the previous CL12228
==== CL 12228 ====
@FIX: automatic retry of agenda via "retrywork" not working properly in perl-based backends.
ZD: 11167
==== CL 12226 ====
@FIX:Fix issue where job_cleanup script would fail if run on a supervisor that did not have the MySQLdb python module installed.
==== CL 12219 ====
@FIX: "sre_constants.error: bogus escape (end of line)" - python-based jobs can crash on Windows at startup if path wrapped in QB_CONVERT_PATH() ends with a fwd-slash and has being converted to a back-slash
==== CL 12177 ====
@FIX: Additional changes to support proper Windows privilege enabling, added in CL12176
==== CL 12176 ====
@FIX: Add call to Windows' AdjustTokenPrivileges() to explicitly enabled required privileges before launching job instance (proxy) process
==== CL 12098 ====
@FIX: support negative frame range in QB_* token parsing
==== CL 12082 ====
@FIX: issue where "modify"-ing the "cpus" value of a running job may incorrectly retire more instances than asked for.
This was due to race conditions of supe threads, and in extreme cases, was
prematurely retire-ing ALL instances of a job while there are still pending
agendas, resulting in the job's instances to be all "complete" but the job
itself to become "failed" since there are still pending agendas.
ZD: 10868
==== CL 12065 ====
@INTERNAL TWEAK: added/modified/corrected comments and symbol names for readability
#################################################
@RELEASE: 6.5-2
#################################################
==== CL 12016 ====
@FIX: worker and supervisor install do not register for all users on Windows
==== CL 12006 ====
@FIX: ERROR 1146 (42S02) at line 87 in file: './create_job_fact.sql': Table 'pfx_stats.memusage' doesn't exist - swap order of table assignment and creation, some versions of MySQL are error'ing
==== CL 11989 ====
@FIX: worker_drive_map and worker_path_map not correctly saved via "Configure local host", format to match API updatelocalconfig expectations
==== CL 11987 ====
@FIX: localized the _user_duties and _prgp_duties IntHash variables to the queuereject() routine for thread-safety, from being data members of the supervisor class.
ZD: 10342
==== CL 11986 ====
@FIX: added code to appropriately handle timing issues where a command,
such as preemption, can be issued multiple times by different threads on
the same running subjob, leaving those jobs to be in odd states. One common
symptom was seeing the "aberrant report" message in the supelog, and those
jobs getting stuck in the "running" state despite all the frames being 100%
done.
==== CL 11985 ====
@FIX: converseWorkerWithRetries() and converseSubSupervisorWithRetries()
routines were fixed so that they properly return success when there are no
communication errors. These routines were retrying when the server
responded with a rpy.tag() of QB_MESSAGE_ERROR, which doesn't mean there
was a communication error, but rather means that the server encountered
some general internal error, causing unwanted retries.
ZD: 10527
==== CL 11982 ====
@FIX: contradictory job log entries saying a failed frame is being reported as complete when a few lines ago it was actually (correctly) reported as failed.
==== CL 11980 ====
@FIX: QB_CONVERT_PATH() not getting evaluated when worker_path_map is undefined or empty
==== CL 11963 ====
@FIX: catch jobs with package data the cause _qb.packageStrToDict to raise an exception
==== CL 11961 ====
@CHANGE: add additional sanity checks to cleanup script, limit number of log directory deletions to a fraction of total jobs in qube, can be overridden by option flag.
==== CL 11931 ====
@CHANGE: create the backfill_fact (supervisor dispatch efficiency) dataWarehouse "12-hour" table every 5 minutes rather than every 15 to keep the chart data more current - full-range table is small enough to support this
==== CL 11915 ====
@FIX: fixed cross-dependency created in CL11893.
JIRA: QUBE-176
==== CL 11908 ====
@CHANGE: changed/added code to set up the following default my.cnf parameters
all OSs:
-------------------
query_cache_size = 0 # disable the query cache, hit rate is almost 0% due to qube being very write-intensive
thread_cache_size = 16 # acts like supervisor_idle_threads
Linux-only
-------------------
table_open_cache = 2500 # mysql will cache the file handles necessary to hold this number of tables f/h's
open_files_limit = 50000 # table_open_cache will drive the number of open files, MyISAM needs a max of 2 per table, but MySQL can also open other files past the table_open_cache*2 value - refer to:http://dev.mysql.com/doc/refman/5.1/en/table-cache.html
JIRA: QUBE-175
==== CL 11899 ====
@FIX: made the path map translations case-insensitive on OSX and Windows platforms.
@NEW: added 3rd optional parameter to QbString::replace(), which specifies the case-sensitivity, which defaults to TRUE.
JIRA: QUBE-177
==== CL 11895 ====
@NEW: exposed the C API routine "qbisadmin()" as "qb.isadmin()" in Python API and "qb::isadmin()" in Perl API.
JIRA: QUBE-174
==== CL 11893 ====
@CHANGE: "qbadmin {s|w} -configuration" now displays both the integer AND string values of all "*_flags" (such as "supervisor_flags") parameters for readability
JIRA: QUBE-176
==== CL 11856 ====
@FIX: added code to fix jobs getting stuck in the "dying" state, that can occur due to race conditions.
Dispatched instances of jobs that were requested to be "killed" before they
properly finished starting up on the workers were ending up getting stuck
in the "dying" state.
ZD: 10369
==== CL 11850 ====
@FIX: C4D AppFinder jobs crash when paths or filenames wrapped in QB_CONVERT_PATH() start with a number
==== CL 11829 ====
@FIX: Issue with grid jobs where some instances would start running multiple times on the dispatched host, causing the job to eventually fail.
ZD: 10325
==== CL 11820 ====
@FIX: disable permission check of worker_logpath, as it was creating false-alarms and putting the worker to be in panic mode unnecessarily.
ZD: 5445 5236
BUGZID: 63683
See also CL9234
==== CL 11815 ====
@FIX: on Linux in the /etc/init.d/worker script, we're now allowing a longer timeout (15 seconds) for the worker to shutdown cleanly before forcefully killing (i.e. "kill -9") the processes.
The default short timeout of 3 seconds was not sufficient on many systems for all child worker threads to exit and the main thread to release the running subjobs and report to the supervisor that it's "down".
JIRA: QUBE-90
==== CL 11807 ====
@FIX: added dependency ("requires") on the "expat" package for qube-core RPM packages.
JIRA: QUBE-68
ZD: 8499
==== CL 11801 ====
@FIX: fixed qbjoborder() routine so that it respects the queuing algorithm's job-host pair rejection routine, queuereject().
This bug, for example, was causing the routine to return jobs that shouldn't qualify to run on
the given host because of the "worker_restrictions" settings of the worker.
ZD: 10231
JIRA: QUBE-158
==== CL 11795 ====
@FIX: issue where the python API qb.convertpath() will cause a bus error (crash) in the caller, if called with no args.
@FIX: issue where the 2-argument invocation of qb.convertpath() was not
working, and may cause a bus error. Turned out to be a bug in the
internal conversion routine _qb_py_dict_pathmap().
==== CL 11793 ====
@FIX: bug with modifying user and group permissions.
Operations such as adding or deleting a group or users would generate an
error message in the supelog, like the following:
[Sep 20, 2013 11:39:35] HOSTNAME[25107]: group permissions modified for user foobar by user USERNAME
[Sep 20, 2013 11:39:35] HOSTNAME[25107]: ERROR: database query error: 127.0.0.1 via TCP/IP - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') = 'test' AND LOWER(user) = 'foobar'' at line 1 (1064)
[Sep 20, 2013 11:39:35] HOSTNAME[25107]: SELECT access FROM grp WHERE valid = 1 AND LOWER(name)) = 'test' AND LOWER(user) = 'foobar'
JIRA: QUBE-157
==== CL 11790 ====
@FIX: fixed inaccurately reported host.processor_speed (CPU frequency in MHz) property on OSX workers.
JIRA: QUBE-153
==== CL 11788 ====
@CHANGE: added "GRANT" statement to "GRANT SELECT ON *.*" to the qube_readonly user on "localhost".
JIRA: QUBE-105
==== CL 11771 ====
@FIX: problem where it was impossible to undefine worker_properties and worker_resources once they were defined in qbwrk.conf or qb.conf, even if the lines were removed from the config files.
JIRA: QUBE-85
ZD: 10227
==== CL 11767 ====
@FIX: Setting "worker_cpus=0" or removing a "worker_cpus=N" line from qbwrk.conf had no effect, and the previous setting would get stuck.
JIRA: QUBE-80, QUBE-112
==== CL 11748 ====
@FIX: helloWorld example jobtype can't create job archive file job.qja below QBDIR/examples, area is read-only, write to a temp directory
==== CL 11740 ====
@CHANGE: remove dependencies between Windows MSI installers, any qube component can be installed or uninstalled independent of the others
==== CL 11733 ====
@FIX:qbtail.py now prints out the help screen if run without arguments.
#################################################
@RELEASE: 6.5-1
#################################################
==== CL 11687 ====
@NEW: add support for Adobe's Creative Cloud 'CC' version numbering scheme
==== CL 11685 ====
@FIX: erroneous "timecumulative" of instances (subjobs)
JIRA: QUBE-148
==== CL 11682 ====
@FIX: removed "Configure Qube" menu item from workertray.exe
==== CL 11654 ====
@NEW: added worker_boot_delay to qb.conf.template
==== CL 11631 ====
@CHANGE: modified the new worker_boot_delay (CL11605) to default to 0 seconds on ALL platforms.
ZD: 9386
JIRA: QUBE-118
==== CL 11614 ====
@FIX: The "Administrator" user didn't properly get permissions to manipulate others' jobs, due to case sensitivity.
JIRA: QUBE-142
==== CL 11605 ====
@FIX: added worker_boot_delay to qb.conf, which specifies the number of seconds to artificially delay (i.e. sleep) before the worker boots.
This was added in order to work around a boot-time timing issue with networking/hostname assignment on the Mac OS X platform, where many machines are incorrectly identified as "localhost".
As such, by default, worker_boot_delay is set to 30 seconds on the Mac, and 0 (i.e., no delay) on other platforms.
To override the default, worker_boot_delay *must* be set in the local qb.conf file on each worker-- setting it in qbwrk.conf won't work.
ZD: 9386
JIRA: QUBE-118
==== CL 11484 ====
@FIX: appFinder jobs error out at setup with "raise error, v # invalid expression, bad group name" exception raised by python re module
==== CL 11470 ====
@FIX: random Worker crashes on Windows DU mode-- worker "commiting suicide", i.e. killing it's own worker.exe process when removing job processes.
==== CL 11460 ====
@FIX: added code to "generate movie" jobs, to support frame ranges that don't start at 1 with conversions using ffmpeg
@FIX: also added a leading "." to the movie_ext choice strings, which is required.
ZD: 9745
==== CL 11428 ====
@CHANGE: large re-write, not backward-compatible with previous argument spec
@CHANGE: will only delete jobs where all jobs in the job's pgrp completed more than X days ago
@CHANGE: added --removeOrphanedLogs functionality, will delete log directories for jobs no longer present in Qube
@CHANGE: does not require MySQLdb module, but runs faster with it.
==== CL 11424 ====
@FIX: patched crash bug in supervisor (QbSupervisorQueue::_subjobProcReservation())
ZD: 9654
==== CL 11394 ====
@CHANGE: add deprecation warnings to python qb module accessor method, prints once per location
==== CL 11390 ====
@FIX: Global resource collector error occurs on MySQL servers running in STRICT mode: "Field 'total' doesn't have a default value"
#################################################
@RELEASE: 6.5-0
#################################################
==== CL 11260 ====
@NEW: add Python/Perl API access to the new "timecumulative" subjob and work data
==== CL 11257 ====
@NEW: add support to the data warehouse for the new "cumulative time" instance and work columns
==== CL 11254 ====
@NEW: added ReadMe.rtf for the osx supervisor installer, so that the person installing the supervisor is made aware of the upgrade potentially taking a long time to complete.
==== CL 11247 ====
@NEW: add tracking of cumulative time spent in the "running" state for subjobs (instances) and work (agenda items), stored as an additional DB field in the Nsubjob and Nwork tables.
These values are computed by taking the elapsed number of seconds since the last start time of the instance/work, and then multiplying it by the number of actual job slots occupied by the instance, as dictated by the "host.processors" reservation.
JIRA: QUBE-124, QUBE-125
==== CL 11240 ====
@NEW: add support for job tags to the data warehouse schema
==== CL 11237 ====
@NEW:qb.query module's jobinfo function now accepts a "where" argument that will pass through a mysql "where" statement (without the "where" word). Example: where="name like 'foo' OR user like 'foo'"
==== CL 11219 ====
@FIX: issue with automount in desktop user mode.
@INTERNAL: also cleaned up some Win32 automount code.
ZD: 9434
==== CL 11217 ====
@CHANGE:Python API change: all python classes can now be constructed without data. In other words, one can create empty objects.
==== CL 11216 ====
@NEW: add support for real-time log parsing and progess percentages to job instances for non-agenda-based jobtypes, currently only supported by python-based jobtypes
==== CL 11208 ====
@NEW: label text for worker parameter widget changes colour to indicate a value which will be saved
@FIX: properly support worker_path_map and worker_drive_map in new qbwrk.conf configuration dialog
@FIX: properly indicate when the value for a given parameter varies between workers, even if it's not defined for a worker but defined for others
@FIX: support greater than 5 mapping definitions for worker_path_map and worker_drive_map
==== CL 11201 ====
@NEW: add DB conversion scripts (to add new subjob table columns introduced in 6.5, DB version 32) that run when the rpm/pkg/msi supe installer runs, to provide forward-compatibility for old (pre-6.5) job/subjob data
JIRA: QUBE-119
==== CL 11200 ====
@TWEAK:Minor Python API change: You can now create an empty qb.Job()
==== CL 11190 ====
@CHANGE: added "enable_windows_job_object" flag, and deprecated "disable_windows_job_object"
The "disable_windows_job_object" flag is silently ignored. Windows Job
Objects are always disabled now, unless the job explictly specifies
otherwise with the new "enable_windows_job_object" flag.
JIRA: QUBE-117
==== CL 11185 ====
@CHANGE: modified user name and group name authentications to be case-insensitve
JIRA: QUBE-98
==== CL 11183 ====
@NEW: add supervisor_default_hostorder parameter to qb.conf
JIRA: QUBE-113
==== CL 11178 ====
@CHANGE: Added "post" to the default callback language list, supervisor_language_flags
==== CL 11175 ====
@NEW: advanced worker resource reservations, including N-M, N+M and N* reservation specifiers, and live tracking of resource "allocations" and "slots" (actually allocated host.processor value) of running job instances.
JIRA: QUBE-91
==== CL 11174 ====
@NEW: add new worker DB table schema, QbTableVersion8
==== CL 11155 ====
@NEW: Add QB_JOBSLOTS and QB_ALLOCATIONS environment variables to be set, indicating the initial jobslot allocation and the more general initial resource allocations, respectively, when jobs execute.
==== CL 11127 ====
@FIX: editing supevisor config with WranglerView->Admin->Configure causes the 'submit_job' privilege to be removed from supervisor_default_security
@CHANGE: Admin->Display Config (local) changed to Admin->Display Running Config, shows supervisor and/or worker running config if these services are running locally
@CHANGE: Admin->Configure (Local) changed to Admin->Configure Local Host
@INTERNAL: add all supervisor and worker flag values to qb.utils.flags, now used directly by configuration dialog, instead of the config dlg items being order-dependent. Allows for sorting configuratin dialog items alphabetically for ease of use.
@NEW: Admin->Configure Local Host now creates a timestamped backup of the qb.conf file in the same location as the original
@NEW: Admin->Configure Local Host is now disabled on Windows and Linux if not invoked by root (linux) or Admin-equivalent (Windows)
@COSMETIC: File->Install AppUI menu items now sorted alphabetically
==== CL 11122 ====
@NEW:New utility qbtail.py in $QBDIR/utils. This is a *nix tail-like utility implemented in Python that runs on OS X, Linux, and Windows.
==== CL 11096 ====
@NEW:Updated and cleaned up all Python API examples in QBDIR/examples/python
==== CL 11088 ====
@INTEG: dev-supervisor-additional-job-params>main,CL11087
----
@NEW: added prod_{show,shot,seq,client,dept,custom[1-5]} fields to the job object.
The C++, Python, and Perl APIs have been updated.
The qbsub, qbjobs, and qbmodify commands and their online help text have been updated.
JIRA: QUBE-79
==== CL 11024 ====
@NEW: supervisor reread qb.conf file feature.
The qbadmin command has been updated with a "-reread" option, to be called as in "qbadmin s -reread" to
instruct the supervisor to reread its qb.conf file and update the dynamically modifiable parameters, which are
as of this writing:
* qb_domain
* supervisor_default_group_security
* supervisor_default_p_agenda_priority
* supervisor_default_pgrp_subjob_limit
* supervisor_default_priority
* supervisor_default_security
* supervisor_default_user_subjob_limit
* supervisor_flags
* supervisor_global_resources
* supervisor_highest_user_priority
* supervisor_job_flags
* supervisor_max_priority
* supervisor_p_agenda_max
* supervisor_pgrp_subjob_limits
* supervisor_user_subjob_limits
* supervisor_verbosity
* supervisor_worker_configfile
(Note: DB schema change was involved, and QbTableVersion32 was added)
JIRA: QUBE-92
==== CL 10972 ====
@CHANGE: modified qbworkerpathmap() to return the localhost's worker_path_map when called outside of a jobtype back-end environment.
JIRA: QUBE-95
==== CL 10953 ====
@FIX: remove digit/number from worker's journal file name (worker6.jnl -> worker.jnl)
==== CL 10946 ====
@TWEAK: error code now prints when QbTrackOSX in trackAssignment() encounters an error.
==== CL 10945 ====
@FIX: fix yet another issue with previous CL, concerning graceful worker shutdown.
Also swithced a couple of calls to qbvcout to qbvout so that more useful info such as timestamp and pid print.
==== CL 10943 ====
@INTERNAL TWEAK: added/corrected comments, and removed unneeded #ifdef/endif macro, while working on previous CL
==== CL 10942 ====
@CHANGE: modified worker shutdown code so that it immediately returns all running subjobs to the supe, and report a status of "down", so the supervisor marks it "down" promptly.
JIRA: QUBE-90
==== CL 10927 ====
@FIX: changed the initial state of a newly dispatched instance on a worker to "running" instead of "pending", so that early calls to qbjobobject() in the back-end code will return "running" as the job's status
JIRA: QUBE-45
==== CL 10827 ====
@NEW: a new interface for configuring multiple workers and writing out the qbwrk.conf
@FIX: add "convert_path" flag to client_ and supervisor_job_flags control
@FIX: default value in config UI for worker_max_threads was 8, now 256
@FIX: "Confgure on Supervisor" worker menu item is only enabled if user has the qube "admin" privilege AND is on the supervisor
@COSMETIC: all checkbox lists from "Choices" buttons are now sorted alphabetically
==== CL 10667 ====
@CHANGE:Python API-level change: QBObject is now simply a python dictionary. It no longer re-implements any functions
@CHANGE:Python API-level change: qb.updatelocalconfig now uses the subprocess module rather than the depricated popen2.Popen4.
==== CL 10589 ====
@FIX: job list not updating when switching supervisors, always show jobs from the default supervisor.
==== CL 10255 ====
@INTEG: rel-6.4 -> main
-----
@NEW: pyCmdline and pyCmdrange do run-time path translation
==== CL 10061 ====
@CHANGE: index all datawarehouse fact tables on time_sk column, since it's so frequently accessed.
==== CL 10056 ====
@FIX: PFX_CREATE_DATASUBSET_TABLE doesn't use an indexed column for the WHERE clause, now does an INNER JOIN to the time dimension table
==== CL 10037 ====
@FIX: allow qube-core to be repaired if other qube products are already installed
==== CL 9730 ====
@TWEAK: modified so that worker name and IP print when job is accepted by worker, in assignJob()