Home » Headline, Houdini, Python

Deadline Submission Tool For Houdini

7 July 2017 1,722 views 2 Comments

I’ve released a new tool for Houdini imaginatively named Hou Farm that provides a more convenient workflow for submitting jobs to Deadline. You can find the hou_farm project on Github: https://github.com/fxnut/hou_farm

The main reason for writing it was to get away from the monolithic submission form that Thinkbox provide. While fully featured, it does suffer from presenting too much information to the user, and it isn’t able to remember job submission settings on a per ROP node basis meaning that you need to manually retype the submission settings each time.

Hou Farm works by patching existing ROP nodes using a shelf tool. It respects simple dependencies between ROP nodes by looking at their connection tree.

Here are some screenshots of the interface for the Mantra node:

  

Note that the Deadline submission settings are all stored under a new parameter folder, leaving the original ROP parameters untouched. This means that Hou Farm will easily work across multiple versions of Houdini without having to manage Digital Asset compatibility.

These screenshots show the IFD and Mantra tabs when the “Advanced” folders are opened:

 

At the moment, this tool only supports Deadline but it can potentially be expanded to support other render farms too. Platform-wise, it was developed and tested under Windows, but there’s no reason it can’t work under Linux or OSX, albeit I suspect, with a bit of tweaking.

For more information, I’ve copied in the contents of the project’s readme.md file below.

Features

  • Do away with the cluttered and over-complicated submission dialog. Instead, it places all the submission settings on the ROP nodes. This saves having to remember the settings for each ROP task that you submit to the farm.
  • Hides many of the less common Deadline settings under an Advanced section (hidden by default)
  • Supports the following ROPs (not all fully tested yet, although they should be fine):
    • Mantra ROP
    • Geometry ROP (and SOP version)
    • Alembic ROP
    • IFD Archive ROP
    • Comp ROP
    • Channel ROP
    • BakeTexture ROP
    • OpenGL ROP
    • DOP ROP
  • Respects simple dependency indicated by ROP tree network. Creates jobs on a ROP by ROP basis, and does not split into individual frame dependencies.
  • Allows submission of:
    • Individual nodes
    • Nodes above the submitted ROP node
    • The entire connected ROP tree.
  • Shelf tool patches existing ROP nodes with settings. No custom nodes
  • Deadline interface for the nodes is built on the fly from config file. Easy to reuse parameter layouts and add new nodes.
  • Allows you to split tasks into N chunks, or automatically calculate number of chunks based on number of frames.
  • Performs validation of ROP nodes before submission and reports errors and warnings. For example, it catches common errors such as:
    • Missing camera
    • Disabled IFD export
    • Using default IFD path
    • Missing specified Deadline pool
    • Camera background image enabled
  • Gracefull fallback should hou_farm or Deadline not be present on the system. Won’t break your scene or generate annoying errors.
  • Allows you to develop/test Hou Farm without Deadline installed by defining HOUFARM_VIRTUAL_DEADLINE in the environment.

Installation

  1. Open hou_farm_example.env and copy the lines into your own $HOME/houdini/XX.Y/houdini.env file.
  2. Change the HOUFARM environment variable to point to the location of hou_farm.
  3. Launch Houdini.
  4. Add the hou_farm shelf to your toolbar at the top by clicking on the + button and selecting Shelves->Hou Farm.

Usage

  1. Now that hou_farm is installed you should be able to select ROPs in the network view and use the hou_farm “Patch” shelf tool to add the settings to the ROPs.
  2. Patching a node puts all existing parameters under a top-level folder, and creates a new folder called Deadline where you can find all the submission settings.
  3. When you first do this in a session of Houdini, hou_farm will cache the Deadline pools and groups to save having to re-interrogate the Deadline database everytime you select a new node. If you need to update the lists, then go to the Settings tab and click on “Refresh Pools/Groups”.
  4. Once you’re happy with the settings, just click on the appropriate Submit button to send the scene to the farm.

Future Work

  • Better usage documentation
  • Provide checkboxes to allow validation tests to be enabled/disabled.
  • Add support for custom validation python code.
  • Support Fetch ROP
  • Add Wedge functionality (possibly through custom ROP rather than existing Wedge ROP)

Development Notes

  • Only tested on Windows so far. In theory, it should work fine on Linux and OSX, although I’m sure there’ll be some fixes required.
  • Mostly written a functional style, and relatively little OOP. This was done mainly because I’ve been using OOP for a long time and after working on a pure C project, I wanted to experiment with seeing how necessary OOP actually is. Sometimes OOP just gets in the way. May decide at some point to rework some of this into classes to reduce code bloat.
  • Hou_farm was written to potentially expand to support other farm software. It would still need a lot of factorisation for better reuse of much of the Deadline code.
  • There’s a makefile for generating code documentation using Sphynx in the docs folder.

SaveSave

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

2 Comments »

  • Fernando said:

    Thanks for this, it is waaaay better than the default deadline submission tool in terms of interface and ease of use, looking forward to the wedge functionality update! 😀

  • AndyN (author) said:

    Thanks Fernando! Glad it’s useful to you.

    Unfortunately I’m having to put development on hold at the moment because we don’t use Deadline here at Framestore. So without access to a working farm, it’s incredibly difficult to develop and test new features such as the Wedge ROP.

    But… if someone out there wants to start contributing to the project, I’m very happy to assist with getting it working.

    In the meantime, hope you get some good use out of it.

    Cheers!
    Andy

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.