Importing Python Modules

Python modules offer a great way to reuse code. Often you’ll find that you want to share code between your plugins, especially in a pipeline situation.

XSI isn’t particularly flexible with how it lets you import modules. At the moment, you have to install your custom written modules in a folder and use the PYTHONPATH environment variable to point to that location (or add it to the list of paths stored in sys.path at run time.)

Wouldn’t it be good if we were able to import Python modules so that:

• You can keep your modules with the plugin that uses them.
• You never have to modify your Python path, even when you move the installed plugin to a different location.
• The module is only loaded when you need it.

Of course, you can just use a standard XSI plugin and register some commands, but there are situations where it might be preferable to use a module. For example:

1. If the functions to be included are very small helper functions that need to be called repeatedly. It might be best to avoid registering commands since they have a slight performance overhead to their execution.
2. If the functions are non XSI specific and don’t make much logical sense to be added as XSI commands.
3. If the functions need to be pass or return Python types such as classes, tuples, sets, lists and dictionaries. There are workarounds for these, but they don’t really assist with making your code clean and maintainable.
4. If you want to keep your code hidden, you can put your implementation inside the module and only distribute the .pyc compiled version.

A new way to import modules

The good news is that you can use the following Python class to mimic the behaviour of an imported module.

import win32com.client
from win32com.client import constants

import os.path
import imp

#Note: You only need the following line if you're
#putting this code in a file that isn't an XSI plugin.
Application = win32com.client.Dispatch("XSI.Application").Application

class Import:

    def __init__(self, pluginName, moduleName):
        self.pluginName = pluginName
        self.moduleName = moduleName
        self.module = None

    def __getattr__(self, name):
        #Has the module already been loaded?
        if self.module:
            return self.module.__dict__[name]

        #Try and load the module
        path=""
        try:
            path = Application.Plugins(self.pluginName).FileName
        except:
            Application.LogMessage("[Plugin Python Module: " \
                             +self.pluginName+"] " \
                             +"Plugin name doesn't exist or hasn't been loaded" \
                             ,constants.siError)
            return None

        #Import the module
        result = imp.find_module(self.moduleName, [os.path.dirname(path)])
        self.module = imp.load_module(self.moduleName, result[0], \
                                      result[1],result[2])

        if not self.module:
            Application.LogMessage("[Plugin Python Module: " \
                             +self.pluginName+"] " \
                             +"Couldn't load module: "+self.moduleName \
                             ,constants.siError)
            return None

        #Evaluate the attribute
        return self.module.__dict__[name]

Assuming you’ve placed the above code into a globally accessible module called xsiimport, then you would use it like this:

Note: I’ve highlighted the specific bits of the code that are relevant

import win32com.client
from win32com.client import constants

import xsiimport

g_PluginName = "TestCommandPlugin"

#Think of this next line as a normal import statement
testpymodule = xsiimport.Import(g_PluginName, "testpymodule")

def XSILoadPlugin( in_reg ):
    in_reg.Author = "Andy"
    in_reg.Name = g_PluginName
    in_reg.Major = 1
    in_reg.Minor = 0

    in_reg.RegisterCommand("MyCommand","MyCommand")
    #RegistrationInsertionPoint - do not remove this line

    return true

def MyCommand_Execute():
    #Use testpymodule as you would any other python module

    #Call functions
    result = testpymodule.DoSomething("inputString")

    #Create classes
    myClassInst = testpymodule.MyClass()

As shown, the easiest way to use this class is to import it from a module located in your standard Python path. As a result this would generally only work in a studio environment where you are able to make sure your art team are working on the same file system and with the same environment variables.

If you intend to distribute your plugin, then the alternative is to copy the class definition to each of your plugins that needs to use it. While untidy, the benefits it offers are probably worth it, but I’ll leave it to you to decide.

How it works

So if you’re interested in what’s happening under the hood, I’ll briefly run through what’s happening.

This line in our plugin:

testpymodule = xsiimport.Import(g_PluginName, "testpymodule")

doesn’t actually do anything apart from create an instance of the Import class. In its constructor, it just stores the name of the plugin and the name of the module we want to import.

The reason the class doesn’t load the module at this stage is because the plugin probably hasn’t been registered with XSI yet (don’t forget, we’re still running code in global scope here). Therefore, we can’t ask XSI for the file path of the plugin using the Application.Plugins collection.

The module only gets loaded when we ask it to do something in our command like this:

    #Call functions
    result = testpymodule.DoSomething("inputString")

    #Create classes
    myClassInst = testpymodule.MyClass()

At this point, the __getattr__ member function of the Import class is called. It checks to see if the module has already been loaded and loads it if it hasn’t. Either way, the class returns the attribute that is asked for, where it is either called as a function or used to create an instance of a class (as shown above).

This delayed loading mechanism is the reason that this system works and why it’s so robust. All other methods that I’ve tried suffer because the plugin isn’t loaded, which forces you to import modules inside function declarations (which feels a bit hacky)

To load the module, I’m using the imp module. It exposes the lower level commands inside the import mechanism. By calling imp.find_module() followed by imp.load_module(), it forces a reload of the module and flushes any cache stored for that module. Generally, this behaviour is desirable as XSI tends to be quite aggressive with it’s caching.

And in case you’re wondering. Yes. This method for importing works fine with the callback mechanism in the previous article.

How to pass functions to PPG logic in Python

When making user interfaces in XSI, I find that I often need to create custom properties on the fly, rather that fully self-installed custom property plugins. When doing this, I usually need to pass callback functions to the PPGLogic attribute so that XSI knows what to do when a button is clicked or a value changes. In JScript, this is simply done using the toString() method of the JScript function object, like this:

function MyJScriptCommand_Execute(  )
{
    prop = XSIFactory.CreateObject("CustomProperty")
    prop.AddParameter2("myValue",siInt4,0,0,100,0,100,
                            siClassifUnknown,
                            siPersistable | siKeyable);

    layout = prop.PPGLayout;
    layout.AddItem("myValue");
    layout.AddButton("myButton");

    layout.language="JScript"
    layout.logic=myValue_OnChanged.toString()+
                     myButton_OnClicked.toString();
    InspectObj(prop);
    return true;
}

function myValue_OnChanged()
{
    LogMessage("myValue was changed");
}

function myButton_OnClicked()
{
    LogMessage("myButton was clicked");
}

Despite the flexibility that Python offers as a language, it doesn’t have a mechanism able to do this. So we need to find another method.

Triple Quoted Strings

One common way to do it is to write your functions inside a triple quoted string. Here’s the Python equivalent of the above code:

def MyPythonCommand_Execute(  ):

    prop = XSIFactory.CreateObject("CustomProperty")
    prop.AddParameter2("myValue",constants.siInt4,0,0,100,0,100,
                            constants.siClassifUnknown,
                            constants.siPersistable | constants.siKeyable)

    layout = prop.PPGLayout
    layout.AddItem("myValue")
    layout.AddButton("myButton")

    layout.language="Python"
    layout.logic='''
def myValue_OnChanged():
    LogMessage("myValue was changed")

def myButton_OnClicked():
    LogMessage("myButton was clicked")
'''
    Application.InspectObj(prop)
    return true;

This is perfectly acceptable and works fine, but it does have a few drawbacks:

• It makes the code look ugly and less readable. Python is formatted in a specific way due to the indentation being linked to the syntactical interpretation. Putting code inside a triple quoted string goes somewhat against this ideal.
• Most text editors will not show the syntax highlighting for the callback code, and will highlight it as a string. This is a minor problem but can be annoying, particularly if you’re building a complex string with escaped characters.
• All the code inside the triple quotes is executed in a new scope. This means that any global constants, helper functions, or module imports will have to be repeated inside the triple quoted section. This can lead to a lot of code duplication, which is obviously a bad thing.

Using the Logic File attribute

It would be fair to say that the triple quoted string method isn’t particularly convenient. Are there any other ways that we can do this?

Yes. If you look at the PPGLayout attributes, you’ll see that we have the ability to specify a PPG logic file. This is a file in which we can put all our callback functions, and as long as we tell XSI where to find it, everything should work fine. For example:

def MyPythonCommand_Execute(  ):

    prop = XSIFactory.CreateObject("CustomProperty")
    prop.AddParameter2("myValue",constants.siInt4,0,0,100,0,100,
                            constants.siClassifUnknown,
                            constants.siPersistable | constants.siKeyable)

    layout = prop.PPGLayout
    layout.AddItem("myValue")
    layout.AddButton("myButton")

    layout.language="Python"
    layout.SetAttribute(constants.siUILogicFile,
                        "C:\\Workgroup\\Application\\Plugins\\PPGCallbacks.py")
    Application.InspectObj(prop)

…and in a separate file called PPGCallbacks.py

def myValue_OnChanged():
    LogMessage("myValue was changed")

def myButton_OnClicked():
    LogMessage("myButton was clicked")

This certainly gets around the problem of syntax highlighting, but it still means we have to redeclare global constants, helper functions, and reimport modules. It also gives us two extra problems of having to hardcode the location of the logic file, and having to make sure that the plugin file and the callback file are always installed together. If anything goes wrong with either of those, you’re not going to know about it until you open the PPG and press a button or change a value.

Okay, why not eliminate the extra file and just point it at the original script?

def MyPythonCommand_Execute(  ):

    prop = XSIFactory.CreateObject("CustomProperty")
    prop.AddParameter2("myValue",constants.siInt4,0,0,100,0,100,
                            constants.siClassifUnknown,
                            constants.siPersistable | constants.siKeyable)

    layout = prop.PPGLayout
    layout.AddItem("myValue")
    layout.AddButton("myButton")

    layout.language="Python"
    layout.SetAttribute(constants.siUILogicFile,
                 "C:\\Workgroup\\Application\\Plugins\\MyPythonPlugin.py")
    Application.InspectObj(prop)

def myValue_OnChanged():
    LogMessage("myValue was changed")

def myButton_OnClicked():
    LogMessage("myButton was clicked")

That’s much better. We don’t need to manage two files and we can put all our callback code in the same file as the PPG definition. It starts behaving a lot like a self-installed custom property, although we still have the problem of hard coding the path.

Using XSI’s plugin registry

The solution is to look up the path to the current plugin file using Application.Plugins.

Let’s look at the details of how this works.

Application.Plugins is a collection of XSI plugin objects. We can retrieve any particular one by asking for it by name. This is the one we declared it with in the registration function.

def XSILoadPlugin( in_reg ):
    in_reg.Author = "Andy"
    in_reg.Name = "My Python Plugin"
    in_reg.Major = 1
    in_reg.Minor = 0

    in_reg.RegisterCommand("MyPythonCommand","MyPythonCommand")
    #RegistrationInsertionPoint - do not remove this line

    return true

By using this name to index into the plugin collection, XSI will return us a Plugin object:

pluginObject = Application.Plugins("My Python Plugin")

Once we have the plugin object we can ask it for its filename:

LogMessage(Application.Plugins("My Python Plugin").FileName)

# INFO : C:\Workgroup\Application\Plugins\MyPythonPlugin.py

So to set the logic file to point to the current plugin, we can just do this:

layout.SetAttribute(constants.siUILogicFile,
                    Application.Plugins("My Python Plugin").FileName)

To obey the rule I mentioned earlier about not defining the same thing twice, we can define the plugin name as a global variable right at the top of the script. The final script looks like this:

import win32com.client
from win32com.client import constants

g_pluginName = "My Python Plugin"

def XSILoadPlugin( in_reg ):
    in_reg.Author = "Andy"
    in_reg.Name = g_pluginName
    in_reg.Major = 1
    in_reg.Minor = 0

    in_reg.RegisterCommand("MyPythonCommand","MyPythonCommand")
    #RegistrationInsertionPoint - do not remove this line

    return true

def MyPythonCommand_Execute(  ):

    prop = XSIFactory.CreateObject("CustomProperty")
    prop.AddParameter2("myValue",constants.siInt4,0,0,100,0,100,
                            constants.siClassifUnknown,
                            constants.siPersistable | constants.siKeyable)

    layout = prop.PPGLayout
    layout.AddItem("myValue")
    layout.AddButton("myButton")

    layout.language="Python"
    layout.SetAttribute(constants.siUILogicFile,
                        Application.Plugins(g_pluginName).FileName)
    Application.InspectObj(prop)

def myValue_OnChanged():
    LogMessage("myValue was changed")

def myButton_OnClicked():
    LogMessage("myButton was clicked")

That’s all there is to it. Now we have a workable strategy that has the following advantages:

• Tidy code
• Full syntax highlighting
• All the code is in a single file
• All the constant definitions, helper functions, and imports only need to be written once

Note that the callback code is still being executed in a different context. There’s no way around that. This means that changes to the values of global variables will not get passed through to the callback code. Global constants work fine because they are declared in global scope and don’t change.

In the second installment of this article, I’ll be showing you how to load python modules that are specific to your plugin using a similar system.

I had assumed that XSI embeds the definition of compounds used within exported compounds. By default, this isn’t the case, and I had forgotten that I had renamed the internal compound to something else.

So it turns out that there’s an innocent little option on the Export Compounds dialog called “Embed internal compounds”. It’s off by default, and I’d strongly recommend turning it on every time you export your compounds.

If you don’t tick that little box, XSI only stores the names of the nested compounds. That means that if any of the internal compounds get moved, renamed, or deleted, your main compound will no longer work.

What’s potentially worse, is that if you update one of the nested compounds with a slightly different functionality, then the behaviour of your main compound will change too. This is particularly bad because this would happen invisibly to you. No warning, no error messages, no nothing.

Aha, you say, but what if there’s a bug in that nested compound? Isn’t it desirable that fixing it will propagate to any other compounds that use it? Yep, sure that’s handy. However, if you’re working in a pipeline and store your compounds in an SVN, your compound has just changed behaviour and it has no associated version checked in to allow you to roll back if needed. It makes it impossible to track versions.

In short, if you don’t tick this innocuous little box, you’ll end up creating compounds with a spiders web of dependencies on each other. You could end up with a situation reminiscent of Windows DLL hell.

Believe it or not, it’s actually possible to create infinite recursion with circular references. If you make one, it’ll export okay, but when you drag it back in, XSI will hang as it tries to add nodes inside of themselves. Admittedly, this is a bit of a contrived situation, but it should serve as a warning that not embedding your compounds is a bad bad idea.

Both articles are written informally to assist with clarity and so aren’t mathematically precise in their notation. That comes later once understanding and intuition has been developed.

Introduction

Previously, we looked at how a matrix can be thought of as being made up of three axes. It was an easy way to help visualise what the numbers in the matrix represented. We’ll continue using that idea in this article to see how 3×3 matrices are able to rotate and scale points in space.

If you’re new to some of these concepts and find that you get lost halfway through, it’s often a good idea to try leaving it for a short period of time and revisiting later. You should find it a lot easier to understand on subsequent readings. If you still find that a particular section is hard to understand, please feel free to let me know in the comments, and I’ll either modify the article, or post an explanation in reply.

Defining a point

So starting with the basics; what do we mean by a point in space? Usually, we express a position using 3D cartesian coordinates, which has three components X, Y, and Z.

For example:

describes a point one unit along the X axis, two units along the Y, and three units along the Z axis.

For now, let’s just consider this point:

Why? Because it makes it much easier to observe what’s happening. The following animation shows how the point is reached by travelling along each axis.

You should notice that at any time during the video, the point position (shown in white) is equal to the sum of the yellow numbers that appear along it’s route. Just to clarify; by this I mean that all the X components of the yellow numbers along the X, Y, and Z axes are added to make the X component of the final point position, and similarly for the Y and Z components.

So looking at the video again, the first number of each yellow set of numbers are added to become the first number of the position, the second are added to form the second, and the third make up the third. Play it back and stop it when the point is halfway along any of the axes and this should be clearer to see.

This all means that the final position [ 1, 1, 1 ] is made up like this:

So we can see that in this case, a point position at [ 1, 1, 1 ] is just made from adding the end point positions of each axis.

Local Space versus World Space

The following clip shows this same point position [ 1, 1, 1 ] which has been transformed by a matrix that has a 30 degree rotation about the Z axis.

Again, we can see that at any stage during the animation, the point position is made up of the sum of the axes it has travelled along. In a way, it’s more obvious than the previous clip, since here the numbers are more identifiable as forming the sum of the final point position.

The final point position is given by:

Remember, these point positions being added together are just the positions of the end points of each axis in world space.

Just for show, lets look at what happens when we animate the rotation of the matrix. You can think of this as representing what happens when you interactively rotate an object in your 3D application. For clarity, the clip shows the animation first, and then overlays the resulting point position.

Again, pause the animation at any moment in time, and you’ll see that the final point position is made up of the sum of the yellow numbers along the axes. (Note: if it doesn’t add up exactly, it’ll be due to rounding issues of the display)

The next page of this article goes on to look at the more general case.

The XSI Tab Menu tool lets you accelerate your workflow by allowing easy access to run common commands and add nodes to your ICE tree and Render tree. It does this using a pop up window (accessed by hitting <Tab>) which temporarilly captures keyboard input to determine what to do. You’ll probably recognise that this is a similar functionality to that found in Houdini and Nuke.

Installation

The plugin is provided as an XSI Add-On file, and you can simply drag and drop the download link into XSI to install it. Alternatively, download the file, and then drag and drop the file into XSI, or use the File -> Add-On -> Install menu option.

XSI Tab Menu V1.3 (32Kb)

Versions

1.0: Initial release
1.1: Fixed issue with using Tab Menu in alternative layouts
1.2: Tab menu now always appears in new window (avoids problem with property pages embedded in the viewport). Fixed keyboard conflict with PickObject() command. Now allows closing of menu by hitting escape.
1.3: Fixed conflict with ‘B’ key Bounding Box Manipulator tool.

Getting Started

Once you’ve installed the add-on, pressing <Tab> should make the Tab Menu appear. Pressing <Tab> again will close it. If the window does not appear, try refreshing your plugins or restarting XSI.

Take a look at this video to give yourself some idea of how you can speed up your workflow. (It might help to go full screen so you can read the text on the Tab Menu a bit easier)

Give it a go yourself by following the instructions below.

Note that if you ever need to close the Tab Menu, just press <Tab> and the window will disappear. If you close the window with the mouse, then the menu will still be capturing the keyboard input, and you’ll need to press <Tab> again to resume using the normal XSI keyboard shortcuts.

Try typing the following:

<Tab> g <Enter>

That should have created a new Grid polymesh in your scene. Now type:

<Tab> e <Enter>

That will have just created a new point cloud and got it emitting particles from your grid. Try this:

<Tab> i <Enter>

You should see the ICE tree editor appear in the lower part of the XSI layout. (This is assuming you’re using the default 4-view layout of XSI). Check that the point cloud is still selected and refresh the ICE tree editor to view it’s tree.

This bit is important. Click inside the ICE tree editor. This tells the Tab Menu to change context to the ICE tree, and means that you are now in ICE node creation mode. To check this works, type:

<Tab> setpc <Enter>

That should have created a Set Particle Color node in your ICE tree.

Contexts

The XSI Tab Menu has 3 different contexts: Main, ICE Tree, and Render Tree. You can see which context you’re currently in by looking at the top of the Tab Menu window. This tells you if you’re going to be running commands in the main workspace, or if you are going to be adding nodes to your currently viewed ICE or Render tree.

Tab Menu layout (click to enlarge)

How does it work?

So you’re probably wondering how do you figure out what to type? It’s actually quite simple. The easiest way to understand it is to follow it step by step:

Note: If you make a mistake, you can either press <Backspace> to undo the last key press, or press <Tab> to cancel entirely and start again.

1. Click in the ICE tree editor (in case it’s lost focus), press <Tab> and then type the letter ‘g’. The Tab Menu window will offer the suggestion “Generate 3D Point Grid”. This is the first option available to you (alphabetically) based on the first letter.
   
   
2. Now type the letter ‘e’. The suggestion remains the same as it is still consistent with the letters you’ve typed so far. Note that at any stage you can press <Enter> to create the node indicated by the current suggestion, or you can hit <Tab> to cancel.
   
   
3. Type the letter ‘t’. Now the suggestion has changed to “Get Angle Between”.
   
   
4. Type the letter ‘p’. The suggestion reads “Get Particle Age”, although the text above it just reads “Get P”. This is because there are two possible letters that can go after ‘p’ according to the nodes available.
   
   
5. Now type ‘o’. The suggestion reads “Get Point ID” and so does the text above it. This means that there are no more options available. It doesn’t matter what you type, the suggestion will never change.
   
   
6. Press <Backspace> to take back the ‘o’ you just typed. The suggestion should have returned to “Get Particle Age’
   
   
7. Here’s where it gets interesting. Press ‘a’. You’ll notice that the upper text just jumped to read “Get Particle”. This means that the Tab Menu has auto-completed the rest of the word “Particle” for you as there are no other options within that word that are available. The next letter you type will dictate what comes after the word “Particle” so there is no need to type “rticle” to finish the word.
   
   
8. Pressing ‘c’ will cause another bit of auto-completion. This time it suggests “Get Particle Collision Location”, with the upper text reading “Get Particle Col”.
   
   
9. Pressing ‘o’ again will show the final suggestion of “Get Particle Color”. Pressing <Enter> will create this node. Alternatively you can press <Backspace> to go back and try some other key presses to see how they take you between choices.
   
   

The approach I take when using this tool is to watch the upper text when typing. You’ll notice when auto-completion occurs and then you can continue typing the rest of the command/node after. It doesn’t take long to get extremely fast and efficient at remembering the sequences, and you’ll make a few jaws drop around you if you can get good at it.

Limitations

There are three fairly trivial limitiations you need to know about. Firstly, the context sensitivity for detecting if we’re in the ICE tree editor, the render tree, or the main viewport is reliant on embedded windows (i.e. in one of the A, B, C, or D viewports). In other words, if you press ‘7′ to create a floating render tree window, the Tab Menu will be unable to figure out that context.

Secondly, the <Tab> key by default is used by XSI to “Show Next Manipulator”. This usually means that pressing <Tab>will cause XSI to cycle between translation, rotation and scale manipulators. The Tab Menu overrides this functionality, so if you use it a lot, you’ll need to remap it to another key.

The final limitation is to do with the fact that the Tab Menu relies on three configuration files to specify the commands and nodes available. This means that if you create and register your own ICE tree or render tree compounds, then they won’t be available as a Tab Menu shortcut until you manually add them to the configuration files. The reason it was done like this was to give minimal delay to you when loading the Tab Menu plugin. If it had to scan (as XSI has to) then there would potentially be a few seconds extra delay when starting up or reloading plugins.

Adding your own command shortcuts

The good news is that it’s very easy to add your own commands to this system. There are three text files located in the same directory as the plugin called:

Main_TabDefinitions.txt
ICE_TabDefinitions.txt
Render_TabDefinitions.txt

From the names, you can see that each text file describes each different context that the Tab Menu supports. If you open the “Main_TabDefinitions.txt” file, you’ll see that each command is contained on it’s own line and is of the form:


<command shortcut>,<python script line to execute>
 


So if you look at the first line, you’ll see the shortcut used to create a cone:


Cone,Application.CreatePrim("Cone", "MeshSurface", "", "")
 


Adding your own ICE and Render tree node shortcuts

For defining shortcuts for the ICE and Render tree, the command that adds nodes needs to be told which tree to add them to. This is done using the symbol %TREE% in the text. This is automatically replaced at runtime with the correct tree that is being currently viewed.

So for example, the shortcut to add a Phong node is as follows (without the line break):


Phong, Application.CreateShaderFromPreset("$XSI_DSPRESETS\\Shaders\\Material\\
Phong.Preset", "%TREE%", "")
 


It is a summary of an explanation I’ve frequently used to develop intuition surrounding matrices. I’ll be assuming very little prior knowledge apart from that you are familiar with 3D applications (it’s not strictly necessary, but the analogies will make more sense). By the end, I’m hoping that you can look at a matrix and have a fair idea what it represents. It’s not an approach I’ve often seen, but try following it and see if it makes sense to you.

Introduction

A 3×3 matrix is used by a 3D application to describe the orientation and a scale of an object in space. (Note that if you want to include the object’s position, then you need a 4×4 matrix, but we don’t want to go there yet.)

It is made up of 9 numbers arranged like this:

This particular 3×3 matrix is known as the “identity” matrix. We have zero rotation and a scale of 1. It doesn’t do anything to a point in space, no rotation or scale, so it’s a good place to start.

Okay, so this is the important bit. Instead of looking at this matrix as a bunch of numbers, let’s think about it as representing the axis icon in the corner of your XSI/Houdini/Maya viewport.

The first line of the matrix represents the X axis, the second line the Y axis, and the final line the Z axis. This makes more sense when we think of the end of each line of the matrix being a point in space. This means that each axis looks like this.

So the first line is a point in space that is 1 unit along the X axis, the second line is a point at 1 unit along the Y axis, and similarly for the final line 1 unit along Z.

So now, if we want to figure out what a particular matrix looks like, just imagine doing the transform to the axis icon and see what happens.

Scaling

What would a 2x uniform scale do to the icon? It’d move each point to double it’s original position along each axis. Okay, well that’s easy to write:

The video clip below shows what happens to the axis icon when we scale it uniformly by a factor of 2. I’ve annotated the end of each axis with it’s position. In the top left you can see the three point positions in order, shown to appear as a 3×3 matrix.

Rotation

Starting with the identity matrix again, what would a 90 degree rotation around the Y axis give us? Well, that depends which way we rotate it. Looking down on the Y axis, an anti-clockwise rotation would move the X axis to point along the negative Z axis, and the Z axis to point along the positive X axis.

So for each point, just visualise where it ends up and write down its position.

Yep, it’s that simple. Check out the short clip below to see it in action.

If you happen to know a bit of scripting, you should be able to obtain the matrix of an object in your 3D scene and check for yourself that this really works.

Reading the Matrix

So why don’t we try to take a matrix and work backwards to see if we can figure out what it represents.

Compare the first line to the first line of the identity matrix at the top of this article. It’s exactly the same, so we know the X axis is unchanged.

Looking at the second row, the Y axis is reversed and now pointing down. The third row shows that the Z axis is reversed too. How would you do this in your 3D software if you were going to use the transform tools?

Actually, there are two ways. You could either negatively scale the Y and Z axes with the scale tool, as shown in the following clip:

Or you could rotate it around the X axis by 180 degrees, shown below:

So are we able to say which one? No, the matrix only tells us about the destination, not the journey.

Using What You Know

Here’s just one quick example of how you can use this knowledge:

Q. Imagine we have a scene with a boat bobbing around on a rough sea. How do we find out how upright this boat is? Another way of thinking about this is how aligned is the boat’s y-axis is with the world’s y-axis?

A. If we have the boat’s 3×3 matrix, we know from the discussion above which row represents the y-axis of the boat (the middle row). With a bit of thought, we can figure out that if the boat is fully upright, this middle row of the matrix should read [ 0 1 0 ]. Taking this further, we know that all we care about is the height of this point (who cares if it’s offset to the side in x or z right?), so really we’re only interested in the y coordinate of this point, i.e. the middle value.

This value gives us the perfect measure of how upright the boat is. A value of 1 indicates it is perfectly upright, and -1 means it is completely upside-down.

Going Further

Now that you’re hopefully familiar with the basic anatomy of a 3×3 matrix, the next step is to have a look at how it can be used to transform geometry. This isn’t a big leap; you’ve already seen visually how it works for the three points that lie on the axes.

After that, you’ll be more than ready to look at 4×4 matrices which incorporate translation as well, but this is all material for future articles.

For the Sky Sports “Super Sunday” ident we worked on this year (Mill article), almost every shot had some amount of speed ramping. Needless to say, this is usually where the FX TD has a minor heart attack in anticipation of the hell that he or she is about to go through. However, the flexibility of ICE in XSI now makes the process relatively easy, and this three page article will show you how to do it.

Why is speed ramping a problem?

Physics engines in 3D applications tend to do their calculations on a per frame basis and they assume that the length of time that passes in each frame (known as the simulation step in ICE) isn’t changing. It’s an important optimisation as it simplifies the problem for them.

Numerical accuracy for collisions, and forces such as damping, are highly dependent on the simulation step. If the simulation step was allowed to change on each frame, it would introduce erratic behaviour into your object or particle motion.

Another problem you’d notice is that for different speed ramps, you would get different results from the simulation. Small inaccuracies in the way the simulation is calculated give rise to larger effects later on, despite starting with the exact same initial conditions. The key point to note is that larger simulation steps are more inaccurate than smaller ones, and changing the speed ramp means that you’d be introducing those inaccuracies in different places. This isn’t much good as you might have to redo your simulation if your speed ramp ever changes (which on our Sky job, it did on a number of occasions).

Methods for dealing with speed ramping

There are three ways to deal with speed ramping at the 3D stage:

1. You add support for speed ramping into your physics engine. Actually, for particle simulations this isn’t as hard as it sounds. You just have to create your own “Simulate Particles” node. For rigid bodies… well… good luck! There’s a lot of maths involved, and it’s far from easy. Plus you’ll have to write your own collision system. (Yeah… no thanks!)
   • Pros:
     If you can get it to work, it’s a simple way of achieving the right result. You just run your simulation and you get the result.
   • Cons:
     For anything other than particle simulations in ICE, it’s not particularly viable unless you’re good at physics programming. It would introduce inaccuracies. If your speed ramp needs to change later, you’ll have to re-simulate, and there’s no guarantee you’ll get the same results.
2. You simulate and cache the simulation at the same frame rate as you’re delivering the final sequence (in our case, 25 fps). At render time, if you need to go to slow motion, you’ll have to stretch out the simulation to slow it down. This means you’ll be trying to render at non-integer frame numbers and you’ll have to interpolate the particle positions and other attributes.
   • Pros:
     It’ll be fast and you won’t have much data to cache. You’re doing the minimum amount of work needed to get the result. If the speed ramp has to change, you don’t need to re-simulate since you’ve cached out all your data.
   • Cons:
     It’s unlikely that this will look very nice if you have any collisions, as any sharp changes in velocity will look sluggish due to the interpolation. You might get away with it though, so it’s worth a try if you are up against a tight deadline.
3. You simulate and cache the simulation at the highest frame rate that you think you’ll need (in our case that was 100 fps). At render time, you’ll usually be replaying the cached simulation faster (e.g. when it’s running at 25 fps) by missing out the frames you don’t need. To guarantee smooth animation, you still need to interpolate between frames, as more often than not the dynamically changing frame rate will result in non-integer frame numbers.
   • Pros:
     Calculating at the highest frame rate means that your simulation will be super accurate. As with (2) if the speed ramp has to change, you don’t need to re-simulate since you’ve cached out all your data.
   • Cons:
     You’ll be spending a lot of time simulating and have a lot of data to cache.

What we used on Sky

The third option is nearly always the best way to go. It’s a good solution, providing you’ve got plenty of disc space, and have enough time to do the simulations.

For Sky, we animated and simulated everything at 100 fps and cached all that data to disk. (If an analogy helps, just think of it as filming live action with a high speed camera).

For each shot, we had a curve that represented the current frame to load from the disk cache. This curve was an incredibly important asset in our pipeline as it was used to synchronise multiple packages (XSI, Houdini, Maya, Massive, Cinema 4D) to render everything at precisely the same moment in time.

Note that since the speed ramps gave smooth changes in frame rate, the frame numbers given by the retime curve weren’t usually integral values (e.g. 1, 2, 3, 4, etc.), but were fractional (1.4, 2.8, 4.2, 5.6, etc.). This meant that camera and object transforms, meshes, and particle attributes had to be interpolated to be properly evaluated at these fractional frame values. More on that later.

There’s one last issue you need to take into account when doing any sort of retiming of a cache. You can read about that on the next page.

The brief was pretty simple. We had to create a lightning storm over Frankfurt and show a direct strike on a bridge over the river. We were told to “Just make it look awesome!”. Sure, we can do that!

I’ll write a post soon on how we made it. But for now, here’s the finished piece, as published on YouTube:

So it turns out that one of the best things about doing a viral is that you get to see some feedback from the public in the comments they leave on YouTube. Most people spotted it as a fake, and I really don’t mind as they were quite complimentary on the whole. If we’d kept it totally within the bounds of reality the piece wouldn’t have been so effective. The point of the viral was to make sure that it got everyone’s attention.

Other people totally bought it.

“Your all f******s. Why would someone fake something like this? What’s the point? Wow I faked a lightning storm for 100,000 views? This is most likely real. But the dudes who shot this are idiots for being in the water during s lightning storm”

One guy thought it was fake and started making assertions such as:

“actually, if you look closely, you can see they are not in the water – they are driving on a road at the riverside!”.

Nope, we were definitely in a boat! I think it’s amazing what people convince themselves they’re seeing.

This made me laugh too:

“Oh my god, is this REAL?

And can Audi protect me from lightning?

PLEASE HELP!”

Finally, this one’s probably my favourite (which only makes sense if you’ve seen Total Recall):

“haha, is the governator on that boat? are they on their way to ze mines to turn on the reactor….damit cohagen you know the reactor makes air, why don’t you just turn it on? Quaid….Quaid….’start the reactor’ “

Overall the campaign was a success. At the time of writing this, the spot has had well over 288,000 hits, the vast majority of which happened in the first week of it being posted. There was a lot of press coverage too, particularly in the automotive press as you’d expect. It was all to publicise Audi’s new e-tron concept car. The e-tron has one battery, four electric motors, and an insane amount of torque which means it can go from standing to 62 mph in 4.8 seconds.

Here’s the link to Audi’s viral campaign website www.electricityuntamed.com where you can see a video of the launch, or you can go here to view a short article and Audi’s official press release about the e-tron.

You can read The Mill’s article on the job here.

In addition to “Storm Over Frankfurt”, two other virals were shot. Minimal post production was needed by The Mill except for some rig removal work. They’re great spots and very funny, so I’ve embedded them below for your convenience. Enjoy!

Back to School

There’s no doubt that leaving my job at Lionhead Studios to go to film school for a year was a risky move for me to take. It was a tactical decision to try and rebrand my career towards film and commercials work; something that I’d been trying to do for a while.

One of the other reasons that I decided to sign up for the VFX/SFX course run by the NFTS was because of the pyrotechnics module that they offered. Having grown up watching films like Raiders of the Lost Ark and being utterly consumed with fascination about the stunts and effects, I felt that this was my opportunity to satisfy that craving.

I wasn’t disappointed. Not even close.

The Setup

There were six of us on the course: Kate, Kia, Mattias, Nick, Seru, and myself. We were all from a wide range of backgrounds which made for an great mix of skills and experience. The first month on the course was spent learning about fire, explosives, rain, snow, firearms, and many other areas that an SFX technician has to have a good understanding of. The car explosion stunt was the climax to the course module.

I’m not sure where the idea came from, but I think I suggested that it would be a cool idea to blow up the car with a rocket launcher. I figured that blowing up a car without a good explanation would be a bit dull. Far better to make it look like we were hitting it with an RPG. Michael Bay eat your heart out!

One thing we all learnt in that first month as trainee SFX technicians: in a low budget scenario, you literally need to beg, steal, or borrow whatever you need to get the job done. Resourcefulness can make the difference between getting the shot, or not.

The begged items came from Graham Brown in the form of fishing wire and some fireworks. Although to be honest, I didn’t need to do too much begging. Graham as usual was more than happy to help out.

In true Blue Peter style, the rocket launcher was made from a cardboard tube; this time, stolen from our communal TV room at the film school (it was in a pile of rubbish and no one was using it). I painted it green, and fixed a wall bracket to it as a makeshift sight. A bit of yellow and black tape finished it off.

To make handbrake turns easier, I rigged the steering wheel with a cylindrical door handle borrowed from a door found on the NFTS main stage. I should point out that it wasn’t from a real door, just one that was lying around previously used as part of a set on an earlier production. Before we blew the car up, I got it back and reattached it to the door the next day.

One thing I didn’t mention in the guerrilla SFX “beg, steal, or borrow” toolkit was bribing. The NFTS course managers persuaded Arri to come along with some of their high speed cameras. Not through any monetary bribing (this was low budget, don’t forget), but as a promise of some great stock footage for any of their product demos. In the end, they brought two cameras which, despite the poor light conditions, were run at 200 fps and 400 fps.

The Car

We didn’t have any sort of story line or plot as we were more interested in the SFX, but we still wanted to shoot some footage with the car. If we’d had two cars it would have been better, but again, our budget didn’t stretch that far. So we kept it simple and just used the deserted market stalls as a background for some interesting shots.

You might be surprised how tricky it is to do a handbrake turn in a Ford Escort, even in the wet. It took a bit of practice but I got there in the end. The door handle on the steering wheel helped, but I found the key was to initiate the turn, slam on the brakes to throw the weight forward, and then yank on the handbrake to throw the rear end round. The one that you see in the video nearly went 180º.

After we’d done the other shots with the car, we did a final run ending up with the handbrake turn. With the car in the final position, we set about stripping it down to remove any dangerous or unnecessary parts. High on this list was the petrol tank which was drained and removed. We were also told that the pistons for the hatchback boot can be highly dangerous in a fire and can explode violently. Apparently the fire services are quite nervous about these things going off in motorway fires!

Other things that were done: the car battery was removed, tyres were let down, brake fluids and oil were drained, and many other parts were removed. All this was to make sure that we didn’t have any unpredictable results that could be a safety hazard, and also because we didn’t want the car to burn for hours after the explosion.

In case you’re wondering, the car had been bought for about £300 specifically for us to blow it up. Coincidentally, it bore more than a striking resemblance to my own car at the time so I swapped the wheel hubcaps as they were better than mine!

Rigging the Explosives

After the car had been stripped down, we set up the charges. Unfortunately, there’s a limited amount of detail that I can go into without being a bit irresponsible by telling everyone how to blow stuff up, but suffice to say, we used several standard petrol based mortars along with a couple of charges to blow the windscreen.

All the charges had to be wired up with long lengths of electrical wire so that we could be at least 50m away when it went off. The charges were connected to a custom made detonation box belonging to Graham.

The rocket launcher (err… cardboard tube) was connected to the front of the car by the fishing wire. We sellotaped a firework rocket to a drinking straw that had the fishing wire threaded through it.

During the set up, we were all aware that it was a race against time. Being November we had a limited amount of daylight, and it was made worse by thick cloud cover and drizzling rain. Trying to film something as bright as a fiery explosion isn’t something you want to do in complete darkness, especially with high speed cameras. You’d see either the explosion or the surroundings, not both; at least not in any detail.

Bang!!

So when the time came to set everything off, we were knackered. Making sure that all the cameras were up to speed, we gave a countdown and lit the firework. It shot along the fishing line, hit the car, which blew up in a series of explosions, one of which sent the windscreen 30 feet into the air. Everything worked perfectly and at the right time; something that we were all surprised by considering the pressure we had been under and the weather conditions.

We had spent a lot of time carefully laying out and labelling the numerous cables that were used to set off the explosions. Since we had limited resources, many of the cables were made up of two smaller ones, so we had to make sure that all connections were working. I think we checked everything at least twice. Graham’s firing box had a special mode for testing connections by sending a low powered current down the cable and through the charge. It wouldn’t be enough to detonate the explosive, but enough to be able to check the continuity of the circuit. We would still treat it as a live firing though… just in case.

Safety First

There are lots of precautions one takes when working with explosives. While I said that I wouldn’t go into any details on how to make something go bang, I think it’s safe enough to discuss some of the safety measures we took.

You have to be very aware of what everyone is doing, especially when you are connecting the charges to the lines. For example, it’s all too easy to leave the battery connected to the firing box with the key in the on position. While that wouldn’t detonate the charge (you’d still need to press the right button), it’s not a situation you want to be in. The standard Health and Safety strategy is that it’s everyone’s responsibility to look out for each other, and to make sure that these slip ups don’t happen. It’s all about minimising risk.

Generally you don’t want to be wearing anything that has a tendency to create static electricity. It is possible (although unlikely) that static can generate enough power to set off a charge. Synthetic materials are well known for being good sources, so they should be avoided.

This one’s important (and you’ll recognise this if you’ve ever bought fireworks). Never return to a charge if it doesn’t go off. There’s a procedure to follow in this situation. Generally the safest thing to do with a dud is to place another charge next to it, and detonate one using the other. We were shown a video of what can happen if you try to take shortcuts. I believe the poor chap lost hearing in one ear and part of a finger.

The best way to make sure a charge is safe is to twist the two connections together. It makes sure that there’s no potential difference between the wires so it’s impossible for the charge to be set off by a voltage. Most pyro charges are supplied in this configuration. Of course, there’s no guarantee that you won’t set them off with excessive or prolonged exposure to heat or some type of chemical agent, so you still need to take care.

Other Considerations

Whenever you do any sort of pyrotechnics work, it’s nearly always a good idea to let the local emergency services know in advance. With an explosion as large as the one we did, it would be pretty likely that someone would call the fire brigade – which they did. About ten minutes after we ignited our Ford Escort bonfire the fire services turned up, blue lights flashing.

If we hadn’t spoken to them first, they would have charged us a call out fee. They were a great bunch of guys though, and were probably happy for the break in their routine. The NFTS also did a leaflet drop in the area a few days before so that local residents wouldn’t be alarmed.

Graham Brown was our pyrotechnics tutor and supervisor. He worked for many years at the BBC providing special effects work for a huge number of different shows. When he’s not tutoring at the school, he still does work on a freelance basis for shows like Waking The Dead (which I was lucky enough to assist him with on three occasions – more in a future post). I’d like to take this opportunity to thank him for all his enthusiasm, dedication, and hard work, and for such an amazing experience that I’ll never forget.

Thanks should also go to John Rowe and Sarah Hayward, the course directors, who worked tirelessly to make sure our time at the school was chock filled with fun things like this to do. And finally a big shout out to the other guys on the course – those were good times eh!

Rear, left to right:
John Seru, Kate Walshe, Mattias Engstrom,
Nick Preston, Kia Coates, Andy Nicholas.

Front:
Graham Brown

(And then we had to clear up the mess)

The Video

Here’s the final cut with a wee bit of compositing to make it look like the car was still moving when we blew it up.

The Add SOP on the left uses a multiparm to get a list of point positions from the user, while the Object Merge SOP shown on the right uses a multiparm to merge geometry from multiple sources at once.

This simple tutorial will show you how to use multiparms in expressions and your own tools.

Introduction

Multiparms are Houdini’s way of getting a list of items from the user. They comprise of a single parameter that stores the number of entries in the list, and a list of parameters numbered sequentially that make up those entries.

Something which is particularly useful is that each entry doesn’t have to be made up of just one parameter. Houdini lets you store multiple parameters in each entry. In effect, you can think of each one being like a record in a database. Each one can contain a number of fields in which you can store information.

In this tutorial, I’ll only be looking at a multiparm that contains single parameter entries.

Initial Setup

To start, make a Geometry object, call it “geo_boxes”, and press Enter to jump inside.

Inside, create a Box SOP, and a Copy Sop with their default parameters, and connect them as shown.

Sticking to the Houdini convention that most people use; add a Null Sop called “OUT” and set its display flag.

On the next page, we’ll create the multiparm.