Developing custom Qt UI widgets for Maya 2011: OMFG it works!

Well after much stuffing around I finally got this working.  It was one of those problems that had several major pieces that all need to fit.

These are the major things I had to work out:

1.        You must compile your widget plugin with the same Qt SDK version as Maya.  Maya 2011 pre-release uses 4.5.3. 

2.       Your plugin DLL and therefore the base Qt libraries must be compiled with the same compiler that was used to create Maya.  On windows this is MS Visual Studio 2008.  I don’t know about other platforms sorry.

3.       It’s best if you put your plugin DLL in the
C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\

Now basically what we need to do is compile Qt from scratch using nmake which is the Visual Studio compiler.  We are also setting up Visual Studio as our main development environment so your final plugins will also be compatible.  Different compilers actually make DLLs in different ways so a DLL compiled with minGW won’t work with the Maya binary because it was compiled with Visual Studio.    And since we are using VS to compile our plugin the base Qt libraries also need to be compiled using VS.  This is a once off this and I’ve found that VS is a much better IDE than Qt creator.  You can even still use Qt designer to create your UI!

 

Here’s what you’ll need to get this working on Windows:

·         Microsoft Visual Studio 2008 (Best to have the latest service packs etc)

·         Qt SDK version 4.5.3 source package.  You won’t find any links to this on the Qt site.  You can get it here:   http://get.qt.nokia.com/qt/source/qt-all-opensource-src-4.5.3.zip.  FYI if you want to browse the available source packages navigate to http://get.qt.nokia.com/qt/source

·         Qt Visual Studio Add-in.

·         About a day of time for Qt to compile.

And these are the exact steps I did to get Qt 4.5.3 compiled:

1.       Go to this blog post then read my next points BEFORE following the instructions: http://dcsoft.wordpress.com/2010/01/30/how-to-setup-qt-4-5-visual-studio-integration/  Huge props go out to DCSoft for writing this up.

2.       Skip the DOWNLOAD QT SOURCE CODE section. Just download the source package from the link above and unzip it here: c:\Qt\ 4.5.3-vc.  Ie after unzipping the files you should see a structure that has this path: C:\Qt\4.5.3-vc\src.

3.       In SET ENVIRONMENT VARIABLES step 1 you need to change the path: QTDIR = c:\qt\4.5.3-vc

4.       In BUILD VC++ VERSION OF QT substitute our Qt version everywhere ie replace 4.6.1-vc with 4.5.3-vc

5.       Wait about 8 hours or so for Qt to compile.

6.       In INSTALL VISUAL STUDIO ADD-IN step 3 replace 4.6.1-vc with 4.5.3-vc

Now we should have a fully functional version of Qt 4.5.3 complete with Creator and Designer binaries and VS project files for all the source examples.       

Here’s what I did to get an example widget plugin compiled and working in Maya 2011:

1.       Boot up Visual Studio and load this project:  C:\Qt\4.5.3-vc\examples\designer\worldtimeclockplugin\ worldtimeclockplugin.vcproj

2.       Change the configuration to Release and compile.  The compiled DLL will be here: C:\Qt\4.5.3-vc\plugins\designer\worldtimeclockplugin.dll

3.       Copy the DLL into the Maya folder:  C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\

4.       Run this Python code:

And with a bit of luck you should see this: 

Awesomely the connections between the widget elements still work which means you can define quite a bit of functionality in QT Designer

Next up is investigating what needs to happen to compile a 64bit version.

Qt Fun and Games

I love being a noob again, all the fun and exitement of trolling forums for hours and ripping large clumps of hair out.

I have been doing my usual thing of diving head first into a project and over the last couple of day been trying to create a basic custom OpenGL widget for Qt on windows 7.  Now remember I think these are all windows related bugs.

The first peice of Qt awesomeness I found is that when you compile any of the test custom widgets you can't actually use them in the designer.  Why? Because apparently Qt Creator and Designer have been compiled with MSCV 2005, and by default Qt uses mingGW as it's compiler, making the plugin binaries incompatible with the main app.  So to fix this you either need to download the Creator/Designer src and compile it yourself with minGW (which looks to be a Shit Onion all of it's own).  Or create your own build chain using MS nMake for which there is little information on and no help.

So I thought "Who needs the designer, I can just code all the UI for now" *vomit*. So I started trying to load my plugin into another test app, then awesomeness part two came along.

Qt seems to have a way of interrogating DLLs similar to .Net reflection, but for this to happen you need to invoke the Meta-Object-Compiler (moc).  This generates moc src files that are used for linkage.  And of course my project wasn't generating them.  So after some trolling I found this post which pointed to the path to moc.exe being wrong in the make files.  That's a Bingo!  I changed the forward slashes to back slashes in my make files and it worked fine.

It really seems Windows was kinda forgotton by the Qt developers...

Developing custom Qt UI widgets for Maya 2011: A first look

This has been solved!!!

Check out my newer post here


Now that I’ve been using the new look Maya I kinda like it. The Maya UI always seemed cold and impersonal compared to Max which now for me is like a third brain lobe. I’ve also been looking for an excuse to learn Qt and Maya 2011 fits the bill.

For those who don’t know Qt is a cross platform system mainly used for the front end of applications but has other libraries such as database and web access. When Nokia bought Qt it sent good vibes out into the industry. Basically everyone now knows a large corporation is investing in this thing so it must be good. It’s also good to know someone is there to support the community and drive innovation in Qt. The CG industries have especially taken liking to Qt because it’s UIs look and perform very well, have GL integration and are consistent between platforms. These are things that help sell a product to end user artists.

Here’s a link to some hype, but this time you can probably believe it:

http://qt.nokia.com/qt-in-use/qt-in-visual-effects

And now Qt’s use in Maya has sealed the deal as an early industry standard.

Now for the disclaimer: I’m currently learning Qt and relearning Maya development so please take anything I say here with a pinch of salt.

Here are some early finding:

You need to be developing with the same Qt version as the app you are developing plugins for (I’m talking Qt system plugins here not Maya SDK plugins).

So I found a MEL command that returns the Qt version used by Maya:




Now as of time of writing the latest Qt version is 4.6.1. The latest release of the full Qt SDK package using 4.5.3 framework is 2009.04 , which can be found here (Windows):

http://get.qt.nokia.com/qtsdk/qt-sdk-win-opensource-2009.04.exe

To be clear, a UI element in Qt is called a Widget.

Qt has it’s own nifty plugin system for all sorts of stuff which has standard plugin folder names. From trawling the Qt help and Maya 2011 folders I found these:

From the Qt help

Locating Plugins

Qt applications automatically know which plugins are available, because plugins are stored in the standard plugin subdirectories. Because of this applications don't require any code to find and load plugins, since Qt handles them automatically.

And it just so happens that we have such a folder here:

C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins

Now Qt plugins need to be in strictly named subfolders.

From the Qt help

Base Class	Directory Name	Key Case Sensitivity
QAccessibleBridgePlugin	accessiblebridge	Case Sensitive
QAccessiblePlugin	accessible	Case Sensitive
QDecorationPlugin	decorations	Case Insensitive
QFontEnginePlugin	fontengines	Case Insensitive
QIconEnginePlugin	iconengines	Case Insensitive
QImageIOPlugin	imageformats	Case Sensitive
QInputContextPlugin	inputmethods	Case Sensitive
QKbdDriverPlugin	kbddrivers	Case Insensitive
QMouseDriverPlugin	mousedrivers	Case Insensitive
QScreenDriverPlugin	gfxdrivers	Case Insensitive
QScriptExtensionPlugin	script	Case Sensitive
QSqlDriverPlugin	sqldrivers	Case Sensitive
QStylePlugin	styles	Case Insensitive
QTextCodecPlugin	codecs	Case Sensitive

But there is no mention of Widgets here you might ask, so did I. So from reading some help and looking in the Qt install folder I found that you need to put Widget plugins in a designer folder. Ie

C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\myWidgetPluginProbablyGoesHere.DLL

If you’re looking for examples of custom Widgets look in the Qt Designer or just Designer sections. I’m using the World Time Clock Plugin example till I make my own widget.

Now I booted by Qt Creator 2009.4 using framework 4.5.3 and compiled the World Clock project which produces a file called worldtimeclockplugin.dll. So I copied it here:

C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\worldtimeclockplugin.dll and then loaded the UI file in the same project using Python:




This code created a window with the other standard widgets but the clock widget was not there. One of my main asumtions about this was that Maya would actually load my DLL so I booted up Process Monitor and checked what was going on when I ran the Python code. Sure enough Maya was accessing my DLL where I put it but it was also looking for it here:

C:\Program Files (x86)\Autodesk\Maya2011\bin\designer

I can remember reading somewhere in the Qt help that it will look for the standard plugin folders in the execution path so fair enough, I put the DLL there as well.

Still no widget. Maya is not generating any errors and according to Process Monitor all the DLLs it’s looking for are being found.

And that’s where I’m up to. Not much to go on except stuffing around with versions etc and looking at the Qt help. Hopefully I’ll have some actual results soon.

Photoshop and Max Example Tool Videos

Howdy,

Here's a couple of videos I've made recently to demo some Photoshop and 3DS Max tools.  The videos don't have sound but I added comment bubbles thoughout.

Thought I'd make this post for two reasons:

• Show off some of my work
• Learn how to embed video into Blogger

All videos and tools are copyright Krome Studios 2009-2010.

Photoshop Exporter and Exporter Panel

play video

3D Studio Max Selection Set Manager and Export Manager

play video




3D Studio Max Scene Sync Manager

play video

3D Studio Max Convex Volumes

play video

ExtendScript (JavaScript) in Photoshop Part 1: Work Flow Setup

Photoshop's primary method for creating new tools is through ExtendScript. ExtendScripts can be implemented in three different languages: JavaScript, VBScript and AppleScript. JavaScript is by far the most popular and the only cross platform option. This article will explain all the elements needed to start developing scripts for Photoshop as well as some more advanced topics such as loading external libraries. I will also be concentrating for now on the Windows platform when discussing external libraries and file paths and assuming Photoshop CS4 or above.

What can you do with scripts in Photoshop:

• Write tools without UI that can be executed from the File >> Script menu or from actions.
• Write tools with modal UI that can be executed from the File >> Script menu or from actions.
• Assign scripts to execute when events are triggered within Photoshop.
• Access the local file system.
• Load external libraries for extra functionality.
• Access XMP data.
• Control almost all parts of Photoshop.

What can't you do with scripts in Photoshop:

• Scripts cannot create non modal windows, ie tool palettes.
• Modify the standard Photoshop menus or tool panels.
• Give Photoshop the ability to handle new image file types.
• Create Filters.

I don't like the word “can't” so I will be listing ways to achieve the above points with other technologies later.

Setting Up Your Work Flow

First thing you need to do is make sure you have the following apps and documentation installed and handy:

1. Adobe ID. If you haven't already go create yourself an Adobe ID, will make life easier and you will need it to post on the Adobe support forums. If you are into privacy or in a hurry you can use BugMeNot to login.
2. Photoshop CS4 or above.
3. ExtendScript Toolkit. This is the IDE you will be working in.  
4. BareTail. This is a real-time log viewer with basic keyword highlighting. 
5. Photoshop CS4 Scripting Guide. General Scripting info. Is installed with Photoshop or get it HERE.
6. Photoshop CS4 JavaScript Reference. JavaScript specific info plus Photoshop DOM reference. Is installed with Photoshop or get it HERE.
7. Javascript Tools Guide CS4. I don't know why Adobe hides this doc away. It contains information about XML, XMP, file system access and accessing external libraries from JavaScript.

Here are some locations of interest and places to get help:

Official Adobe Photoshop Scripting Forum – The best resource for information and help.

PS Scripts – Go straight to the forum. Another great resource with plenty of code snippets hidden away in posts.

Scripting Photoshop from C# - For those of you adventurous enough to try some COM action.

Dr WooHoo! – A great resource for anything on the cutting edge of Adobe development with plenty of tutorials, especially if you are doing anything Flash related.  Make sure you check out his blog.

W3 School JavaScript Tutorials – A good resource for learning the JavaScript language.

Locally you want to check out these folders (assuming standard install):

Scripts Folder

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Scripting

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Scripting

Here you will find some of the above mentioned PDFs in the documentation folder and also sample scripts.

Presets Scripts

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Presets\Scripts

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Presets\Scripts

Here you will find the scripts available in the File >> Scripts menu, also the scripts available by default to assign to events.

Script Listener

The Script Listener is a plugin which logs all actions and events running through Photoshop to a text file on your desktop. Why do we need this? Well not all Photoshop commands are natively scriptable through the JavaScript DOM. That means there aren't specific commands for all actions in Photoshop. However 99% of Photoshop's actions are logged by the Script Logger which can be copied and pasted into a script. This is also useful when scripting the functionality of 3^rd party plugins.

To install the Script Listener close Photoshop and copy this file:

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Scripts\Utilities\ScriptListener.8li

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Scripts\Utilities\ScriptListener.8li

to here:

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Plug-ins\

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Plug-ins\

If you want you can make a ScriptListener Folder in the Plug-ins folder and put it there.

To check it is working correctly open Photoshop and create a new image. Go to your desktop and you should find a file called ScriptingListenerJS.log which is a standard text file.

BareTail is a real-time log viewer which means it automatically displays any new log entries. It also has some simple keyword highlighting to make it easier to view the log. You can download a BareTail prefs file here to get you started. Once BareTail is installed choose Preferences >> Load From File....

The only thing BareTail doesn't do is clear the log. If you want you can create a batch file on your desktop which does this and add a shortcut to it:

ClearPhotoshopLogs.bat

And now you are ready to start scripting any solution in Photoshop. Stay tuned for part 2 of this adventure when we explore some scripting concepts that are not covered in most tutorials.

Expanding Photoshop's Horizons: Plugins, Scripts and Flash

Working in any large scale creative team means big work flow problems at one time or another. Solving these problems often means creating custom tools for key pieces of software such as Maya, 3D Studio Max and yes, Photoshop. Compared to other major DCCs Photoshop is very inflexible which can lead to both frustration and innovation. Maya for example allows you to strip back the entire application if need be and customise almost every facet of it's inner workings. Photoshop does not. However the introduction of CS4 and soon the next iteration CS5 brings some major improvements to customisability of Photoshop bringing it closer to Game and Film production pipelines.

In this and the following posts I will explain the different ways Photoshop can be expanded and customised, giving examples of exposing some development voodoo, some of which may not be exactly what Adobe wants you to hear but it gets the job done. I will be posting links to the latest section articles on this page as I write them.  When giving more specific examples of of writing scripts etc I will be focusing more on development work flow rather than language How Tos.

In this post I will introduce the main concepts of expanding Photoshop CS4 from simple to complex. These articles may be read by people of all skill levels so all I'll assume is a good understanding of Photoshop concepts and UI.

Actions and Automation

Actions and automation are the simplest ways to make tasks easier and faster. An action is a macro recording of a series of events triggered by the user such as opening a file, deleting a layer or applying a filter. Automation is applying an action automatically over multiple images such as a folder of PSDs.

Pros:

• Very easy for anyone of a team to create and use.
• Easily imported and exported from Photoshop.
• Can include functionality from 3rd party plugins.

Cons:

• No automatic way of distribution to team members.
• Not every function in Photoshop is captured by the action system.
• Linear execution, no logic etc.

ExtendScripts
ExtendScript (JavaScript) in Photoshop Part 1: Work Flow Setup

Scripting has been the main way of expanding Photoshop since the early days. Scripts can be created in Javascript, VBScript and Applescript. I will only cover JavaScript in these articles since it is by far the most popular and the one that I know. Most functionality has native Javascript DOM support and there is also a plugin which allows you to listen to Photoshop's internal workings which can be scripted with a bit of work. Scripts can be hooked into Photoshop events and called when a file is opened for example. Modal tool windows can be created to add UI elements and also communication with Flash panels is implemented.

Pros:

• Large community support base.
• Many free scripts are available to download and study.
• Using the Script Logger output any Photoshop functionality can be scripted.
• Excellent IDE and debugging tools which integrate well with Photoshop.
• Works on all platforms (Win x86, Win x64, OSX) without the need to recompile.

Cons:

• Can be slow to execute.
• Not all Photoshop functionality has native scripting exposure and resorting to using the Script Logger output can be difficult and cryptic.
• Can only create modal tool windows.

Flash Panels

Flash Panels are the newest and best way to add floating panels and UI heavy tools. Flash Panels are developed in Flex Builder (Soon to be Flash Builder which is currently in beta) using ActionScript and MXML. Flash Panels come in two flavours: PatchPanel and CSXS Panels which expose different elements of Photoshop to Flash. Anything that can be done in Flash can be done in these panels including Pixel Bender shaders and network support.

Pros:

• Can create floating tool panels which integrate into the Photoshop UI.
• Can be used in other packages such as Illustrator.
• Can control most of Photoshop and hook into Photoshop events.
• When setup can debug Flash Panels as they execute in Photoshop.

Cons:

• Need to learn Flex IDE and MXML.
• Can slow down if large amounts of complex UI are shown.
• Can be tricky if MXML includes are not setup correctly.
• Debug setup can be a pain and often crashes.

COM Automation (Windows only)

COM Automation is an older windows technology that has been superseded by parts of .Net, however it still has it's uses.  It defines a standard way of applications to publish an external interface so other applications can hook ino them.  The greate part is with a little Visual Studio magic the COM interfaces can be converted to .Net assemblies and used with C# and other .Net languages.

Pros:

• Can utilise your existing programming skills in other languages such as C#.
• Easily integrates inot existing external pipelines and systems.
  

Cons:

• Tools are completely external so no UI integration etc.
  
• Can be hard to debug.
  

Pixel Bender Filters

Pixel Bender is Adobes propriety shader language similar to Cg or HLSL. Pixel Bender shaders can be used inside flash or as a filter in Photoshop, After Effects etc. This is a bit of a niche but is still a big part of Photoshop use in some companies / markets and much easier to create than native plugins.

Pros:

• Shader language is straight forward.
• Fast execution speed.
• Utilises graphics hardware.

Cons:

• Pixel Bender IDE is basic pre-release software and sometime frustrating to use.
• No debugging options.
• May be discrepancies between IDE and Photoshop results.

JavaScript Extensions

JavaScript extensions can be very powerful. I have only had experience creating DLLs for windows but am sure its is just as easy to create OSX extensions. JavaScript extensions are coded in C++ so almost anything is possible including using other DLLs and even managed code. For a test example I created a DLL which was compiled with CLR (enabling the .Net framework) which then loaded an assembly DLL created in C# which spawned a windows form which was correctly parented to the main Photoshop window using Win32 API calls. A recent development on the Adobe Labs Patch Panel forum suggest that native wrappers for JavaScript extensions DLL might be written in Flex and used with Flash panels but I have not confirmed this.

Pros:

• Can expose almost any functionality to JavaScript such as calls to .Net.
• MIGHT be able to be integrated into Flash panels natively.
• Existing DLLs can be used to access databases etc.
  

Cons:

• Need at least C language knowledge.
• Not many examples from Adobe and almost no information online.

Plugins

Plugins are the ultimate in extending Photoshop. They are written in C++ and can be used to reach into the depths of Photoshop's API. There are several major types of plugins including File I/O, Filters, Layer types etc. Since they are the most tightly integrated extension they do things on a lower level and execute quickly.

Pros: 

• Fast Execution. 
• Low level API access allowing integral parts of Photoshop to be expanded such as adding new file types etc.

Cons:

• Knowledge of advanced C++ techniques needed.
• Some examples come with the SDK but little documentation on “How To”.

Where to Start

ExtendScripts (using JavaScript) seems to be the best place to start assuming you have a good understanding of Photoshop's UI and work flow. Most general features of the JavaScript language will work so if you have any prior knowledge of C, C#, ActionScript etc you will pick it up quickly. The IDE is mature and makes debugging your scripts while they are executing within Photoshop very easy. Once you have some scripting experience under your belt it will be much easier to create Flash Panels, JavaScript extensions and plugins since most Photoshop programming terms run common through all areas.

Links

This page will serve as a repository for all the information I have run aground on during some rough seas.  Some of then turned out to be rather nice islands with coconuts and those little umbrellas in your drink...

Photoshop Development and Related Links

• Photoshop CS4 SDK - Photoshop Software Development Kit.  Worth a look if you are getting into some more advanced scripting or obvously plugin development.
  
• Bridge CS4 SDK - This has some examples of creating JavaScript extensions for use with your scripts and panels.
• 
  

3DS Max Maxscript and Plugins

Maya, Qt and Python

.Net and C# 

• Reflector - Awesome tool for exploring  .Net assemblies and decompiling code.
  

Tools 

Network and Web Tools

• HTTrack - This is a web site copier and offline browser.  Highly configurable to hoover down anything it can from a web site.  Useful for archiving sites and pulling down image catalogues.
• HTTP Recorder, TCP Spy and TCP Viewer - All from the same company, they are great for snooping and reverse engineering the TCP protocols between any client/server connection on a machine.
  

Windows Tools

• Windows Modifier - A small util that is not very user friendly but is great for testing out Win32 API ideas if you need to hack and slash your way through a problem.  Can send messages to windows as well as reparent controls and windows.  Unfortunately there is no x64 version.
  
• Winspector Spy - Another great tool for looking at the internal structure of windows.  Great if you need to get the HWND of a control for message testing etc.  Also invaluable for snooping windows messages to work out exactly what sneaky business is happinging under the sheets.  The main site seems to have gone but I managed to find the above link.
• DebugView - Lists any debug output any open applications are outputting.  Can be very handy for diagnosing some applications or looking at debug output if you don't have a debugger installed. 
• XMLPad - Awesome XML viewer / editor.  Can display XML data inall osrts of ways including tables.
• Process Monitor - Lets you spy on what resources each process is accessing on your system including files and registry entries.  A lot of spam at the start but once you get your filters right is invaluable for seeing what DLLs a process is accessing and when.
•  Process Explorer - Neat replacement for Task Manager and has a lot of extra info about processes.

 Work Flow Tools

• UltraVNC - One of if not the best VNC implementation.  Has many plugins and now worries on multiple monitors.
• Input Director - Most of us have multiple machines on our desk now days and this is the way to control them from a central keyboard and mouse.  Even supports multiple monitors on both the server and client. 
• Synergy - Like Input Director only cross platform and less features but still good.
• Dirkeys - Dirkeys is an awesome program that lets you assign short cut keys to any folder visible by explorer.  These short cuts can be used anywhere you can browse for files like windows explorer or file save dialogs.  Unfortneatly it only works in Windows XP.