Principal Component Analysis, Collision Volumes and You

It's been a loooong time between posts, and I guess a lot of distance as well.  This whole AAA business really takes it out of you :P

One of the last major coding projects I did before the awesomeness of asset wrangling hit me was an idea I had about auto collision volume creation.  My dream is to automate everything, leaving artist to worry about the big stuff, and collision volumes isn't something they should worry about.  So I started searching for a solution to my plan:

1. Find a way to automate the creation of best fit Bounding Box, Bounding Cylinder and Bounding Sphere collision primitives, mainly for environment art/props but could work for characters.
2. Find a way to automatically partition an arbitrary mesh into smaller parts to create bounding volumes.

This post will be addressing part 1.


First lets talk about bounding volumes, specifically bounding boxes.  There are 3 main types of bounding boxes:

1. World Space BBox:

This is where a bounding box is generated from the model's verts translated into world space.

2. Local Space BBox:

This is where the axises of the bound box are translated into the same local space as the model before being computed.

3. Oriented BBox (OBB):

This is the trickiest BBox to compute and the one mostly created by hand/eye by artists.  It's hard to compute because we're looking at an arbitrary cloud of vertex data with no reference orientation.  Sure we could use World or Local BBoxes but they don't really get the job done when you want a tightly fitting BBox.  Another advantage to Oriented BBoxes is that, well they're oriented.  This means that they not only fit the model very well, but they are also transformed correctly, this is very handy later on when you want to compute oriented bounding cylinders and capsules:

 

How the hell do we calculate an Oriented BBox you might ask?

The first solution I discovered wasn't very pretty.  Basically you just rotate a bounding box around the model (well really rotate the model around world space) and calculate the world space BBox for each rotation.  Repeat this A LOT and take note of the volume of each BBox and pick the BBox with the least volume.  Problems here are this can take a long time and it's not very accurate.  This might be ok for packing algorithms but not for our purposes.

Next I found out about PCA.


 PCA stands for Principal Component Analysis.
Here's the obligatory wiki entry for it:
http://en.wikipedia.org/wiki/Principal_component_analysis

Basically, given a random collection of points, ie mesh verts, PCA picks out the "best fit" vectors.  PCA is traditionally used in a lot of statistics maths but like most algorithms it's found a place in 3D.

Here's the pic from Wikipedia:

If you want to know how this works, PLEASE go and read this excellent tutorial by Lindsay I Smith:
www.cs.otago.ac.nz/cosc453/student_​tutorials/​principal_​components.pdf

It helped me no end to break down how to actually implement this.  It's been a while since I did this so I'm not going through the exact process here.  I basically made sure I had what Standard Deviation,Variance, Covariance and Covariance Matrix meant in my head, and wrote code to calc these values from a set of verts.

There's always a dirty word with these projects, and this time it's "Eigen"!  Well Eigenvectors and Eigenvalues to be exact:
http://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors

In true coding by numberz style I reached for the closest open source library I could find to calc Eigenvectors and Eigenvalue: Eigen!
http://eigen.tuxfamily.org/index.php?title=Main_Page


From Lindsay's article, here are the main steps:
Step 1: Get some data
Step 2: Subtract the mean
Step 3: Calculate the covariance matrix
Step 4: Calculate the eigenvectors and eigenvalues of the covariance matrix
Step 5: Choosing components and forming a feature vector

Steps 1 to 3 I implemented in Maxscript.  They're all pretty simple stats stuff.  Step 4 is a bit more tricky and this is where the Eigen library steps in.

I integrated the library into the 3DS Max SDK and exposed several functions to Maxscript.  Sorry peeps, I don't own any of the code so I can't post it here.

Here's a vid of the final tool in action with "Simple Arse"TM UI:



play video



 So that's got the automation of individual parts done.  Now for the big question:

How the hell do we automatically split up a model into primitive shapes?

Did somebody say "Approximate Convex Decomposition of Polyhedra"?  I hope so because that's where I'm going with this.  I've done A LOT of study on this sort of thing over the years and was quite chuffed when I wrote my first convex hull generator in the 3DS Max SDK.

Here's a link to mad SIGGRAPH scientist who came up with this concept:
http://masc.cs.gmu.edu/wiki/ACD

I've read through the paper and it's not that difficult, just very involved and I haven't been able to justify the time cost in developing a product.  The basic idea would be:

1. Preprocess the models mesh to identify any non contiguous mesh, open edges etc.  Either treat these separately or create a shrink-wrap mesh around the original mesh.
2. Decompose all resultant mesh elements into approximate convex parts.
3. Generate best fitting bounding volumes or convex hulls from approximate convex parts.
4. Run some quick tests to optimize volumes.  If for example a part of the mesh was very pointy you could reduce the size of the volumes so they were slightly inside the mesh, giving a more realistic physics volume.

That's the plan anyway.  Peace out.

Developing custom Qt UI widgets for Maya 2011: OMFG it works!

Well after much stuffing around I finally got this working.  It was one of those problems that had several major pieces that all need to fit.

These are the major things I had to work out:

1.        You must compile your widget plugin with the same Qt SDK version as Maya.  Maya 2011 pre-release uses 4.5.3. 

2.       Your plugin DLL and therefore the base Qt libraries must be compiled with the same compiler that was used to create Maya.  On windows this is MS Visual Studio 2008.  I don’t know about other platforms sorry.

3.       It’s best if you put your plugin DLL in the
C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\

Now basically what we need to do is compile Qt from scratch using nmake which is the Visual Studio compiler.  We are also setting up Visual Studio as our main development environment so your final plugins will also be compatible.  Different compilers actually make DLLs in different ways so a DLL compiled with minGW won’t work with the Maya binary because it was compiled with Visual Studio.    And since we are using VS to compile our plugin the base Qt libraries also need to be compiled using VS.  This is a once off this and I’ve found that VS is a much better IDE than Qt creator.  You can even still use Qt designer to create your UI!

 

Here’s what you’ll need to get this working on Windows:

·         Microsoft Visual Studio 2008 (Best to have the latest service packs etc)

·         Qt SDK version 4.5.3 source package.  You won’t find any links to this on the Qt site.  You can get it here:   http://get.qt.nokia.com/qt/source/qt-all-opensource-src-4.5.3.zip.  FYI if you want to browse the available source packages navigate to http://get.qt.nokia.com/qt/source

·         Qt Visual Studio Add-in.

·         About a day of time for Qt to compile.

And these are the exact steps I did to get Qt 4.5.3 compiled:

1.       Go to this blog post then read my next points BEFORE following the instructions: http://dcsoft.wordpress.com/2010/01/30/how-to-setup-qt-4-5-visual-studio-integration/  Huge props go out to DCSoft for writing this up.

2.       Skip the DOWNLOAD QT SOURCE CODE section. Just download the source package from the link above and unzip it here: c:\Qt\ 4.5.3-vc.  Ie after unzipping the files you should see a structure that has this path: C:\Qt\4.5.3-vc\src.

3.       In SET ENVIRONMENT VARIABLES step 1 you need to change the path: QTDIR = c:\qt\4.5.3-vc

4.       In BUILD VC++ VERSION OF QT substitute our Qt version everywhere ie replace 4.6.1-vc with 4.5.3-vc

5.       Wait about 8 hours or so for Qt to compile.

6.       In INSTALL VISUAL STUDIO ADD-IN step 3 replace 4.6.1-vc with 4.5.3-vc

Now we should have a fully functional version of Qt 4.5.3 complete with Creator and Designer binaries and VS project files for all the source examples.       

Here’s what I did to get an example widget plugin compiled and working in Maya 2011:

1.       Boot up Visual Studio and load this project:  C:\Qt\4.5.3-vc\examples\designer\worldtimeclockplugin\ worldtimeclockplugin.vcproj

2.       Change the configuration to Release and compile.  The compiled DLL will be here: C:\Qt\4.5.3-vc\plugins\designer\worldtimeclockplugin.dll

3.       Copy the DLL into the Maya folder:  C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\

4.       Run this Python code:

And with a bit of luck you should see this: 

Awesomely the connections between the widget elements still work which means you can define quite a bit of functionality in QT Designer

Next up is investigating what needs to happen to compile a 64bit version.

Qt Fun and Games

I love being a noob again, all the fun and exitement of trolling forums for hours and ripping large clumps of hair out.

I have been doing my usual thing of diving head first into a project and over the last couple of day been trying to create a basic custom OpenGL widget for Qt on windows 7.  Now remember I think these are all windows related bugs.

The first peice of Qt awesomeness I found is that when you compile any of the test custom widgets you can't actually use them in the designer.  Why? Because apparently Qt Creator and Designer have been compiled with MSCV 2005, and by default Qt uses mingGW as it's compiler, making the plugin binaries incompatible with the main app.  So to fix this you either need to download the Creator/Designer src and compile it yourself with minGW (which looks to be a Shit Onion all of it's own).  Or create your own build chain using MS nMake for which there is little information on and no help.

So I thought "Who needs the designer, I can just code all the UI for now" *vomit*. So I started trying to load my plugin into another test app, then awesomeness part two came along.

Qt seems to have a way of interrogating DLLs similar to .Net reflection, but for this to happen you need to invoke the Meta-Object-Compiler (moc).  This generates moc src files that are used for linkage.  And of course my project wasn't generating them.  So after some trolling I found this post which pointed to the path to moc.exe being wrong in the make files.  That's a Bingo!  I changed the forward slashes to back slashes in my make files and it worked fine.

It really seems Windows was kinda forgotton by the Qt developers...

Developing custom Qt UI widgets for Maya 2011: A first look

This has been solved!!!

Check out my newer post here


Now that I’ve been using the new look Maya I kinda like it. The Maya UI always seemed cold and impersonal compared to Max which now for me is like a third brain lobe. I’ve also been looking for an excuse to learn Qt and Maya 2011 fits the bill.

For those who don’t know Qt is a cross platform system mainly used for the front end of applications but has other libraries such as database and web access. When Nokia bought Qt it sent good vibes out into the industry. Basically everyone now knows a large corporation is investing in this thing so it must be good. It’s also good to know someone is there to support the community and drive innovation in Qt. The CG industries have especially taken liking to Qt because it’s UIs look and perform very well, have GL integration and are consistent between platforms. These are things that help sell a product to end user artists.

Here’s a link to some hype, but this time you can probably believe it:

http://qt.nokia.com/qt-in-use/qt-in-visual-effects

And now Qt’s use in Maya has sealed the deal as an early industry standard.

Now for the disclaimer: I’m currently learning Qt and relearning Maya development so please take anything I say here with a pinch of salt.

Here are some early finding:

You need to be developing with the same Qt version as the app you are developing plugins for (I’m talking Qt system plugins here not Maya SDK plugins).

So I found a MEL command that returns the Qt version used by Maya:




Now as of time of writing the latest Qt version is 4.6.1. The latest release of the full Qt SDK package using 4.5.3 framework is 2009.04 , which can be found here (Windows):

http://get.qt.nokia.com/qtsdk/qt-sdk-win-opensource-2009.04.exe

To be clear, a UI element in Qt is called a Widget.

Qt has it’s own nifty plugin system for all sorts of stuff which has standard plugin folder names. From trawling the Qt help and Maya 2011 folders I found these:

From the Qt help

Locating Plugins

Qt applications automatically know which plugins are available, because plugins are stored in the standard plugin subdirectories. Because of this applications don't require any code to find and load plugins, since Qt handles them automatically.

And it just so happens that we have such a folder here:

C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins

Now Qt plugins need to be in strictly named subfolders.

From the Qt help

Base Class	Directory Name	Key Case Sensitivity
QAccessibleBridgePlugin	accessiblebridge	Case Sensitive
QAccessiblePlugin	accessible	Case Sensitive
QDecorationPlugin	decorations	Case Insensitive
QFontEnginePlugin	fontengines	Case Insensitive
QIconEnginePlugin	iconengines	Case Insensitive
QImageIOPlugin	imageformats	Case Sensitive
QInputContextPlugin	inputmethods	Case Sensitive
QKbdDriverPlugin	kbddrivers	Case Insensitive
QMouseDriverPlugin	mousedrivers	Case Insensitive
QScreenDriverPlugin	gfxdrivers	Case Insensitive
QScriptExtensionPlugin	script	Case Sensitive
QSqlDriverPlugin	sqldrivers	Case Sensitive
QStylePlugin	styles	Case Insensitive
QTextCodecPlugin	codecs	Case Sensitive

But there is no mention of Widgets here you might ask, so did I. So from reading some help and looking in the Qt install folder I found that you need to put Widget plugins in a designer folder. Ie

C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\myWidgetPluginProbablyGoesHere.DLL

If you’re looking for examples of custom Widgets look in the Qt Designer or just Designer sections. I’m using the World Time Clock Plugin example till I make my own widget.

Now I booted by Qt Creator 2009.4 using framework 4.5.3 and compiled the World Clock project which produces a file called worldtimeclockplugin.dll. So I copied it here:

C:\Program Files (x86)\Autodesk\Maya2011\qt-plugins\designer\worldtimeclockplugin.dll and then loaded the UI file in the same project using Python:




This code created a window with the other standard widgets but the clock widget was not there. One of my main asumtions about this was that Maya would actually load my DLL so I booted up Process Monitor and checked what was going on when I ran the Python code. Sure enough Maya was accessing my DLL where I put it but it was also looking for it here:

C:\Program Files (x86)\Autodesk\Maya2011\bin\designer

I can remember reading somewhere in the Qt help that it will look for the standard plugin folders in the execution path so fair enough, I put the DLL there as well.

Still no widget. Maya is not generating any errors and according to Process Monitor all the DLLs it’s looking for are being found.

And that’s where I’m up to. Not much to go on except stuffing around with versions etc and looking at the Qt help. Hopefully I’ll have some actual results soon.

Photoshop and Max Example Tool Videos

Howdy,

Here's a couple of videos I've made recently to demo some Photoshop and 3DS Max tools.  The videos don't have sound but I added comment bubbles thoughout.

Thought I'd make this post for two reasons:

• Show off some of my work
• Learn how to embed video into Blogger

All videos and tools are copyright Krome Studios 2009-2010.

Photoshop Exporter and Exporter Panel

play video

3D Studio Max Selection Set Manager and Export Manager

play video




3D Studio Max Scene Sync Manager

play video

3D Studio Max Convex Volumes

play video

ExtendScript (JavaScript) in Photoshop Part 1: Work Flow Setup

Photoshop's primary method for creating new tools is through ExtendScript. ExtendScripts can be implemented in three different languages: JavaScript, VBScript and AppleScript. JavaScript is by far the most popular and the only cross platform option. This article will explain all the elements needed to start developing scripts for Photoshop as well as some more advanced topics such as loading external libraries. I will also be concentrating for now on the Windows platform when discussing external libraries and file paths and assuming Photoshop CS4 or above.

What can you do with scripts in Photoshop:

• Write tools without UI that can be executed from the File >> Script menu or from actions.
• Write tools with modal UI that can be executed from the File >> Script menu or from actions.
• Assign scripts to execute when events are triggered within Photoshop.
• Access the local file system.
• Load external libraries for extra functionality.
• Access XMP data.
• Control almost all parts of Photoshop.

What can't you do with scripts in Photoshop:

• Scripts cannot create non modal windows, ie tool palettes.
• Modify the standard Photoshop menus or tool panels.
• Give Photoshop the ability to handle new image file types.
• Create Filters.

I don't like the word “can't” so I will be listing ways to achieve the above points with other technologies later.

Setting Up Your Work Flow

First thing you need to do is make sure you have the following apps and documentation installed and handy:

1. Adobe ID. If you haven't already go create yourself an Adobe ID, will make life easier and you will need it to post on the Adobe support forums. If you are into privacy or in a hurry you can use BugMeNot to login.
2. Photoshop CS4 or above.
3. ExtendScript Toolkit. This is the IDE you will be working in.  
4. BareTail. This is a real-time log viewer with basic keyword highlighting. 
5. Photoshop CS4 Scripting Guide. General Scripting info. Is installed with Photoshop or get it HERE.
6. Photoshop CS4 JavaScript Reference. JavaScript specific info plus Photoshop DOM reference. Is installed with Photoshop or get it HERE.
7. Javascript Tools Guide CS4. I don't know why Adobe hides this doc away. It contains information about XML, XMP, file system access and accessing external libraries from JavaScript.

Here are some locations of interest and places to get help:

Official Adobe Photoshop Scripting Forum – The best resource for information and help.

PS Scripts – Go straight to the forum. Another great resource with plenty of code snippets hidden away in posts.

Scripting Photoshop from C# - For those of you adventurous enough to try some COM action.

Dr WooHoo! – A great resource for anything on the cutting edge of Adobe development with plenty of tutorials, especially if you are doing anything Flash related.  Make sure you check out his blog.

W3 School JavaScript Tutorials – A good resource for learning the JavaScript language.

Locally you want to check out these folders (assuming standard install):

Scripts Folder

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Scripting

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Scripting

Here you will find some of the above mentioned PDFs in the documentation folder and also sample scripts.

Presets Scripts

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Presets\Scripts

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Presets\Scripts

Here you will find the scripts available in the File >> Scripts menu, also the scripts available by default to assign to events.

Script Listener

The Script Listener is a plugin which logs all actions and events running through Photoshop to a text file on your desktop. Why do we need this? Well not all Photoshop commands are natively scriptable through the JavaScript DOM. That means there aren't specific commands for all actions in Photoshop. However 99% of Photoshop's actions are logged by the Script Logger which can be copied and pasted into a script. This is also useful when scripting the functionality of 3^rd party plugins.

To install the Script Listener close Photoshop and copy this file:

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Scripts\Utilities\ScriptListener.8li

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Scripts\Utilities\ScriptListener.8li

to here:

x86 Systems:

C:\Program Files (x86)\Adobe\Adobe Photoshop CS4\Plug-ins\

x64 Systems:

C:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Plug-ins\

If you want you can make a ScriptListener Folder in the Plug-ins folder and put it there.

To check it is working correctly open Photoshop and create a new image. Go to your desktop and you should find a file called ScriptingListenerJS.log which is a standard text file.

BareTail is a real-time log viewer which means it automatically displays any new log entries. It also has some simple keyword highlighting to make it easier to view the log. You can download a BareTail prefs file here to get you started. Once BareTail is installed choose Preferences >> Load From File....

The only thing BareTail doesn't do is clear the log. If you want you can create a batch file on your desktop which does this and add a shortcut to it:

ClearPhotoshopLogs.bat

And now you are ready to start scripting any solution in Photoshop. Stay tuned for part 2 of this adventure when we explore some scripting concepts that are not covered in most tutorials.

Expanding Photoshop's Horizons: Plugins, Scripts and Flash

Working in any large scale creative team means big work flow problems at one time or another. Solving these problems often means creating custom tools for key pieces of software such as Maya, 3D Studio Max and yes, Photoshop. Compared to other major DCCs Photoshop is very inflexible which can lead to both frustration and innovation. Maya for example allows you to strip back the entire application if need be and customise almost every facet of it's inner workings. Photoshop does not. However the introduction of CS4 and soon the next iteration CS5 brings some major improvements to customisability of Photoshop bringing it closer to Game and Film production pipelines.

In this and the following posts I will explain the different ways Photoshop can be expanded and customised, giving examples of exposing some development voodoo, some of which may not be exactly what Adobe wants you to hear but it gets the job done. I will be posting links to the latest section articles on this page as I write them.  When giving more specific examples of of writing scripts etc I will be focusing more on development work flow rather than language How Tos.

In this post I will introduce the main concepts of expanding Photoshop CS4 from simple to complex. These articles may be read by people of all skill levels so all I'll assume is a good understanding of Photoshop concepts and UI.

Actions and Automation

Actions and automation are the simplest ways to make tasks easier and faster. An action is a macro recording of a series of events triggered by the user such as opening a file, deleting a layer or applying a filter. Automation is applying an action automatically over multiple images such as a folder of PSDs.

Pros:

• Very easy for anyone of a team to create and use.
• Easily imported and exported from Photoshop.
• Can include functionality from 3rd party plugins.

Cons:

• No automatic way of distribution to team members.
• Not every function in Photoshop is captured by the action system.
• Linear execution, no logic etc.

ExtendScripts
ExtendScript (JavaScript) in Photoshop Part 1: Work Flow Setup

Scripting has been the main way of expanding Photoshop since the early days. Scripts can be created in Javascript, VBScript and Applescript. I will only cover JavaScript in these articles since it is by far the most popular and the one that I know. Most functionality has native Javascript DOM support and there is also a plugin which allows you to listen to Photoshop's internal workings which can be scripted with a bit of work. Scripts can be hooked into Photoshop events and called when a file is opened for example. Modal tool windows can be created to add UI elements and also communication with Flash panels is implemented.

Pros:

• Large community support base.
• Many free scripts are available to download and study.
• Using the Script Logger output any Photoshop functionality can be scripted.
• Excellent IDE and debugging tools which integrate well with Photoshop.
• Works on all platforms (Win x86, Win x64, OSX) without the need to recompile.

Cons:

• Can be slow to execute.
• Not all Photoshop functionality has native scripting exposure and resorting to using the Script Logger output can be difficult and cryptic.
• Can only create modal tool windows.

Flash Panels

Flash Panels are the newest and best way to add floating panels and UI heavy tools. Flash Panels are developed in Flex Builder (Soon to be Flash Builder which is currently in beta) using ActionScript and MXML. Flash Panels come in two flavours: PatchPanel and CSXS Panels which expose different elements of Photoshop to Flash. Anything that can be done in Flash can be done in these panels including Pixel Bender shaders and network support.

Pros:

• Can create floating tool panels which integrate into the Photoshop UI.
• Can be used in other packages such as Illustrator.
• Can control most of Photoshop and hook into Photoshop events.
• When setup can debug Flash Panels as they execute in Photoshop.

Cons:

• Need to learn Flex IDE and MXML.
• Can slow down if large amounts of complex UI are shown.
• Can be tricky if MXML includes are not setup correctly.
• Debug setup can be a pain and often crashes.

COM Automation (Windows only)

COM Automation is an older windows technology that has been superseded by parts of .Net, however it still has it's uses.  It defines a standard way of applications to publish an external interface so other applications can hook ino them.  The greate part is with a little Visual Studio magic the COM interfaces can be converted to .Net assemblies and used with C# and other .Net languages.

Pros:

• Can utilise your existing programming skills in other languages such as C#.
• Easily integrates inot existing external pipelines and systems.
  

Cons:

• Tools are completely external so no UI integration etc.
  
• Can be hard to debug.
  

Pixel Bender Filters

Pixel Bender is Adobes propriety shader language similar to Cg or HLSL. Pixel Bender shaders can be used inside flash or as a filter in Photoshop, After Effects etc. This is a bit of a niche but is still a big part of Photoshop use in some companies / markets and much easier to create than native plugins.

Pros:

• Shader language is straight forward.
• Fast execution speed.
• Utilises graphics hardware.

Cons:

• Pixel Bender IDE is basic pre-release software and sometime frustrating to use.
• No debugging options.
• May be discrepancies between IDE and Photoshop results.

JavaScript Extensions

JavaScript extensions can be very powerful. I have only had experience creating DLLs for windows but am sure its is just as easy to create OSX extensions. JavaScript extensions are coded in C++ so almost anything is possible including using other DLLs and even managed code. For a test example I created a DLL which was compiled with CLR (enabling the .Net framework) which then loaded an assembly DLL created in C# which spawned a windows form which was correctly parented to the main Photoshop window using Win32 API calls. A recent development on the Adobe Labs Patch Panel forum suggest that native wrappers for JavaScript extensions DLL might be written in Flex and used with Flash panels but I have not confirmed this.

Pros:

• Can expose almost any functionality to JavaScript such as calls to .Net.
• MIGHT be able to be integrated into Flash panels natively.
• Existing DLLs can be used to access databases etc.
  

Cons:

• Need at least C language knowledge.
• Not many examples from Adobe and almost no information online.

Plugins

Plugins are the ultimate in extending Photoshop. They are written in C++ and can be used to reach into the depths of Photoshop's API. There are several major types of plugins including File I/O, Filters, Layer types etc. Since they are the most tightly integrated extension they do things on a lower level and execute quickly.

Pros: 

• Fast Execution. 
• Low level API access allowing integral parts of Photoshop to be expanded such as adding new file types etc.

Cons:

• Knowledge of advanced C++ techniques needed.
• Some examples come with the SDK but little documentation on “How To”.

Where to Start

ExtendScripts (using JavaScript) seems to be the best place to start assuming you have a good understanding of Photoshop's UI and work flow. Most general features of the JavaScript language will work so if you have any prior knowledge of C, C#, ActionScript etc you will pick it up quickly. The IDE is mature and makes debugging your scripts while they are executing within Photoshop very easy. Once you have some scripting experience under your belt it will be much easier to create Flash Panels, JavaScript extensions and plugins since most Photoshop programming terms run common through all areas.