[zz] Pixar’s OpenSubdiv V2: A detailed look

http://www.fxguide.com/featured/pixars-opensubdiv-v2-a-detailed-look/

Pixar’s OpenSubdiv V2: A detailed look

By Mike Seymour

September 18, 2013

Subdivision is key modeling tool that allows greater accuracy and the OpenSubdiv project aims to standardize and speed up the process. It is production proven and about to be adopted for wide scale use by products like Autodesk’s Maya and Side Effects Software’s Houdini. In this exclusive and in-depth piece we explore the history, the uses and the innovative new approaches OpenSubdiv offers.

The subdivision of a surface is really a process whereby one can represent a smooth surface via a coarser piecewise linear polygon mesh, and then calculate a smooth surface as the limit of a recursive process of subdividing each polygonal face into smaller faces that better approximate the smooth surface. OpenSubdiv is at the heart of Pixar’s modeling pipeline and it is now being expanded to many other pipelines due to its open source approach.

Catmull-Clark subdivision surfaces were originally invented in the 1970s. The specification was extended with local edits, creases, and other features, and formalized into a usable technique for animation in 1998 at Pixar. Until recently the technology had not enjoyed widespread adoption in animation. This is about to change. Pixar has now decided to release its subdivision patents and working codebase in the hope that giving away its high-performance GPU-accelerated code will create a standard for geometry throughout the animation industry.

---

 The “limit surface” is the surface produced from the subdivision process being iteratively applied infinitely many times. In practical use, however, this algorithm is only applied a limited number of times, often as few as say 3 for a background character, perhaps 5 or 6 for a hero character. Each iteration multiplies the number of faces by 4, so 6 iterations multiplies the ‘draw quality’ by 4096.
Think of the limit surface as the ‘true’ shape.

Subdivision surfaces are used for final rendering of character shapes
and models as they produce a smooth and controllable ‘limit surface’.
However, subdivision surfaces when represented at desktop level or say
in an animation package are often drawn as just their polygonal control
hulls purely for performance reasons. The ‘polygonal control hull’ is an
approximation that is offset from the true limit surface. But looking
at your work as an approximation in the interactive viewport makes it
difficult to see exact details and achieve a good performance. For
example, it is hard to judge fingers touching the neck of a bottle or
hands touching a cheek without a limit surface. It also makes it
difficult to see poke-throughs in cloth simulation if the skin and cloth
are both only rough approximations.

---

Is Subdivision new?

People
have built high polygon models for years, but the same results can be
achieved now with a much lower poly count with OpenSubdiv. Subdivision
surfaces generally provide great advantages such as arbitrary topology
and fantastic control. You can put the points where you need them and
one does not need to add extra points – one just models naturally. Many
people switched to some form of subdivision in around 2005-2006 when
Maya first offered subdivision. Prior to Maya’s adoption, studios with
RenderMan would simply model in poly cages and pass those to RenderMan
for subdivision. In 2005 Maya began offering an accurate CPU based
“preview” which artists found invaluable. But the definition of
subdivision varies greatly and it is hard to move a model from package
to package or renderer to renderer and have them load and look the same.
If the industry is fairly agreed that subdivision is a good idea – and
even the geometry primitive of the future – then, as Pixar TD Manuel
Kraemer points out, “we should at least agree on what the definition of a
subdivision surface - Catmull-Clark or whatever – we should agree on
what it really is and the devil is in the detail!”

OpenSubdiv is a consistent implementation of subdivision, with GPU
and CPU versions, and one that promises greater interoperability and
interchange – in short, it offers a professional standard. The resulting
output from an OpenSubdiv pipeline is streamlined but it also means
that models will look the same and have the same controls, weights and
look in a variety of packages, something not true before OpenSubdiv.

OpenSubdiv offers features like creases (edge sharpness), limit
surface evaluation, hierarchical edits. Plus it offers modeling speed
and memory reduction (very important with sculpting), and it offers
interoperability.

An example in practice

Both Maya’s animation interface and Pixar’s proprietary Presto
animation system can take up to 100ms to subdivide a character of 30,000
polygons to the second level of subdivision (500,000 polygons). But doing
the same thing with OpenSubdiv’s GPU codepath takes 3ms and allows the
user to see the smooth, accurate limit surface at all times. The aim
of OpenSubdiv is to allow the artist to work with a more faithful
representation of what you are doing. It can render faster, and allow
for greater interoperability. But this is not a plugin. You cannot
simply run your existing high poly count models through OpenSubdiv to
make them faster. Rather, you need to think about modeling at much lower
resolution and let the subdivision process – managed by OpenSubdiv –
provide the high quality, high poly count surface at render time.

There are some key aspects to the OpenSubdiv project, including some really interesting new opportunities:

1. Simplicity with edges/creases
2. Efficiency with textures
3. GPU implementation
4. Hierarchical modeling

But before looking at those current issues it is worth looking at some of the background of OpenSubdiv.

Heritage and background

“We’re very excited…It’s nice to have a standard implementation
that’s going to produce the same results in all software. This is huge
for interoperability between software packages.”

  Mark Elendt

  Side Effects Software – Houdini

OpenSubdiv is an open source initiative that saw Pixar allow use of
key patents and technology in the general community. Pixar has heavily
promoted OpenSubdiv in an effort to one day have a powerful interchange
format that both works very quickly on the GPU and very reliably on the
CPU. While subdiv did exist outside the work Pixar was using internally,
most implementations were based on papers and research done by staff at
Pixar. And ever since Pixar moved to allow their research to be in the
open source community many other companies have started to come on
board. The team at Luxology licensed the core technology before there
was even a formal ‘OpenSubdiv program’.

Autodesk has aggressively supported Pixar both for their internal
tools and now in Maya so that soon everyone can benefit. Side Effects
Software is coming on board with Houdini and now even realtime
applications are appearing. Remarkably it is only a year since Pixar announced their decision to
put subdiv into the open source community. The idea came from a pitch
that Manuel Kraemer made to Bill Polson, Pixar’s Director of Industry
Strategy. As Polson recalls it, pretty much everything that Kraemer
pitched in that meeting has come to fruition today. “I don’t think
enough credit can go to Manuel Kraemer,” Polson says. “A few years ago
he came into my office and said, ‘Bill, I think we need to do some real
work with subdivision and try and get it to be an industry standard. He
kind of laid it all out on my whiteboard – I wish I’d taken a picture of
that whiteboard!”.

The concern Pixar’s internal team had was that it just made too much
sense to not have a standard, but as Pixar owned many of the key patents
and the keys to RenderMan, it would be hard for anyone else to do as
good a job, or deal with some of the complexity like facevarying,
hierarchy or the important ‘creasing’. This meant that a less than
optimal version could, de facto, become a standard if Pixar kept quiet
and withheld support of their technology and patents. Pixar would then
be faced with having a standard that was inferior to their internal
model and all hopes of interoperability would be lost. Furthermore, for
subdiv to work well Pixar wanted a great GPU implementation and any
graphics card company would, for the same reasons, end up not supporting
the possibly ‘secret’ superior Pixar version. By Pixar adopting the
open source model it stands a great chance of companies like Autodesk,
The Foundry,  Smith Micro Software (Poser) and Nvidia completely and faithfully supporting the Pixar model, and that benefits everyone.

In giving their subdiv away Pixar would get back both industry
adoption (and thus hardware support) and Pixar artists – and artists
worldwide – could work more effectively. Pixar has actually done
comprehensive studies on artists animating with IK rigs etc that show
not only is responsiveness cost effective, but as systems pass certain
responsiveness thresholds (50 ~ 60 frames per second in terms of
interactive refresh times) artists grip their pens or mice less firmly,
relaxing arm and wrist muscles and thus reducing strain injuries. By
making their artists able to enjoy faster responsiveness, Pixar saves
production time – and more importantly their artists are healthier.

Geri’s Game: the first use of subdivision inside Pixar in production.

The current open version is based on the fifth-generation subdiv
library in use by Pixar’s own internal proprietary animation system in a
lineage that started with code written by Tony DeRose and Tien Truong
for Geri’s Game in 1996. Each generation has been a complete
rewrite that has built upon Pixar’s experience using subdivision
surfaces on animated features year after year. The open source code is
live and real, so Pixar’s own changes to OpenSubdiv are released back
into the open source community at the same time they are rolled out into
Pixar’s animation productions.

1. Simplicity with edges/creases

Base mesh

Professional models use a lot of bevels – they mirror the reality of
the real world while also catching the light and making any manufactured
object seem less computer generated. Bevels are common place but one
level of bevel on a cube just on the face count from 6 to 26 and the
edges to 48 (from 12). To make that bevel a subtle curved edge, the
faces jump to say 98 and the edges jump to 192. OpenSubdiv has a single
crease value that can adjust a bevel from sharp to rounded.

The same model with subdiv and creasing weights

The crease editor does not change the base edge count. This adding of
detail and polish without exploding the data size of the model has been
used extensively and until now entirely by Pixar.

Creases was an major extension of Subdiv. The first feature film to use creases was The Incredibles (2004).

The complexity solution of creases was useful on a character such as
Buddy Pine and Syndrome’s gadgets – but was vital on say the hubs of
Lightning McQueen’s wheels in Cars (2006).

Below are the base mesh (left) and the subdivided final (right).

---

McQueen is a very interesting example. He was 6 to 10 times heavier
in faces than anything that had come before at Pixar (partly as he has a
fully modeled engine and transmission that few is any viewers have ever
noticed or seen). But then came Wall•E in 2008. “This guy in a single
one of his tracks has more faces than McQueen put together. Wall•E is
equivalent of almost 3 McQueens,” says Kraemer.  ”Heavy as Wall•E is,
which is an order of magnitude more complex – if he was modeled out of
polygons and you included all the bevels and softening of edges that
Wall•E really needs, if done with polygons – it would have been
completely impossible to work with, you would not be able to animate it,
you would not be able to deform it, pose it, articulate it – nothing.
All these operations have to move the vertices around, not just draw
them on the screen but actually move them in space, but it would have
been impossible (without Subdiv).”

Wall•E would not have been possible to animate and work with without subdiv and creasing.

Complexity of parts and their bevels was much higher than any other lead character had been before for Pixar.

---

2. Efficiency

Using OpenSubdiv versus a simple polymesh can mean a big difference
in mesh count. To match a simple coffee cup using OpenSubdiv allows a
base mesh of 472 Tris to produce a nice curved OpenSubdiv final coffee
cup model, with creasing control etc. To produce a high level poly mesh
you would need to go to around 7552 triangles (level 2) for a smooth cup
to animate with. To subdivide to 4 levels one would end up with a cup
model equal to 120832 Tris, which is quite possible if you expected to
get close to complex compound handle shapes (vs a base mesh of only 472
Tris).

It is important that an individual model be efficient and so characters such as Woody from Toy Story
result in a workable model which is 30,000 faces. McQueen’s car
represents some 70,000 faces, but there is another important
consideration. If one moves to focusing on background props and elements
the multiplying effect can be even more impactful. It is great to save
complexity on hero characters for animator feedback and responsiveness
but it is absolutely vital to manage background complexity. In Monsters University
there was a simple balustrade object. It was very simple compared to a
hero character, just a mere 581 faces and 664 verts when modeled.

---

 Below is a background balustrade used in Monsters University, (left) Base mesh, (right) Subdiv.

---

But consider that one object is re-used on a host of buildings and
places on the MU campus. The one simple balustrade was used and modified
2083 times in one scene. If the item was identical it could be
instanced, but if each is different and/or instancing is not possible,
then the cost really adds up. Any simplification in faces and verts on
that object is a saving realized thousands of times over. Double its
complexity and you could equally add 40 McQueen hero car characters for
the same cost, or a hundred Woody level characters. In making a film,
not spending your memory and model complexity budget on background
objects without compromising the film’s subtle complexity is vital.

A map showing the deployment and wide use of the Balustrade object

Perhaps a better example than say the Balustrade would be dorm room set from Monsters University.

The first image (left below) is the Subdiv cage, and it has 415K
faces, or some 55 times the complexity of a coffee cups model above. The
second image (right below) is the polygonalized version (at two levels
of subdivision), this would offer a similar quality result in a poly
based pipeline. The second image is 6.75 M faces, which would be
equivalent to 900 ‘coffee cups’ in comparative terms.

Click to enlarge to see polygon edging

click to enlarge higher quality version

---

And there are more gains to be made.

In the GPU currently there is partial refinement of the subdivision
surface. The GPU has Feature Adaptive Refinement, it refines the area of
the mesh around an adaptive refinement area more densely. Subdivision
is a process that does not care about topology at all.

Subdivision gives the same result as Bspline patches wherever the
surface is regular,  all quads in a grid. But whereas Bsplines fail in
irregular areas, subdivision always works. That was the original
motivation for its invention. The OpenSubdiv GPU codepath uses this fact
to aggressively render Bspline patches wherever it finds regular grids.
But where the surface is irregular, OpenSubdiv continues subdividing
and refining, finding more Bsplines, and so on. Both approaches give the
same limit surface, except the Bspline code path is much much faster on
the GPU. Typical models have regular grids on 90% or more of the
surface, so this provides a huge speedup.

In the GPU codepath, the amount of subdivision and the density of
Bspline tessellation are both independently controllable over the
surface. So a part of the model close to camera can have more detail
than a part of the same model far away.

Unfortunately, for now, the CPU codepath supports only uniform
subdivision – i.e. the entire model undergoes the same amount of
subdivision even if part is close to camera and part is far away.
Without the flexibility offered by modern GPU tessellation, the
OpenSubdiv CPU codepath is less performant than some current popular
implementations, such as found in Arnold. Bill Polson agrees that the
CPU code could benefit from the same level of optimization that the GPU
code has had, and says the Pixar OpenSubdiv team hopes to address this
in 2014.

This is why Solid Angle is not supporting OpenSubdiv right now in Solid Angle’s Arnold.

Marcos Fajardo, founder of Solid Angle, has already given Bill Polson
the exact details, but the summary is that Sony Pictures Imageworks
(SPI) researchers did an experiment a year ago where they integrated
OpenSubdiv into the Arnold renderer at SPI to see how it performed. It
was much slower, and using more memory, than the subdiv engine already
used in Arnold. (SPI’s version of Arnold and Solid Angle’s more general
version are in sync and run in parallel in this respect).

“So we don’t use it,” says Fajardo. “Apparently OpenSubdiv is
blazingly fast on the GPU, but the OpenSubdiv team (Pixar) haven’t paid
equal attention to optimizing the CPU code. At least for uniform
subdivision, which is what we use, not the adaptive subdivision that
Pixar favors. This is because the REYES algorithm is camera-dependent
and optimizes meshes for the current camera, unlike with Monte Carlo
physically-based renderers where it’s less clear that adapting
tessellation to the camera is a big win, because most rays are not
camera rays anyway but shadow and GI rays. I think the OpenSubdiv team
is aware of this and they would like to optimize things, and there may
even be other studios who can contribute to fixing this, since it’s open
source. So, we are keeping an eye on OpenSubdiv, and once it’s more
efficient for our use cases we’ll re-evaluate.”

3. GPU implementation

To be able to work at the desktop, Pixar felt it was important to
have a GPU version of OpenSubdiv, so as a key part of OpenSubdiv 2.0
Pixar worked with others to implement an open GPU solution. The GPU
version supports OpenCL, CUDA, GLSL, DirectX11, OpenGL, Open GL ES. It
allows for uniform subdivision and for feature adaptive subdivision on
GPUs with tesselation. A GPU mobile computing OpenSubdiv working
implementation was shown at Siggraph, and while impressive this did not
have hardware based tesselation. When this is added – as it will be in
upcoming hardware revs – the results will be even more remarkable. The
GPU aspect of OpenSubdiv is really important, it drives the huge
difference in artist responsiveness but as stated earlier it is not a
miracle add-on or plugin. One can not just drop OpenSubdiv into a
standard pipeline and expect it to take already heavily subdivided geo
and render it auto-magically faster. The approach of OpenSubdiv is to
avoid getting the geo heavy in the first place, while still providing a
great close vision of the limit surface.

The
CPU version of OpenSubdiv was not previously fast enough to, for
example, display Woody’s face, which is a subdivision surface – as are
all Pixar characters. This meant animators who are trying to produce
very subtle and moving performances – with comedy and real nuance –
could not see exactly what they were getting. “You are looking at a wire
frame cage and you are kind of guessing you have the right amount of
emotion when he is smiling or winking or twitching his nose,” explains
Polson. “For example, you are kind of guessing you are in the right
ballpark – and that you have it – but you don’t really know you’ve got
it until you get the animation back (from the Renderfarm) the next day.
The fact that we could get all of that actually running – on the GPU –
means that now at the animator’s workstation they see the actual face –
the limit surface of the face. So when they make him wink or smile or
whatever – they see the final shape and they can judge. And they know it
is good. And when they get it back there are no surprises. That has
been the big win for us on our current films in production.”

4. Hierarchical modeling

A little-mentioned aspect of OpenSubdiv but one that holds enormous
promise is hierarchical model edits. The reason it is not discussed much
is that no major software package yet provides direct tools to
manipulate models fully in this way. However, there is no reason to not
think that once the various contributors to OpenSubdiv implement this
new approach it will not be very influential. The best way to understand
hierarchical edits is to look at an internal example from the Pixar
team, using a test scene from Brave.

Hoof prints are only located here – note: click for a larger version on any of the images in the story.

The hierarchical edit allows level based specific shaders, creasing, weights and offsets in local areas.

---

Few know that the film was originally intended to be set in the
Scottish winter. As such, the team realized that they would need to
handle their characters walking in the snow. The Pixar team did a test
of this using the hierarchical editing to create the effect of a horse
character riding in soft snow. Clearly with large expanses of snow it
would be idea to not have to subdivide the whole hill side for just
localized hoof prints and disturbances. “Normally when you are building a
model you sort of have to build the mesh density fine enough to handle
the finest feature,” says Polson. “You are in a bad spot if that fine
feature is small compared to the rest of the model, because you have
these tiny meshes but you have to span the rest of the model. In our
case it was a hoof print but we needed the mesh to cover an entire hill
side.

---

Download Video

---

A test from Brave – using hierarchical edits.

---

Hierarchical edits is the solution. Subdiv is a progressive
refinement that produces a successively finer mesh at every level but
hierarchical edits allow for the model to refine down to say level 4 and
then make these changes, move these points in this way etc, and then
continue with the process.

Even Pixar does not use this feature very much. They have used it in
an effects sense, with simulation work, but really Pixar would like to
see more tools using this feature so they do have ready made tools to
take advantage of the hierarchical edits, and this is coming (see
Houdini below).

It might be reasonable to ask, why do I need hierarchy when I can use
vector displacement? Certainly the question is one Pixar has asked
itself.  ”Look again at the snow field example,” says Polson.
“Displacement mapping effectively means moving the very small polygons
you get after subdividing many times, say 6 levels or more. So they have
to move a long way from the undisplaced limit surface, and this
typically leads to stretching and shearing and other artifacts. But with
hierarchical edits we can add some of the displacement after level one
subdivision, some more after level two, and so forth, so that each
level’s intermediate surface is artifact free. The resulting
displacement at level 6 is much more convincing and high-quality than
we’d get by subdividing down to microploys and then trying to move
everything at that level.”

While hierarchical edits seem powerful and a possible huge benefit,
Maya has in fact had hierarchical edits for a few years and they have
recently removed them as no one was apparently using them. It is unclear
if this was due to the current complexity or limitations on rendering
them or if users just don’t want them. They are also not free from a
computational point of view. In terms of rendering and memory,
hierarchical edits can slow down in certain situations, and perhaps
vector displacement maps provide enough power in this area already.
Either way Pixar has included them to see if they will be adopted and
tools will be developed to use them. Even inside Pixar they are not used
all the time and they remain a ‘jury is still out’ aspect of
OpenSubdiv.

---

Open source – who’s using it?

A host of companies are adopting OpenSubdiv, especially with the official release of version 2.0.

Maya, Autodesk

Pixar has worked closely with Autodesk and the Maya team for some
time. Not long ago, Bill Polson and his Pixar team approached the Maya
team to show them their GPU implementation inside of just ‘vanilla’
Maya. The idea was to gauge response from the Maya team and to show a
test, so they just “hijacked the draw path to the GPU,” jokes Polson.
The Maya team were impressed. “They said yeah that looks amazing – we
want that in Maya and not just in a plugin that only Pixar has – we want
it in core Maya, we want it available at all times not just preview
mode,” Polson adds.

It is complex to add a new draw pipeline into an existing product
such as Maya. The work has been challenging and the combined teams have
been exploring solving this now for over 12 months at both Pixar’s
Emeryville location and Autodesk’s Maya Montreal development offices.

Gordon Bradley is heading up Autodesk’s effort and is a part of the
OpenSubdiv group: “Autodesk is really excited about the OpenSubdiv
technology, and we’ve been working closely with Pixar on it for a while.
Having an industry standard for interactive and final frame rendering
of subdivision surfaces not only solves a lot of challenges we’ve
struggled with for years, but it gives our industry a standard platform
on which to build new tools, workflows and content – so we’re really
happy to be playing an active part in helping establish the technology.”

While the company cannot commit to dates as yet, this year at the
Autodesk user group in Anaheim during SIGGRAPH, Pixar and Autodesk
showed a custom-built prototype of how OpenSubdiv technology could be
natively integrated into Maya. The prototype let a user create a
standard Maya mesh, hit the ‘3’ key to smooth it, and instantly one was
able to see and work with an OpenSubdiv powered surface natively in
Maya’s viewport, using all the standard tools and workflows. This
provided a more accurate representation of the subdivision limit surface
in Maya than anything that an artist would normally see, let alone
interact with, before in the application.

---

“There are some incredibly talented and dedicated developers working
on OpenSubdiv – both in terms of the underlying maths and the
implementation on various architectures. It’s a real pleasure to get to
work with them,” says Bradley. “The API itself is still evolving – but
the core code is really clean, principled and fast. Another thing
that’s been really impressive about the OpenSubdiv project is the whole
ecosystem that’s been pulled together by Pixar to get to this point.
It’s not just the authoring tools; it’s the other studios, the
renderers, the game engine guys, the graphics APIs, the hardware
vendors. There’s a lot of collaboration and support there.”

fxguide put some detailed questions to Gordon Bradley about how Autodesk will work with OpenSubdiv and where it might be used.

FXG: Maya modelers can already sub-divide can’t they – why is this new?

GB: There are two new things here.

Firstly, Pixar has very generously donated its subdivision IP and
libraries to the community. So now we’ve got the potential to literally
have the same code powering the authoring tools, like Maya, and the
renderers. So, once everyone has had a chance to adopt it, you should be
able to view something in Maya’s viewport, hit render in any renderer
supporting OpenSubdiv’s and see the same thing in the final frame – or
export it to another application/cache, and still see the same thing.
Right now, that’s not the case: we’ve all got our own implementations
with varying feature sets and parameter semantics. So while at a glance
they might look similar, more often than not there are a lot of
differences, which can lead to unpredictable results. We saw this when
we compared the native Maya subdivision with the OpenSubdiv in the Maya
prototype we put together with Pixar.

Secondly, this is the first time we’ve had access to on-GPU
tessellation of the limit surface. This means we can now have the
potential to work interactively with a representation of the surface in
the Maya viewport that would not have even loaded in before. And it’s
not just a really highly subdivided surface – it also means we get final
frame compliant animation and displacement too. Combined, this would
let you see and work with a much better representation of the mesh,
interactively, and with greater confidence than what you’re seeing in
the viewport with will match what you render.

FXG: Why should a Maya artist reading this care about OpenSubdiv?

GB: You’re going to be able to work with a much
higher-fidelity representation of the mesh, it’s going to be faster,
smaller (so you can have more characters/objects in your scene), and
what you’re seeing in the viewport will match what gets rendered at the
end. It’s WYSIWYG for subdivision surfaces.

FXG: Can you comment on the benefits in version 2 such as the limit surface evaluation for collision detection?

GB: Being able to see your geometry in the viewport
and final frame render are obviously the first things you think about
when you think about geometry. But there are a lot of other things we
need to do with our objects: from simple things like snapping, to
dynamics and collisions, to complex ray tracing, growing things on them,
and procedural FX. That’s where v2 comes in – it gives us the APIs we
need for these other features. And it’s really important that they’re
part of the same library – this makes sure that when you snap to
something, or collide with it, or grow something on it – it matches the
surface you (and your audience) are seeing.

FXG: Will Autodesk be contributing code back into the open source community?

GB: We’ve been contributing code back to open source
projects for a while now! We have done so with both Alembic and
OpenEXR. These technologies are really transforming our pipelines and
our industry – and the teams of people setting these projects up and
making them available to the community are just awesome. Although we
cannot talk about our future plans we definitely want to be a part of
that where we can.

FXG: Is Autodesk exploring OpenSubdiv beyond Maya?

GB: We’ve certainly had a few early discussions
around how we could leverage this in products other than Maya, but since
Pixar use Maya the collaboration on the prototype has been Maya-based
and this has been our primary area of focus so far.

FXG: Can you comment on the power and usefulness of open standards ?

GB: There’s something really powerful about taking a
well-defined problem, and giving the community a solution they can not
only use, but also improve. And at some point, it becomes a solved
problem. As we hope will be the case with subdivision surfaces: if you
need the technology – there it is, it works, it’s well tested, it’s
optimized, it’s cross platform, and it’s supported in all these
products. You don’t have to spend the thousands of hours that Pixar (and
now the OpenSubdiv community) has spent researching, developing,
testing and optimizing this awesome technology. You just get to build on
top of it.

So yes, there are a lot of great benefits to something being an open
standard: it helps focus us as a community around one solution instead
of many; It lets us build an ecosystem of tools that support it; it
gives us interoperability; it gives us access to features we just
otherwise wouldn’t have; it avoids redundant repetition of work on
non-essential problems. Bit by bit we’re also moving our industry
forward – and letting the next generation of artists and films do even
more amazing and innovative things.

Above: see a Meet the Experts video with Pixar and Autodesk discussing OpenSubdiv.

---

Houdini, Side Effects

Exclusive: An early look at the new OpenSubdiv in a prototype form in Houdini.

Mark Elendt, Side Effects Software’s senior mathematician, is very
excited by OpenSubdiv. He sees it as a great way to
have interoperability between various software packages. There are at
least three places that Side Effects Software are looking forward to integrating OpenSubdiv in Houdini. “Of course, each task has its own unique challenges,” notes Elendt. He sets these out below:

1) As a modeling tool. This is likely the easiest place for us to
start using OpenSubdiv. Preliminary results look very promising.

2) OpenSubdiv has some very clever code for rendering
subdivision surfaces on the GPU. This will likely take a little
more effort than the modelling tools, but can provide a huge benefit to
our users, to be able to visualize the refined surface in real-time will
be a huge win.

3) The other obvious place to incorporate OpenSubdiv is in
our software renderer Mantra. This is likely the most difficult of the
integration tasks. Mantra uses specialized rendering algorithms which
currently perform refinement in a way that doesn’t seem to have support
in OpenSubdiv. There are a couple of ways we could try to solve this,
but it isn’t clear to us which would be best at the moment.

Elendt is also very excited to see in the latest version 2.0 release
that the limit surface evaluation is built in. “Because OpenSubdiv
provides a standard implementation, this means that tools like fur
generation and particle effects can now provide results that match
exactly between Houdini and other software like PRman or Maya,” he
explains. With the limit surface evaluation tool or command it is
possible to make sure intersections are correct, that water correctly
follows the form of a face, or that cloth correctly intersects and does
not allow breakthroughs – something that could happen with release 1.0
which did not have the ability to collision detect with the limit
surface.

Interestingly, Side Effects had also entered into agreements with
Pixar related to this before Subdiv was open source. For example,
Houdini has two styles of creasing in their modeling tool. One version
is licensed from Pixar and allows an artist to produce creases that
would render correctly in RenderMan but would not render in Side
Effects’ own Mantra renderer. Mantra is not RenderMan compliant – but it
is very similar to RenderMan. To render in Mantra a user needed to use
Side Effects’ own version of creasing that was not in violation of any
of Pixar’s patents. “We provide both options in the modeler and we offer
the non-patent infringing one inside the Mantra render, so it is very
exciting for us for us to use OpenSubdiv to provide the Pixar Subdiv
inside our renderer,” says Elendt.

Actually Side Effects was not alone, AIR renderer also had the same
approach and had a different creasing model that was outside of Pixar’s
patents. “This is one of the big things that OpenSubdiv solves, now
everyone can produce the same creasing and so the interoperability
between software packages is greatly improved,” adds Elendt. Now with
OpenSubdiv Houdini will be able to have one version that is allowed to
be rendered inside Mantra and RenderMan.

The other interesting aspect is the hierarchical edits. Houdini has
had a version of this for many years, but the OpenSubdiv hierarchical
edits are a super-set of what Houdini currently allows. For example,
Houdini could apply a different shader per face of a sub division
surface, “and the only way to output that to a RIB file is with a
hierarchical edit,” Elendt explains. But with OpenSubdiv you can define
for different creases and/or weight level at several different levels on
different parts of the master model. This additional flexibility and
power is something that Side Effects very much wants to investigate for
Houdini. “The hierarchical edits are definitely a very interesting
technology and we definitely want to use that in the future,” says
Elendy, adding temptingly, “There may be other ways for us to use that
technology but it will be in Houdini 13 and beyond.”

While many people think of Houdini as an effects animation tool, due
to its industry defining destruction, volumetric and fluid pipelines, it
is of course a complete modeling, design and animation package, with
years of great early adoption of technology like OpenSubdiv, OpenVDB and
earlier technologies like Ptex – all of which Side Effects adopted if
not first then very soon after their introduction, which is why advanced
modeling tools are so relevant to a Houdini pipeline.

Side Effects’ research into more fully embracing OpenSubdiv is
already producing results that could benefit the whole community. Ever
keen to contribute as well as use code in the open source community,
Side Effects hopes to give back some of the interesting enhancements
they have been working on. For example, they discovered that the Linux
implementation is really fast but the Windows implementation has some
speed memory issues. Linux was “blazingly fast and really tuned to work
with large data sets,” says Elendt. “Windows was three or four – at
times ten – times slower. It turns out that there is one little function
which is doing a lot of memory allocation and on Linux we are using
this Firefox library which cut down the overhead of lots of little
memory allocations. so we don’t notice that cost on Linux. But on
Windows we saw the big cost, so right now we are trying to find a
solution to that. And we will try and feed that back into the OpenSubdiv
project and that’s the great thing about open source – when we find a
patch we can feed that back into the community, and this would probably
affect the CPU and GPU implementations.”

Design and Modeling (Modo, The Foundry)

Pixar Subdiv (PSUB) being used inside The Foundry’s Modo for industrial design. Credit: Harald Belker Design

One of the issues faced by industrial designers is producing designs
that can be actually made. In design terms there are some key concepts
that need to be respected to make real world designs. One is being able
to produce designs which are ‘water tight’. The other is make sure that
any model is ‘manifold’. If you look up manifold in google you may find a
definition which says unhelpfully that “in mathematics, a manifold is a
topological space that near each point resembles Euclidean space.” A
simpler way to think of it is that each surface has an inside and
outside – so a mobius is a great example of a non-manifold shape. A
mobius is a surface with only one side and only one boundary component.

Or in simple terms - If an ant were to crawl along the length of a
mobius strip, it would return to its starting point having walked the
entire length of the strip (on both sides of the original paper) without
ever crossing an edge. Designers cannot work in the real world with
non-manifold models if they are headed to manufacturing. Actually, it is
not just a design / 3D printer issue. RenderMan cannot render any
non-manifold 3D object, this does actually affect the broader OpenSubdiv
story outside Modo as RenderMan and thus OpenSubdiv “just does not want
to deal with any faces of points that are not manifold. The code that
is in OpenSubdiv is from RenderMan’s core subdivision library,” explains
Pixar’s Manuel Kraemer.

Modo PSUB editing. Credit: Harald Belker Design

As designers need these more engineering specific models they tend to
work with NURBS, whereas a film designer may have a preference for
straight polygons, which can easily be made to produce an impossible
model from an manufacturing point of view. The problem with NURBS is
that they can be much more complex than polygons. For example, the
problem of matching arbitrary topology by connecting two NURBS patches
can lead to cracks. This is not only a problem for designers but such
cracks on a character’s face for example could be disastrous for a
character pipeline as they can break under animation and allow literally
holes in your character’s head.

By comparison, arbitrary topology with OpenSubdiv is easy and
produces a clean limit surface, which is robust under animation –
something the Pixar modeler relied on in say the modeling of the face of
the mother bear in Brave. It is also exactly what industrial
designers use in MODO when using the PSUB modeling approach. (PSUB is
short for Pixar SubDiv and has been in MODO since MODO 501). In MODO the
implementation is not strictly “Open Subdiv”, as the Luxology team lead
the adoption of Pixar’s approach, under license, before OpenSubdiv was
officially even launched, but the code approach they have taken is
similar. MODO designers are able to easily and yet confidently produce
industrial design knowing the designs will not only be able to be
manufactured but also match their on screen originals. Open Subdiv
wonderfully bridges these two extremes while having really the best of
both. Like NURBS you can produce water tight, manifold designs, and yet
like polygons, OpenSubdiv is easy to work with, fast and user friendly.

Download Video

---

Artibary subdivision of any topology avoids NURBS cracks or breaks which can be worse with animation.

Brad Peebler of The Foundry (ex Luxology) commented, “we are super
positive about OpenSubdiv because it means other apps can feed from our
PSUB (Pixar SubD) meshes. That’s great! Our PSUB work was done with
Pixar prior to the release of OpenSubdiv and it means that we are
OpenSubdiv mesh compatible with the extra advantage that having the
algorithms in the core of our meshing tools means we can also make
topology changes very very fast. We are 100% supportive of the effort
and continue to stay in close communication.”

OpenSubdiv also makes a lot of sense for another Foundry product,
MARI, which is now actively exploring it. “OpenSubdiv for MARI makes a
tremendous amount of sense because there is no topological changes
happening inside the MARI workflow,” says Peebler.
“OpenSubdiv represents a very fast and efficient method for displaying
SDS meshes and as you can imagine that would be very valuable to an app
like MARI. In the UK, Jack Greasley (MARI original developer and Product
Manager) and the MARI team are currently focused on the OSX version of
the software, but officially they can say this: “The guys will be
starting work on it as soon as the Mac version has shipped. Aiming for
early next year,” according to The Foundry’s Ian Hall.

---

Mobile solutions (DigitalFish & Motorola)

DigitalFish is a group originally from inside Pixar, who have set up
their own company and amongst other projects help implement OpenSubdiv
pipelines and solutions. They help their clients solve difficult
graphics and artist-related technical problems. Just one example of this
is DigitalFish embedding engineers within their clients’ teams to
support their goals integrating OpenSubdiv. Such an example was shown at
this year’s SIGGRAPH conference built around a new phone from Motorola
Mobility. As OpenSubdiv provides great performance in the interactive
display of smooth surfaces for characters, vehicles and other
scene content, DigitalFish helped delivered the first-ever realtime
subdivision surfaces on a mobile device. DigitalFish engineers helped
their Motorola clients build software in a very tough framework. The
team had just 7 millisecs refresh, it had to work with time budgets,
power budgets, and of course a memory budget. For DigitalFish much of
their work involves so-called ”GPGPU” programming using GPU-centric
interfaces such as OpenCL or CUDA to do general programming on the
graphics processing unit.

Screen Grab from the Windy Day App. Directed by Jan Pinkava

The result of the effort allowed for Motorola to collaborate with
Oscar-winning animator Jan Pinkava and team of animators, modelers, and
sound experts. “Storytelling is foundational to our history, our
teachings and our relationships. We tell stories to our children and one
another. They bind us together. So it’s exciting for us to start
exploring what happens when we make stories for perhaps the most
personal devices we own – our smartphones,” said Regina Dugan, senior
V.P of ATAP, a division of Motorola Mobility in Computer Graphics World
in July.

DigitalFish now also have a CPU version, which the company says is
optimized for ARM processors and that will be contributed as part of the
Motorola branch. The CPU version runs very close to, and the company
says, in some cases faster than, GPU version on mobile. It enables
uniform subdivision only.

Baback Elmieh

“We’re investigating how a mobile device can permit stories to be
told in a more immersive, interactive way. This is not a flat content
experience or the viewing of stories made for the big screen and shown
on a small one,” explains Baback Elmieh, technical program lead in the
Advanced Technology and Projects (ATAP) group of Motorola Mobility.

Left: Baback Elmieh of Motorola Mobility can be seen demoing and discusses the Windy Day app during the 2013 SIGGRAPH course

---

SIGGRAPH 2013 Open SubDiv course. Nathan Litke (DigitalFish) and Pixar’s Dirk Van Gelder (right)

With so much of the creative opportunity around OpenSubdiv involving
new modeling approaches, it is worth noting that the SIGGRAPH demo did
not include hardware that has built in tessellation. As such the team
could only go to level 2. With new graphics hardware coming with
hardware tessellation, the team could take this much further in the
future with newer hardware, and produce even more responsive and higher
quality visuals.

Bill Polson believes having an independent company like DigitalFish
familiar in the code base being available to help companies adopt
OpenSubdiv is a great benefit. He points to key members of the team like
Nathan Litke, Senior Research Scientist. “Nathan’s particularly strong,
he is a PhD in computational geometry and his PhD thesis was on
Subdivison, and he has production experience,” points out Polson, who
also singles out DigitalFish co-founder Dan Herman.

DigitalFish’s Dan Herman and (left) and Geri’s Game & Windy Day director Jan Pinkava having fun at SIGGRAPH 2013.

“Geri’s Game was being made across the hall from my office,”
says Herman. “When I was at Pixar, that was my first exposure, and that
was 1998. At that time Subdiv was an experiment. “Toy Story had been modeled primarily with NURBS, Woody and Buzz were NURBS and NURBS are not good for organic surfaces. From Geri’s Game in 1998 to Toy Story 2
(2000) Pixar completely converted from a NURBS pipeline with polygons
to a totally Subdiv pipeline. That conversion was because the advantages
of Subdiv were just so compelling.”

---

At Qualcomm UPLINQ 2013 in Sept. Dr Regina Dugan SVP of Motorola Mobility demoed Windy Day. (link)

Download Video

---

---

The whole process of having consultants available is also key as
implementing an OpenSubdiv pipeline is not as simple just just
downloading the code from GitHub and compiling it. Until companies like
Autodesk and Side Effects include the tools as standard, there is still
some real engineering to be done to adopt a pipeline that would mirror
or resemble the one at Pixar. “It is a reasonably complex package with a
lot of dependencies,” notes Kraemer.

In Herman’s opinion, Subdiv is:

• easy to model
• easy to refine if you need to edit
• easy to animate
• very light weight in the pipeline
• good to render

The only downside is modeler
education, and work is needed to help explain them and to help implement
it. “We see the conversion that happened at Pixar primed to happen at
other places all around the entertainment industry,” concludes Herman.