Everyday 3D

A few months ago, I had the pleasure to work with Luciano Foglia and Anrick Bregman on a visual experiment called Format Journey. The installation uses data – a series of images and a sound file – to modify the texture and the shape of a 3D object. Combined with a slight randomization it gives very nice and unpredictable results.

When we first talked about the project, the idea was to use either Flash or Processing, but I though that it’s a good occasion to try Unity3D (of course… :) To achieve the desired visual effect I needed a shader system that’s fast, flexible that and works with 3D. Unity’s ShaderLab seemed like a good option.

The material on the shape is transparent and is composed of a blurred version of an image applied as regular texture and the image itself applied as reflection. The images change over time, and are animated by scrolling UVs.

All the data for the installation is loaded dynamically, the application itself is just composed of some scripts that make it run, so the file is only 55KB! It’s controlled by an external configuration file that allows to define the source images and sounds. There are also quite a few other settings to tweak they way it works, ex. how transparent or how reflective the material should be.

One particular challenge was to make the shape react to the sound. To make this happen I needed to read the spectrum and Unity3D doesn’t offer access to sound data on the code level. One possible solution was to write a plugin that does that, the downside however is that it wouldn’t run in the web player.

I solved the problem by pre-processing the sound in a small Flash app. It dumps the spectrum data to a text file which is then loaded to Unity3D. When the application plays the sound file, it uses this data to check the values of the spectrum at any position in time. Simple, but does the job.

The installation is part of a project called Tango and Hawaii created by Anrick and Luciano. Be sure to check the live demo. It was originally presented at the Magdalena Festival in Maribor, Slovenia.

It was a new experience for me to work on an art piece was. It was very inspiring to see what Anrick and Luciano did with a tool I helped them to create. Thank you guys!

This post is an addition to the last series I wrote about dynamic content loading in Unity3D.

When building a dynamic application, every now and then you might need to load some 3d models at runtime. Either because there's a database with models you want to use or just because you want someone to update the models without rebuilding the project. I actually had this situation a few weeks ago. Surprisingly, this option does not come with Unity out of the box.

Asset bundles and resource folders

But first thing first. I said there's no ready-to-use way to load 3d models at runtime, but it's not 100% true. In fact, there are two options: asset bundles and resource folders. These methods are discussed in depth here.

They can both be useful in some cases, but they both have one fundamental drawback: in order to create them, someone needs to open the project in Unity, import the 3d models, create the bundles or resource folders and then export the whole thing again. To create an AssetBundle you need to run a script in the editor, while the Resource Folders... honestly, I failed to even make them work!

In brief, it's all too complicated for what I was looking for.

A straightforward solution

What I needed and wanted was easy and simple: to load a 3d object (geometry + some materials and textures) at runtime, using just a URL as parameter. The only solution seemed to write my own importer.

Basically, I had the choice between Collada and Wavefornt OBJ. I would choose OBJ any time of the day because it's a simple, concise plain-text format, while Collada is bloated and is XML-based.

It's not that I hate XML (although I'm not a big fan either) but in order to parse XML you need to include a pretty weighty DLL in your *.unity3d file, around 850Kb, which in this case (and in many others) defeats the purpose. Still, it's good to know that it possible, and there are situations when it's ok to use it. If you want to learn more about Unity3D and XML there's a awesome article on this topic by Paul Tondeur.

It turned out that while it's not rocket science to write an OBJ importer, it's not exactly banal either. I spent a few days coding it so I thought I'll share this with everyone - maybe someone will make good use of it.

Here's a package with the source code (v1.1), along with a simple scene demonstrating how it works. In fact it couldn't be simpler. All you have to do is to create an instance of the OBJ class and start a coroutine to load the contents, like this:

Supported features

• Vertex, normals and UV data
• Multiple objects per file
• Vertex groups and multiple materials per object
• MTL files, diffuse and specular materials
• Basic textures

And it's all at a cost of ~12Kb extra added to your final file!

Testing and the universality of OBJ format

In the 3D world the OBJ format is nearly universal - I think that every 3D editor in existence is able to export to this format. This also means that Wavefront files come in many different flavors.

I tested the code against models created with Blender and against common sense assumptions on how an OBJ file can be constructed. If it doesn't work with files exported from your editor send me the OBJ file, and if possible I'll update the code. It's released under the MIT license so feel free to use it in your projects, commercial or not.

That's it! I hope you'll find it useful!

The conference season is upon us!

Following my last year short presentation at Flash On The Beach and at the Warsaw Flash Camp, this year I will continue my adventure with public speaking. In the next months I will be having my sessions at 3 highly interesting events.

1. FITC Amsterdam, February 22nd-23rd

FITC Amsterdam is only a few weeks away. I've been to FITC last year and I really enjoyed the conference (and the parties too). I am thrilled to be back this year as a speaker. If you want to catch my session it's on Monday (Feb 22nd) at 12:30. If you are interested in Unity and in 3D in general you should definitely attend! For more information, here's a detailed session description. I hope to see you there!

FITC it's of course much more than my presentation. There are 2 days packed with interesting sessions, so be sure to checkout the schedule to see what's interesting for you. There is also a party every evening - a great opportunity to do some networking and to have a few beers and some other stuff (remember, we're in Amsterdam!). So grab your tickets before they're all gone!

2. London LFPUG Meeting, March 25th

If you can't attend FITC and you miss my session in Amsterdam, don't panic! You can catch me again next month in London. On March 25th I will speak at the next in the series of LFPUG meetings.

It will be a evening packed with realtime 3D, as I will be speaking together with Rob Bateman from the Away3D team. Here's some detailed info about the event. The event is free, all you need to do is register, so if you are in London don't miss it!

3. FITC Toronto, April 25th-27th

Finally in April I'll be making the move over the Atlantic all the way to Canada for FITC Toronto, where I will also be talking about Unity. There is no schedule yet, but it's should be coming soon. However, the tickets are already on sale. If you hurry you might even get a early bird price!

This is very exciting for me for several reasons. First, it's a major conference with 3 days of sessions, workshops and even a recruiting event. It's not to be missed not only if you live in Toronto, but for anyone in the region. And I've heard people come for FITC from all over Canada and US.

Second of all it's my first time I will attend a conference in North America. I looking forward to meet a lot of people whom so far I knew only from blogs and twitter. So if you are around don't hesitate to get in touch!

Finally, it will be my first visit to Toronto and I heard a lot of good stuff about the city. We also plan a short family vacation afterwards to visit Montreal and Quebec Miami and the Florida Keys.

Next?

I'm currently entirely focused on those upcoming events, preparing all the materials and the presentation. However, I hope for more opportunities in the future. If you happen to organize a Flash and/or Unity event and you are looking for speakers be sure to let me know!

I don't have any further plans at this moment, except that I will also attend OFFF in Paris in June (as guest, not as speaker).

Photo credits [1] [2] [3]

I am proud to announce my latest project. It's a 3D racing game done in Flash with Away3D. I worked on this project together with Calisto Labs and our client was Snowball Media. My task was to create the 3D game part. It was great fun to work on this project. The team at Calisto Labs did a awesome job and everything went smoothly!

The project required to use my Actionscript and 3D modeling skills together on a scale like I never did before. I spent a lot of time with Blender and Unity in the last months, and without this experience I feel the project would be beyond my abilities. Also, having a direct comparison between Flash and Unity allowed be to see things in a larger perspective.

Flash vs. Unity... again

I built the game with Away3D FP10. I must say that Flash 3D has gone a long way since I first started to play with it, and Away3D is a very solid engine. The 2K polygon limitation is a thing of the past, and the correct perspective projection in FP10 makes everything look so much better. The team also added some other crucial features to create 3D environments - ex. frustum clipping.

Before starting this project I was working on a Unity3D racing demo, so I managed to get quite a few ideas on how to build such a thing. However, when I moved back to Flash, I was a bit desperate because I realized how much stuff I got used to in Unity is just not there in Flash. A built-in physics engine would be the most important one.

Home-made physics and collision detection

For physics in Flash I was tempted do use JigLib at first. But soon I found out that JigLib is way too complex, creating a realistic car behavior would be very difficult and it would probably eat too many CPU cycles. I estimated that Away3D will take around 90-95% of available processing power, so there's no space for any other complex piece of code to run in the same time. I always think that when it comes to 3D in Flash performance is quality, and there is no excuse for poor performance, even if you want something to look "realistic". So I ended building something very very simple.

Once I got the car driving around, I needed to add some collision detection. In Unity you quickly get used to the fact that any 3d object is a collider which makes collision detection a no-brainer - just implement a listener method and get a call whenever your object collides with another one. And if you need to add realistic collision response, you just make the object a rigid body and there you go! (Ok, you may need to adjust some settings, but still it's pretty straightforward).

In Flash it seemed like a much more complex task. I was thinking in the lines of creating a system based on the geometry of the track. I started to study curve equations just to realize I wouldn't make it even if I had a year to complete the project! Then I remembered, that a good way to test collision with complex shapes is a bitmap.

The way it works is that it takes the car's position and checks against a map of the track that has different colors for different areas - ex. red for the road, green for the sideways and so on... Since it samples only a few pixels per frame and all it does is check their RGB values, it's lighting fast. Truly, Flash is the art of minimal!

Scenery

The most fun part was to create the scenery. I've never tried to import such a large 3D scene into Flash so I wasn't even sure if Away3D would handle it. It turns out it did handle it pretty well. Instead of using Collada I went with Wavefront (OBJ) format for the meshes as I feel it gave me more flexibility. OBJ is a simple format which makes it a bit easier to see what's going on while Collada is bloated and overly complex for my taste. I used Blender to model the track and the few objects that you can see around it - lampposts, houses and the tunnel.

Lightning

Lightning is very important in low poly scenes, because it adds a lot of detail and atmosphere to the scenery, without adding any polygons. Even thought the scenery is simple, I ended up having more that 60 lamps. Of course all those lights needs to be baked on the textures, there's no way to run them in real-time. Blender has a very good texture baking tool and all this works fine for static objects. But what about objects that move, like the car?

For lightning the car, I reused the concept from collision testing. But instead of a collision map I made another one - a light map. Basically it's a bitmap with white areas where the scenery is lit and dark ones where it's not. By sampling a single pixel on every frame I can see if the car is passing through a lit area. Then, all I have to do is to apply a color transform on the car texture to make it brighter. Again - a minimal solution, but it works!

Now that you know all about it, go ahead and try it out!

Ringo from FlashBookmarks asked me if there were modifiers similar to AS3Dmod in Unity. I searched for something similar some time ago, but didn't find anything interesting.

However, when I was starting C# scripting, I ported two of the modifiers from AS3Dmod - Bend and Twist. So I thought to share them with everyone. Don't expect much, it's not the full library, just two classes. If I have some free time, I'll look into how to implement stacking, since for the moment you cannot apply two modifiers to the same object (the second one won't have any effect).

To play around with them grab this package and import it into Unity. Keep in mind that the modifiers work at runtime, so you won't see any effects in the editor until you run the game. If you want to animate them, just add another script that will have a reference to the modifier instance and change it's properties at each Update() call.

If you are interested in this kind of "bricolage" with 3D geometry, I recommend to take a look at the Procedural Examples project provided by Unity. You will find there some highly interesting samples like dynamic extrudes or perlin noise. It's a great starting point to explore runtime geometry transformation.

Of course, take a look into the sources of the modifiers too! You'll see how simple it is to access the geometry at runtime and do some modifications. Basically it goes like this:

Things worth notice: In Unity a vertex is just an instance of Vector3 - there isn't any special class for this, as in Flash-based 3D engines.

You can see a bit of the casting-drama I need to do (lines 1-2) to get to the object holding the actual geometry information (Mesh). That's a bit annoying in C#, with JS that code would be more readable. Anyway, it's better to make that only once, in Start() and not at every Update().

To get correct lightning effects on the materials, do not forget to call mesh.RecalculateNormals() after you've modified the positions of the vertices.

If you want to modify your mesh continuously (i.e. to animate it) it's necessary to keep the original array of vertices, because the code above overwrites the original positions. Take a look at the source code in the package to see how I did it.

That's all for now, hope you enjoy it!

UPDATE Nov 2009 There is a better quality video posted by John from Flash On The Beach. It is available here: http://vimeo.com/7292505.

As I promised during my "3 minutes" in Brighton, I publish all the sources of my presentation. I added some comments in the code and removed the part with the slides.

Here's also the demo and a video of the mini-session. The demo is pretty rudimentary - I made so because I wanted to keep things simple during the presentation. I hope that it will be a solid base for someone who wish to create a full featured bowling game in Flash.

There were a few other implementations of a bowling game in JigLibFlash and Papervision3D. Initially, guys a Blitz Agency published a few interesting experiments with JigLibFlash, including a simple bowling simulation. Devon O. Wolfgang has written a great tutorial about building such a game on Tech Labs - be sure to check it out. I found it only after the presentation and since the tutorial explains a lot of things in details I felt like I was reinventing the wheel here. But hopefully there are still a couple of things I can add.

Thanks to the new plugin API we developped some time ago, setting up a scene with JigLib and an 3D engine has got a bit less complicated. However, tweaking the engine can be a hell. Here's a few things I found out:

1. Simulation speed

When you create the physics engine instance, the default speed of the simulation is 1. This is very slow and unrealistic. It will look much more natural if the speed is increased. Beware however - at higher speeds the collision detection system can be very inaccurate and result in objects running through each other without any collision being detected. I don't think there is a single setting that works fine in every situation, but for this case 9 worked fine for the regular speed simulation, and 2 for the "slow speed". You should always try different settings.

2. Mass

Each rigid body has a mass property. It's easy to forget about it since it has a default value and it never complains if you don't change it. However, setting the masses right is crucial for a realistic simulation. In real world object have different masses, and so they should in a simulation. This is particularly important for a bowling game, where the ball is pretty heavy and the bins are not (I guess... has anyone ever had a bowling pin in his hands?) The trick is that the masses are relative to each other, so the more different objects you have the more you need to tweak the masses to get the results right. Also remember the effect of any forces applied to an object is related to it's mass.

3. Physics material

Each rigid body has a property called material which is an instance of the MaterialProperties class. It has two properties: friction and restitution. I found out that playing with this values has a quite big impact on the simulation. Ex. setting a high value of the restitution results in the object becoming bouncy - I used this for the ball in my older ping-pong example. In the bowling demo I used lower friction on the ball to make it slide more - just as a real bowling ball does.

4. Object rotation

Once a DisplayObject is wrapped into a physics rigid body you can't rely on it's rotationX, rotationY and rotationZ propeties anymore - because they are not being set. The physics engine sets the transformation matrix directly on the DisplayObject, so if you need to check it's rotation you need to extract it from the matrix. Fortunately, there's an easy way to do this:

5. Object activity

As a result of multiple forces being applied to an object and multiple collisions sometimes the objects are left in a state where the shake a bit endlessly. This can also happen when the objects are initially positioned. Calling the RigidBody.setInactive() will fix that. And you can call more than once.

Important conclusion: tweak, tweak, tweak...

It is generally agreed that hardcoded ("magic") numbers are not a good coding practice. However in 3D animations, and especially with physics the important thing is not the beauty of the code, but the what you see at the end. If you browse the source code from this demo you will notice that I not only hardcoded a lot of values, I even left ugly lines like that:

In fact this is what I like the most in doing all those demos and experiments - the moment when code stops being just a list of instructions for the machine and becomes an art of making things look good by adding little tweaks here and there. If you tend to write very clean code and use all possible standards and conventions, from time to time make it ugly... you'll see how good it feels :)

I hope that this few tips will help you with your next JigLib project!

Last but not least, I'd like to say thanks to everyone who woke up early to see the Elevator Pitch session. It was a great experience being there and talking to you. The Brighton Dome packed with people can be intimidating and 3 minutes is not much time, so there was no place for mistakes. Fortunately, the FOTB technical crew made it all seamless. Great job guys!

Finally, I would like to give a special thanks to John Davey for inviting me to Flash On The Beach and for making this great conference happen!

Hope to see you next year!