Everyday 3D

Creative use of technology // A blog about 3D Flash and Actionscript by Bartek Drozdz

Old school effects with shaders


Plasma


Rings


Canyon


Flower


The endless well

What are these?

While at FITC San Francisco I attended the session of Iñigo Quilez who is a demoscene veteran, currently working at Pixar. He showed us lots of interesting stuff he created with GLSL. He also showed examples of some more basic stuff like those above. These are very old school effects known to the demoscene artists since the dawn of civilization (that is the ’90s). On Iñigo’s site you can find a simple tutorial on how to create those effects, and it’s not very difficult to port it to Unity3D using Cg. The cool thing is that all this different effects run on the same exact code, changing just a few settings.

As for the plasma effect, I initially saw it on mrdoob’s blog and found this short tutorial which explains how to do it. Simple, and it looks nice!

TIP: Although they run pretty smoothly in the browser, the demos perform even smoother in fullscreen mode.

UPDATE Some people reported that it doesn’t work on their machines. If this is your case, please run the plasma demo and press SPACE to get some debug info. If possible send me a screenshot. Thanks a lot!

Format journey

A few months ago, I had the pleasure to work with Luciano Foglia and Anrick Bregman on a visual experiment called Format Journey. The installation uses data – a series of images and a sound file – to modify the texture and the shape of a 3D object. Combined with a slight randomization it gives very nice and unpredictable results.

When we first talked about the project, the idea was to use either Flash or Processing, but I though that it’s a good occasion to try Unity3D (of course… :) To achieve the desired visual effect I needed a shader system that’s fast, flexible that and works with 3D. Unity’s ShaderLab seemed like a good option.

The material on the shape is transparent and is composed of a blurred version of an image applied as regular texture and the image itself applied as reflection. The images change over time, and are animated by scrolling UVs.

All the data for the installation is loaded dynamically, the application itself is just composed of some scripts that make it run, so the file is only 55KB! It’s controlled by an external configuration file that allows to define the source images and sounds. There are also quite a few other settings to tweak they way it works, ex. how transparent or how reflective the material should be.

One particular challenge was to make the shape react to the sound. To make this happen I needed to read the spectrum and Unity3D doesn’t offer access to sound data on the code level. One possible solution was to write a plugin that does that, the downside however is that it wouldn’t run in the web player.

I solved the problem by pre-processing the sound in a small Flash app. It dumps the spectrum data to a text file which is then loaded to Unity3D. When the application plays the sound file, it uses this data to check the values of the spectrum at any position in time. Simple, but does the job.

The installation is part of a project called Tango and Hawaii created by Anrick and Luciano. Be sure to check the live demo. It was originally presented at the Magdalena Festival in Maribor, Slovenia.

It was a new experience for me to work on an art piece was. It was very inspiring to see what Anrick and Luciano did with a tool I helped them to create. Thank you guys!

What can you do with Unity3d? FITC San Francisco

I’m happy to announce that I will be speaking at FITC San Francisco in August. I have been speaking at FITC events twice this year, and both times it was a fantastic experience. I’m looking forward for this one, and what really thrills me about it, is the amazing speaker lineup!

There are of course all the usual suspects like Joa, Ralph, Andre, Mario and Seb plus many more great speakers form the community – you don’t want to miss any of them. Here’s a few that particularly caught my attention:

Eugene Zatepyakin The Actionscript/Alchemy mage from Russia. You may know Eugene from his work on ASSURF which is this amazing image recognition library. I had the pleasure to meet Eugene in Antwerp last year and I can assure you that you won’t regret attending his presentation.

Theo Watson At FITC Amsterdam 2009 Theo was speaking as a last minute replacement for another presenter who could not make it. It was a lucky coincidence for me, because I really loved what he was showing! Theo is a digital artist working with openFrameworks. In San Francisco he will be taking part in a panel together with Jer Thorp and Ben Fry, the creator of Processing. I’m sure it will be a very interesting discussion. Theo will also be giving a presentation on interactive storytelling toghether with Emily Gobeille.

Yoko Nakamura Yoko was doing kick-ass Flash stuff on Yugop while I was still learning how to use gotoAndPlay()! I can’t wait to see the legend live.

And there’s me. I feel almost insignificant in face of such great talents, but I will do my best to deliver a great presentation. I was speaking in Amsterdam and Toronto and I was happy to see a full house both times for which I only partly credit my amazing speaking talent… :) It’s rather that Unity3D is such a hot topic right now. In San Francisco I will be giving a new, updated presentation entitled “What can you do with Unity3d?”. I hope to see you there!

But wait, there’s more… so take a look at the complete speakers list. If you live in California (or close to) this is an event not to miss. The super early bird tickets are only available until today (Friday, May 28th) so go grab them before it’s too late!

Loading 3d models at runtime in Unity3d

This post is an addition to the last series I wrote about dynamic content loading in Unity3D.

When building a dynamic application, every now and then you might need to load some 3d models at runtime. Either because there's a database with models you want to use or just because you want someone to update the models without rebuilding the project. I actually had this situation a few weeks ago. Surprisingly, this option does not come with Unity out of the box.

Asset bundles and resource folders

But first thing first. I said there's no ready-to-use way to load 3d models at runtime, but it's not 100% true. In fact, there are two options: asset bundles and resource folders. These methods are discussed in depth here.

They can both be useful in some cases, but they both have one fundamental drawback: in order to create them, someone needs to open the project in Unity, import the 3d models, create the bundles or resource folders and then export the whole thing again. To create an AssetBundle you need to run a script in the editor, while the Resource Folders... honestly, I failed to even make them work!

In brief, it's all too complicated for what I was looking for.

A straightforward solution

What I needed and wanted was easy and simple: to load a 3d object (geometry + some materials and textures) at runtime, using just a URL as parameter. The only solution seemed to write my own importer.

Basically, I had the choice between Collada and Wavefornt OBJ. I would choose OBJ any time of the day because it's a simple, concise plain-text format, while Collada is bloated and is XML-based.

It's not that I hate XML (although I'm not a big fan either) but in order to parse XML you need to include a pretty weighty DLL in your *.unity3d file, around 850Kb, which in this case (and in many others) defeats the purpose. Still, it's good to know that it possible, and there are situations when it's ok to use it. If you want to learn more about Unity3D and XML there's a awesome article on this topic by Paul Tondeur.

It turned out that while it's not rocket science to write an OBJ importer, it's not exactly banal either. I spent a few days coding it so I thought I'll share this with everyone - maybe someone will make good use of it.

Here's a package with the source code (v1.1), along with a simple scene demonstrating how it works. In fact it couldn't be simpler. All you have to do is to create an instance of the OBJ class and start a coroutine to load the contents, like this:

C#:
  1. string path = "http://www.everyday3d/unity3d/obj/monkey.obj";
  2. OBJ obj = new OBJ();
  3. StartCoroutine(obj.Load(path));

Supported features

  • Vertex, normals and UV data
  • Multiple objects per file
  • Vertex groups and multiple materials per object
  • MTL files, diffuse and specular materials
  • Basic textures

And it's all at a cost of ~12Kb extra added to your final file!

Testing and the universality of OBJ format

In the 3D world the OBJ format is nearly universal - I think that every 3D editor in existence is able to export to this format. This also means that Wavefront files come in many different flavors.

I tested the code against models created with Blender and against common sense assumptions on how an OBJ file can be constructed. If it doesn't work with files exported from your editor send me the OBJ file, and if possible I'll update the code. It's released under the MIT license so feel free to use it in your projects, commercial or not.

That's it! I hope you'll find it useful!

Runtime objects in Unity3D

Unity3D can work with meshes created in external 3D editors, but geometry can also be created with code. And not only geometry, but materials and textures as well. To test all these options I created a simple cityscape. I got inspired by this set of awesome pictures and also by a project called Pixel City by Shamus Young.

Creating meshes

All information about the geometry of an object is stored in the Mesh object inside the MeshFilter component. A mesh is composed of an array of vertices grouped into triangles and with UV information added to it. It's a pretty common way to store information on 3d geometry, similar to what you can find in Actionscript - Graphics.drawTriangles expects arguments in the same data format. Many file formats out there store meshes that way too, ex. Wavefront OBJ (which makes writing importers/exporters relatively easy by the way).

Creating a mesh with code means populating the arrays of vertices, triangles and UVs. While it is pretty straightforward, even building a simple cube can be tricky and the code quickly becomes a unmanageable list of numbers. In the source code, the buildings are generated in the BuildingBuilder.cs file, so you can see it for yourselves.

Creating textures

We've already seen that any bitmap you import into a Unity3D project is represented by the class Texture2D. Of course, bitmaps don't have to be imported - they can be created from scratch. The constructor of the Texture2D class take a few arguments, including the width and height of the image.

Once a new texture is created all it's pixels are empty (i.e. black and transparent). Their values can be changed either on a pixel-per-pixel basis using the SetPixel method or by copying portions of other bitmaps using SetPixels.

In the above example I have 6 predefined textures with different types of windows, so I don't create those from scratch. However I use a procedural texture for the illumination maps. I generate a 64x64 bitmap where the value of each pixel determines the brightness of a single window (i.e. how much light it emits). It's pretty basic: after the Texture2D is created I just run through all it's pixels and assign a random value to it's alpha component.

Creating materials

There are 6 different textures representing a different type of window each and one illumination map. How to put these together? That's what materials are used for. As with any other element, materials are represented by the Material object in scripts, and they can be created with code. The constructor takes only one argument: a shader.

Shaders are one of the most crucial elements of Unity3D (and any other 3D rendering engine for that matter). They describe how a 3D geometry will be rendered, what colors and/or textures will be used and how it will interact with light. For the buildings I use a shader called Self-Illuminated Diffuse. It's one of Unity3D built-in shaders.

Shaders usually need some parameters to work with - like colors or textures. A material is essentially a shader and a number of objects that define it's parameters. Self-Illuminated shaders expect one texture for the color and the other that determines the amount of light emitted by this texture. It's great for creating objects that emit light like bulbs, neon signs or... windows in a night scenery (it's important to understand however that the object emits light but this light doesn't illuminate any other objects around it).

I use the window image as the color texture, and the procedural image with random alpha values as the illumination map texture. The trick is that I set the scale of the color texture to 64. This way the window is repeated 64x64 times on the material and each pixel of the procedural illumination map (scaled 1:1) corresponds to one single window. I also set the filtering mode of the illumination map texture to Point, so that it doesn't get blurred when seen form a bigger distance.

A word on optimization

The generated buildings have different width, height and depth. A natural solution to apply textures on all of them was to create a material that repeats the windows the appropriate amount of times for each building. Ex. a 4 story building with 3 windows in a row would have a material where the x-scale of the texture is set to 3 and y-scale to 4. The emission map scale needs to be divided by 64 in this case, giving values of 3/64 and 4/64 respectively.

While this seems quite ok, it poses an optimization problem. Unity3D is much more effective when there's a lower amount of meshes composed of large amount of vertices, rather that the other way around - lots of meshes with few vertices. One building has 20 vertices and I create 2500 separate meshes! It's far from optimal, and you can really feel the framerate going down because of that. Fortunately there is a built-in script called CombineChildren that automatically combines all the buildings into one huge mesh making everything work much faster and smoother. But (of course) there is a trick!

Unity3D will combine objects only if they share the same, exact material. The scale of a texture on a material is unique, so if there are two materials that are identical and the only difference is the scale of the texture, we still need 2 different materials. In the above scenario almost each building uses a different material, unless it has exactly the same number of stories and the same number of windows in a row. I had to come up with a way to use only one material, and to deal with texture coordinates in a different way.

The answer is to use UVs instead of the texture scaling. So I started by creating only one material, which contains 64x64 windows. If a building is ex. 20 stories high and has 6 windows in a row, this means that the wall of the building needs to be mapped to the texture with this coordinates [0, 0, 6/64, 20/64]. Remember that UV values are normalized, i.e have values between 0 and 1, 0 pointing to the left/top pixel of the texture, and 1 to the right/bottom.

This way I can reuse the same material on every building regardless of it's size. It works even on buildings that are higher that 64 stories, because UV values are repeated - ex. an UV value of 1.2 is the same as 0.2. Perfect!

One little problem still persist though. Since each building uses the same material it also has the same pattern of lit and unlit windows. It doesn't look nice. Fortunately there was an easy solution for this: to offset the UVs of each building by a random value multiplied by 1/64. That way each building is mapped to a different area of the of the material and things look more natural.

Effects

To make the cityscape more interesting I added a Glow image effect to the camera. It adds a subtle glow around lit areas of the scene - in this case the windows, giving it a slightly blurry look.

Source

You can grab the package with the source code here.

A cinematic video player

Here's a little piece I made after the image decomposer. This time I wanted to load videos instead of images. I had the idea to project them on a big screen so that the user can actually feel like he is in a movie theater. I started to play with it, I modeled a cinema interior, setup the lightning and I thought it's pretty neat.

You can walk around the theater with cursor keys and space to jump over the seats. If you go into the corridor in the back the movie will pause (wouldn't it be nice if that happened in real life too?). Use the left mouse button to toggle a flashlight.

Unity3D currently supports videos in Ogg Theora format only. Although it's free, it is not very popular on the net. It's mainly used by Wikipedia and Archive.org for the video content.

The demo works the same way as the image decomposer, which means you can append a URL to any valid Ogg video on the web and you should be able to watch it. To start, here are a few interesting movies I found:

Sita Sings The Blues, an animated film produced and directed by Nina Paley. More info...

Big Bucks Bunny, a short computer animated film by the Blender Institute. More info...

The Blob trailer, 1958 movie with Steve McQueen. Back then it was scary, today it's funny.

The Adventures of Flash Gordon, 1955. Probably a very bad movie.

Dynamic loading of videos in Unity is easy and it's well documented. The code is almost exactly that same as with images. I won't publish the sources this time because there is very little code in it, on the other hand there are over 50MB of textures in a pretty big mess.

If you find (or create) an Ogg encoded video worth watching post a link in the comments.
Have fun!



  • FITC10


  • FITC10


  • FITC10