Oblivious Sponge

So the last few months (agh!) have had me coding monkey like at work and at home. The work stuff is finally getting there - we’re hitting beta in a fortnight. The home stuff is also finally getting there - that quality bar is hard to reach. Especially when the app you’re writing is so damn addictive you spend hours playing, getting lost in a four dimensional space looking for just one more funky attractor.

(I hadn’t realised how much better the image quality was compared to my prototype until just now - looking at the Clifford in the previous post with no AA, no autocontrast etc. Yay!)

Anyway - interesting notes on the journey so far:

• Chip away. Small goals keep you travelling in the right direction. You can only finish if you start.
• Always start with something that works. The sample Managed DirectX GUI, for example, is horrible in places - but at least it works!
• Don’t be scared to ditch stuff. My funky 6D GUI widget proved to be confusing. So I’ve nuked it for a 2D one. Mere mortals can use my software again.
• Time stuff before you ‘optimise’. Drawing 100000 points is slow, right? Well not as slow as pulling a 2048x2048 texture from the graphics card, modifying it then chucking it back.
• When things are hard, stop. Mysterious bugs become obvious after a bath, a walk in the park or a glass of wine.
• Don’t overgeneralise. That 2D convolving interpolator wasn’t required in the end.
• Look into existing solutions. That overly complicated special autocontrast routine is actually a gamma correction model in disguise.
• Don’t be put off by competing products. There were MP3 players before the iPod.
• Show off to people. Having others cooing over the software massages the ego and keeps you going.

So, now the hard bit. Getting to the destination and not getting sidetracked. So many of my projects die at this point - I start to worry that it’s going to fail when out of my hands, at least when stuff dies it’s failed due to me. A strange thought process, and one I’m trying to train myself out of.

For now though another snapshot for the road:

After a month of hectic Real Stuff I’ve finally got around to doing some of my own coding. And, as usual, it’s a work colleague’s fault.

I was speaking to Paul about general enthusiasm for coding and realised that sometimes naivity is great. When you first started coding and you’d just write something for the hell of it. You didn’t worry about overall ‘quality’ because you were doing the best you could imagine. I remember writing platform games, GUIs, Lorenz attractors - well, Paul has some bloody great attractor images from a few years ago. It managed to reignite my interest in the subject so I spent a couple of hours with some C#, SdlDotNet and Forms and managed to get a little app to render different types of Clifford attractors:

Nice, huh? Just 2d for now, and fixed to the single kind of attractor. Not a bad start though. And the best bit was the mindset I put myself into. Who cares if it’s one class? Heh, I don’t have to rename the GUI elements, so I won’t. It’s resulted in a hideous looking 508 lines of code, but a fully functional app. Now I can go back and rewrite the same functionality with a higher ‘quality’ bar and with more extensibility.

Macs. Yum. They’d match my iPods nicely and fit my shallow aesthetic needs - even though my current PC is small, white and quiet.

The main pain of my PC is the OS, and more specifically, its failure to do much in a sensible and stable way. My PC doesn’t always shutdown. It doesn’t always startup. Its window redrawing isn’t buffered, and is sooo 1993. It has a hideous look when returning from the screensaver. It handles multitasking in a seemingly clumsy, awkward way. It doesn’t look as pretty as I like.

But the main joy of my PC is .Net, and more specifically, Managed DirectX - I’m so utterly in love with it’s high levelness that I can forgive it’s failings in exception handling and documentation. And anything that hammers another nail into the ‘oh, we must write in C++ for performance’ coffin is a good thing as far as I’m concerned.

So I’m not ready to Switch, yet. An iBook under my coffee table wouldn’t be too bad. (I have my eye on its VGA to S-Video for MAME goodness. But that’s another story). For now I’m making do with a couple of ‘Shell Enhancements’, or Windows Hacks. Newly discovered today is the bizzarely Germanic Entbloess 2 for its Expose mimic. It does it rather well - especially when you’ve tweaked the ugly default font, turned off the dumbass ‘help’ strings and set the mouse trigger areas to be Just Right. It appeared on LifeHacker this morning, and I’d shelled out a whopping 5 new British pounds on it this afternoon. I did have to go to IE to successfully view the Download page however. Oh the irony.

The other Windows agnostic stuff I’m currently mucking around with is the much hyped (but also quite good, especially when viewed from my 2000-era-PHP-Web1.0-experience point of view) Ruby On Rails. I stalled a few months back when I found myself searching for an editor with both good keyboard shortcut support and quick filesystem browser. And which wasn’t Emacs. Failing miserably I sobbed gently into my Rails book (which, it has to be said, is my favourite techie book in the last couple of years) and vowed to try again soon.

Well, the Rails community certainly isn’t slothful. In a matter of weeks some clever bods have built ontop of Eclipse (another sobful aborted experiment from, ooh, about 12 months ago, when I was too tight to buy C# and braved Mono and GTK# instead. Mistake). RadRails certainly feels special. It kicks off the WebBrick server for you, highlights code good, navigates and copes with the filesystem in a good way and (most importantly) supports custom keybindings. The only work I had to do outside of RadRails was to create the database (even the database.yml has a pretty GUI frontend! This impresses me, at least). (and anyway, SQLyog is a pretty straightforward and honest GUI for MySQL, and easily copes with my database needs).

Oh, and RadRails also looks good. Which, you may have gathered by now, I care about a little too much.

Overview:

So we successfully jammed on Saturday. Most important things first (a priority that cost me on the day), a pretty picture:

Ok, so it’s not that pretty - but as a result of less than 8 hours work, it’s not so bad. The green dudes move around the world. Two of them drop purple slug trails and do a very dumb wall following. The other two do a similar dumb wall following but bleet sounds when they hit a purple trail. The bleets move around the world, propogating and stopping when they hit walls. Odd, but lovely and general.

Setup:

• Hardware - Two PCs (Dave’s shiny laptop, Dom’s tiny PC and ridicoulously heavy CRT. I’m buying a bloody TFT if I’m ever doing this again! Actually, scratch that, I’m getting me an iBook), USB hard disk, pad of paper.
• Software - Python 2.4, Pygame, Subversion, TortoiseSVN, Emacs/IDLE, Paint Shop Pro.

10 til 11: Dave makes coffee. Sketching general plan on bits of paper. Decide on a discrete 2D grid where agents can move in straight lines. Agents can sense properties of other agents and can spawn/destroy agents. Every agent takes it in turns to move, so world mutated in place.

11 til 12: Cake. Arrays in Python. Simple colour filled images in Pygame representing array. Setup subversion local server.

12 til 1: Matthew arrives and looks puzzled. Got single dude (a green square) moving towards a wall (red squares) and running through it.

1 til 2: Quiche. Got dude bouncing off wall into oblivion. Paved over boundary to oblivion. Added slug dudes that left purple trails. All dudes bounce of purple trails.

2 til 3: Chocolate covered coffee beans. Check out repository to Matthew’s disk for Dave to work off. Realise how much better svn is than cvs. Dom/Matthew work on making renderer shinier. Dave works on having more than one thing per cell.

3 til 4: Yum yums. Dom has written bad isometric engine. Matthew realises a Better Way. All whoop as walls suddenly pop into glorious 1983-isometric-style existance. Dave writes level reader.

4 til 5: Dom draws some not so pretty pictures for agents and walls. Talk about general voxel engine. Remember the company we work for.

5 til 6: Matthew goes home for parential responsibilies. Work on adding sounds to world. Supposedly propogate around the world, letting other agents avoid stuff. Demonstrates flexibility of what we’ve done, but doesn’t quite work.

6: Fin.

Bad Things:

• It’s not a game!
• The isometric renderer took 2 hours
• Mechanical refactoring in Python is hard to do in a bug free way. Compilers do this well
• Time pressure makes ditching bad implementations hard

Good Things:

• It’s a lovely general tilebased AI engine!
• Writing something graphical with Pygame is very easy
• Having 3 people in one room with a tight timescale avoids coding inertia
• Subversion is great

What I Learnt Today (or Saturday, in fact):

The stuff that I find hard is also stuff that other people find hard (shocking, heh?). The difference is that brighter folk keep the options in the background and go for the best option given current information. I typically stay in a stuck state.

Small concrete goals are good. ‘What are we going to do in the next hour’ and the like.

Having someone watching me work is great. Matthew pointing out how stupid my rendering logic was helpful. A step back saved us yet more pain.

More XP style estimating and planning would have helped. We could/would have dropped the rendering/sound propogation and moved onto something like user movable objects, or something more game like.

Python works well when types are fluid, but is painful once you’ve decided.

I get distracted too easily. People pestering me to get on with it is a good thing.

Cake is good.

More fun with coroutines. Having written off Lua for the job (creating new variables due to typos? Agh!) we decided to have a look elsewhere. There’s not exactly a glut of small embedded languages that support coroutines. In fact, there’s just one, Io. It looks really interesting, but frankly it’s a bit too bleeding age for me right now (F# is about as bleeding as I can cope with).

Anyway, implementing game logic in coroutines is still a cunning plan - the more Dave describes it the more cunning it gets, so I tried (naively) good old Python. It’s had generators since 2.2, so maybe, just maybe, I could write coroutines using some dummy yields. Ah. No. Realised this evening (with a little prototyping and other help) that it’s not possible without switching to Stackless Python. Which would mean breaking all the libraries I want to use - the major merit of Py. Sigh.

In other (kind of related, and better) news, my DS PassMe arrived today. Now I can start ‘real’ DS development. In that vast amount of spare time I have. *cough*

And in further (kind of-kind of related news), myself, Dave and another colleague, Matthew are attempting our first Game Jam this weekend. Hopefully we can get something fun done in a day - I’ll post the results next week!

After using C++ in anger for about four years I’m constantly troubled by coding inertia - when you know you should create a new class/factor out a base class/extract a method - but it’s just a little bit too much effort. Especially when combined with CVS and multiple Visual Studio 6 workspaces/projects. Yagh!

So? Well, C#, Visual Studio 2003 and Subversion working together almost gets rid of this inertia. The only problem remaining seems to be me. I still put off (or at least overestimate and dread) what end up as simple, mechanical refactorings.

The motivation for this upbeatness is that I just managed to rip out the hardcoded particle generation and force application from my particles engine and add fountains that move in the wind in about 10 minutes.

Tomorrow, hopefully, I’ll get them reacting sensibly to some implicit surfaces (well, some kind of floor, to be precise). Oh, and they look better moving. Honest!

I’m quite happy with Visual Studio 2003 (I’ll be even happier with 2005, judging by the beta). I’m determined to use it for as much coding as possible - something I never managed in Visual Studio 6.

Anyway, I’ve semi-happily got it working with F# (an OCaml under the CLR, for those who care. A lengthy post may eventually appear on this subject). Tonight I’ve started using Lua again and have got into a reasonable state of editing/running code in Visual Studio. I downloaded lualite and added lua.exe as an external tool in VS with $(ItemFileName)$(ItemExt) as arguments and $(ItemDir) as the initial directory.

There’s probably a better way of getting it to play nice with project configurations and the like, but this will more than suffice for now.

As for justification for Lua? Well DaveT (a wise work colleague) has a crazy idea That Might Just Work about Lua and DS development. I’ve currently got plans of a yielding PacMan. Don’t ask.

I bought a Nintendo DS when I was over in the States this summer. I’ve become something of a serial importer and my impatience with European release dates (and prices!) has resulted in ridiculously large DS games collection already. I can’t recommend the console hightly enough - it’s unique, robust, plays all my beloved GBA games, is properly lit and the battery lasts for weeks.

Anyway, the DS homebrew scene seems to be trundling along nicely (although nothing like the PSP) and the architecture of the machine really appeals. I tried some GBA coding a while back but the results always felt like simple stuff I could have done on the PC. The DS and its stylus, however, give me lots of scope for interesting ideas.

So, I’ve just ordered a DS PassMe. It took a fair bit of hoops and jumping - I don’t have a PayPal account and that’s the only payment method the site took. After some pursuading a friendly work colleague ordered a couple using his account - now all I have to do is wait. The guy puts the things together himself, so it may be some time.

Bumbling on with the particles engine has led me to pixel shaders in C#, using HLSL. I even have a book. Up until today however, I’d just fiddled with existing examples.

The code already had the alpha blending and coloring of the particles as a very, very simple pixel shader. It needed more per particle variables to find their way down to the shader (e.g. particle lifetime, coloring function etc.) - this requires using a custom vertex type and a corresponding vertex shader that mangles it into a sensible input for the pixel shader.

This is what I ended up with (the odd variables at the top are for EffectEdit, an application that comes with the DirectX SDK and compiles and displays effects. Definitely worth using to find mistakes in the shaders!):

// Stuff for EffectEdit
string XFile = "tiger\\tiger.x"; // model
int BCLR = 0xff202080; // background
float4x4 gTransform : WORLDVIEWPROJECTION;

struct VS_OUTPUT
{
float4 Pos : POSITION;
float4 Color : COLOR0;
};

VS_OUTPUT VS(float4 position : POSITION, float4 color : COLOR0)
{
VS_OUTPUT Out = (VS_OUTPUT)0;
Out.Pos = mul(position, gTransform);
Out.Color = color;
return Out;
}

float4 PS(VS_OUTPUT input) : COLOR
{
float4 result = { input.Color.r, input.Color.g, input.Color.b, 0.5 };
return result;
}

technique Monkey
{
pass P0
{
// shaders
VertexShader = compile vs_1_0 VS();
PixelShader = compile ps_1_1 PS();
}
}


Then the C# code is just:

m_effect = Effect.FromFile(m_device, "../../effect.fx", null, null, ShaderFlags.None, null);

m_device.Clear(ClearFlags.Target | ClearFlags.ZBuffer, Color.Black, 1.0f, 0);
m_device.BeginScene();
m_effect.Technique = "Monkey";
m_effect.SetValue("gTransform", m_device.Transform.World * m_device.Transform.View * m_device.Transform.Projection);
m_effect.Begin(FX.None);
m_effect.BeginPass(0);
// Draw stuff
m_effect.EndPass();
m_effect.End();
m_device.EndScene();
m_device.Present();


And yeah yeah, it’ll just render something in a slightly wimpier color than the color specified for the vertex (due to the 0.5 alpha). The idea’s there though - more can be added to the VS_OUTPUT and vertex structure and the appropriate calculations can be done in the appropriate shader. With hindsight, many calculations that are per-particle can be done at the vertex shader level. Oh well.

As for the stuff that’s actually drawn, well the call to DrawUserPrimitives now uses a custom vertex array (all 6 vertices!):

struct MyVertex
{
public Vector3 Position;
public Vector2 TuTv;
public int Color;
}
private MyVertex[] m_quad_verts = new MyVertex[6];

The only slight faff is setting up the VertexDeclaration (not the VertexFormat, I believe that’s for the fixed pipeline, not shaders). This is done in the renderer construction, around the same time as constructing effects, fonts and the like:

VertexElement[] decl = new VertexElement[] {
new VertexElement(0, 0, DeclarationType.Float3, DeclarationMethod.Default,
DeclarationUsage.Position, 0),
new VertexElement(0, 12, DeclarationType.Float2, DeclarationMethod.Default,
DeclarationUsage.TextureCoordinate, 0),
new VertexElement(0, 20, DeclarationType.Color, DeclarationMethod.Default,
DeclarationUsage.Color, 0),
VertexElement.VertexDeclarationEnd
};
m_device.VertexFormat = 0;
m_device.VertexDeclaration = new VertexDeclaration(m_device, decl);


Slightly scary looking, but it’s just setting up offsets into a datastructure. The important thing is that the VertexDeclaration matches the layout in memory as MyVertex.

I’ve been running a script at work for the last 2 days that goes through a huge filesystem on our network, parsing files and spitting the results into SQL statements. Trouble is, I didn’t think there’d be that many so I outputted them all to one directory. So far I’m up to 318351 files. Each around 5k of SQL. Windows doesn’t seem happy, I just managed to make more crap out, and dir stopped working a while back. Cygwin isn’t any better - I can’t even change into the directory. As for Explorer - well it won’t even let me access the drive.

If it ever finishes I need to push the SQL into MySQL. That’s if my PC doesn’t melt beforehand. Eep.