The first big obstacle was to figure out how to render the dungeon. This is a genre that has existed since the 1980s, so it's not like this is a new problem, but the early crawlers didn't animate your movement. When you stepped forward, your position updated instantly.

This makes it hard enough to get a feel for where you are when the walls are distinct, as in the image above, but the dungeons typically have indistinct walls, and in some cases, moving could give no visual feedback at all. This makes it very hard to tell where you are. Now, arguably, that's part of the appeal... but that's not what I'm going for.
WARNING: The rest of this entry is very technical. If you are more interested in the design side of things, tune in next time.
Attempt 1: JS Raycasting
Raycasting is a technique that projects 2D images into a pseudo-3D space. The original Wolfenstein 3D and Doom are the best-known early examples of the technique, and while those games' visuals haven't necessarily aged well, the technique is still solid.
This tutorial gives you a very competent JavaScript raycasting engine. It's got a good framerate, and it looks pretty nice. It even has some visual effects: the rain, lightning, and lighting are all very nice.
This raycaster would work fine for what I wanted, though I'd have to add in ceiling and floor, but I didn't want to work in JavaScript. It's great if you want to make a browser game, but it's harder to package into a standalone game.
Attempt 2: Porting JS Raycaster to HaxeFlixel
So the next thing I tried was porting the raycaster to HaxeFlixel. Haxe is a sort of metalanguage that can compile into a wide variety of languages, and Flixel was, I guess, some large part of ActionScript, the language used to make Flash games. HaxeFlixel is a Haxe game engine based on Flixel, and it has a lot of nice features and good community support.
But HaxeFlixel is a 2D game engine. That's not, strictly speaking, a big problem: raycasting is a technique that predates 3D hardware acceleration. However, I ran into a large issue porting the JS raycaster: the way it worked was to render vertical slices of the wall image, scaling them to the appropriate height. Flixel lacked the support for this. It was a no-go.
Attempt 3: HaxeFlixel Raycasting
I didn't want to throw in the towel on HaxeFlixel just yet, so I dug around until I managed to find an ActionScript raycasting demo. The fact that it was so difficult to find one should have been a tip-off here, and I can't find the link to it anymore.
The source code for this was terrible. It took me a long time to decipher what the variables were supposed to do, and the entire thing was built around the assumption that your texture images were 64x64 pixels (for example, there was a lot of bitshifting by 6). The hardest part of this attempt was figuring out what in the world the code was doing.

But hey -- it worked! Unfortunately, it worked at an absolutely miserable framerate, somewhere between .5 and 2 FPS depending on what was onscreen. See, what the code was doing was raycasting individual pixels rather than vertical slices like the JavaScript example. It worked fine at the demo's 320x200 resolution (which I guess should have been another tip-off), but I was running at 800x640, more than four times the pixels. The framerate was a bit better when I compiled to Flash, but I didn't want to make a Flash game, and it still wasn't a good framerate.
Interlude
I wasn't sure where to go from here. Raytracing looked like a no-go in Flixel. I tried for a while to reproduce the vertical slices raytracing, but it ended up giving me worse results than pixel raytracing somehow.
I considered a few other approaches at this point. I could go with a 2D-simulating-3D approach, like the games from the 80s did. To get animation out of it, I'd have to create individual graphics for each texture for each frame of animation. This almost didn't sound so bad until I considered rotation. When the player turns left or right, the textures would have to turn, too.
Eventually, I decided that HaxeFlixel wasn't going to work. I had wanted to avoid a 3D game engine (3D is hard!), but I was willing to consider it at this point.
Attempt 4: libGDX
Some amount of searching led me to a Java framework called libGDX. At this point, I should explain what a game engine does for you.
On one end of the spectrum, you have engines like the OHRRPGCE where you insert assets (i.e. graphics and music) and the engine takes care of the rest. These are by necessity very specific: the OHRRPGCE creates RPGs. That's a little reductive, obviously: you can script whatever kind of game you want.
On the other end of the spectrum, you have pure Java, C++, C#, whatever, with no engine at all. You can often find libraries to accomplish very specific tasks, but what you gain in freedom you lose in automation.
Most engines land somewhere in the middle. You can expect a game engine to help you display graphics, play sounds and music, and usually provide some sort of collision detection and other physics. In general, engines are either intended to create 2D games, like Flixel or the OHRRPGCE, or 3D games, like Unity, Unreal, and so forth. You could create a fully 3D game even in the OHRRPGCE, but you'd have problems like the ones I had in HaxeFlixel. It wouldn't run well. In the same vein, I've heard that it's surprisingly difficult to create a 2D game in Unity. I didn't really understand why before fooling around with libGDX.
libGDX is not so much an engine as a collection of Java libraries. That makes it sound worse than it actually is, since those libraries provide tons of utility, and it gives you the tools to make 2D or 3D games, or even to mix the two. (Even in a fully 3D game, you usually want a HUD or other 2D overlay, so a fully 3D engine would still need some 2D support.)
libGDX required a few paradigm shifts. The obvious one is that I was now working in actual 3D rather than raycasting. The less obvious one is that I was no longer working in pixels. This makes sense: 3D space isn't made of pixels, but from a certain point of view, neither is 2D space. It took me a few hours to get a grasp on this and make some simple surfaces show up.

But show up they did, and I even got the early workings of a HUD going. (Thanks, opengameart.org.) Since this was a tech demo, it's not interactive, but it does what I need it to do, and it runs smoothly. I'm going to call this a win.
And now that I've resolved my primary technical concerns, I can turn my attention to what makes the game fun.
Appendix: More about these engines
I went into this accepting that I would have to write a large chunk of the game from scratch. I know of exactly one engine for first-person dungeon crawlers, and it's old enough to drink and not very flexible, either. If you're hoping to minimize the amount of code you write, stick to engines like the OHRRPGCE.
If you're OK writing a lot of code, HaxeFlixel is great. It's based on a well-loved language, it's entirely open-source, and it's got an astounding amount of community support. If you're looking to make a 2D game, take a look at HaxeFlixel. Notable games written in HaxeFlixel include Defender's Quest and its upcoming sequel. Every Flash game you've ever played could also be easily ported to HaxeFlixel.
libGDX is a little lower-level than HaxeFlixel. In HaxeFlixel, you put stuff on the screen. In libGDX, you set up a camera and render the individual objects. You're also responsible for disposing of objects when you're done with them. On the plus side, since this is Java, the IDE support is unmatched, and you get lots of extra capabilities outside of the already strong core libraries. Even so, I have a hard time recommending libGDX for general purposes. If you want a 2D game, go with HaxeFlixel; if you want a 3D game, why aren't you using Unity? But libGDX happens to cover my specific use case: I don't actually want a 3D game, but I happen to need 3D anyway. The best-known libGDX game to my knowledge is Delver, which is also a weird mix of 2D and 3D.

