Introduction

XR has fascinated me for some time now. Since leaving my previous job, I have had time to rediscover my love for game development and explore new things like XR.

Video games, to me, are what books are to many other people. It's a way of escaping reality for a moment and discovering something new. With a game, the immersion comes from suspending your disbelief enough for you to live and breathe the game world. If video game mechanics aren't cohesive, immersion becomes pretty difficult.

For me, XR is about bringing another world to you, to your reality. This means that believability is way more important, but on the other hand, it means XR can be used as a utility. Spatial awareness is an easy way to make your XR experience immersive.

This is why I got so excited when I saw that 8th Wall has recently released Sky Effects (Sky Segmentation) as a feature. Sky segmentation can determine what is and isn't the sky and allow you to use that as the canvas for your XR experience.

Sky segmentation takes spacial awareness to new heights, literally.

Fortunately for me, 8th Wall also dedicated the month of March to a challenge that asked you to use their Sky Effects feature to create an XR experience to showcase how it can be used. Unfortunately for me, I only gave myself a week to do it.

Coming up with the idea

The experience I created was called "Meta Invaders" and was a total rip-off of space invaders. Here are a few screenshots and a trailer which you can find on my Youtube channel.

Coming up with the idea of a space invaders clone was pretty quick. As soon as I saw the feature, I just wanted to see space invaders in the sky. There was something about the concept that I couldn't shake, and since I didn't have long, I just went with my gut.

I dove straight into the design process, thinking about what I wanted my experience to feel like. Taking from game design, I outlined the following core design pillars.

1. Use movement where possible
2. The Aliens have to feel imposing
3. The playspace is unknown

The third core pillar was sort of automatically solved by the game of choice. In space invaders, the aliens go to the end of the screen and come back again. So there was already a mechanic in the original game that helped with this, but there was a time when I wanted the aliens to go all around the player so that the aliens felt even more imposing. After looking at the online test environment that 8th Wall provides, it was clear that a 360-degree experience wouldn't feel accessible to the player. This was because we can't control how much of the sky a player can see.

The Metaverse theme of the game actually came later in development after I decided I wanted to model the assets myself. Originally the game was called Sky Invaders with the intention of using 3D models I found online. But since I wanted to design them myself, I knew I needed to keep them simple to produce anything that looked half decent in the time I had left. The VR game Synth Riders came into my head with its synth-wave style of simple geometry and neon colors. I felt I could achieve this style in the time, but I was very wrong about how "Simple" it would be, but more on that soon.

Process

Blocking out the experience using primitive shapes or assets is incredibly important when designing any experience; websites, apps, or games. Quality assets can take a long time to build and iterate on, so the more you can do without them, the better. Fortunately, since I was using a 53-year-old game as a base, I could get away with a lot for a long time.

Since movement was a core design pillar of mine, I decided that I needed to de-risk that first. It was actually relatively simple. In a showcase after the competition ended, someone asked if I used a library for the accelerometer, but I didn't need to. I just kept it simple and used the rotation of the camera to update the position of the players' turret. Keeping things simple is always the best way.

Next up was focusing on getting the aliens to behave as they did in Space Invaders. This was my first time using 8th Wall seriously, and I made a few mistakes early on. The biggest one was with the movement of the aliens. I got the aliens moving from one side to the other no problem, but I didn't quite understand how entities worked together in A-Frame. So moving the aliens down just didn't work.

This was where core pillars help you make the right decisions for the game. In the original Space Invaders, the aliens move down when they hit the side of the screen; but since I couldn't get this working, I had to decide if it was worth spending extra time replicating this design or moving on. Authenticity wasn't a core pillar, but imposing aliens was. The aliens were imposing enough by just being in the sky. Bringing them any closer ran the risk of making the player feel uncomfortable. So I moved on. It wasn't worth refactoring what I had in order to implement a mechanic that didn't add anything.

Once the enemies and player felt right, it was time to open Blender. I watched some videos that showed me how to create a synth-wave look in Blender. I then took the original designs of the aliens and tried to make them in a synth-wave style, combining simple shapes that looked great when outlined with neon. Since this was my first attempt at producing assets for a game, I think I did a great job realising the design. I did have a few problems with texturing, but it worked out in the end.

Getting the assets into the game is when things started to get really exciting. I started tweaking the basic mechanics, such as enemy placement, firing rate, and speed, to match the assets. The alien models dominated the sky and definitely felt imposing. It made you feel like you had little chance of winning but invited you to try.

One thing I found really funny was that after playtesting, myself and my friends found bugs and broken mechanics. One of those fixes for a bug helped me figure out how to make the aliens move closer to the player. Earlier in development, I abandoned this mechanic when it was too hard to implement because it might have made the player uncomfortable. But after playtesting it again, I found that it worked great, so it stayed.

I am really proud of my first independent XR game. I believe the design pillars help make this experience stand out. They also allowed me to prioritise quickly on the fly when design decisions needed to be made, which is helpful when working to a deadline. The best XR experiences I have seen all seem to have that bigger cohesive picture in mind from the art style, the animations, and the interactions. They all seem to be working towards the same goal, and having design pillars help a lot with that.

Challenges

But the whole process wasn't without its challenges. I have touched upon some already, such as the alien movement initially not being faithful to the original game.

Another problem I hinted at was my difficulties in getting the assets modeled and textured. The modeling was fine. I got all the aliens and the player model in a few hours. My problem was with the texturing. It turns out that Blender's materials do not translate well to anything else, which was a problem because I used a wireframe modifier that added extra geometry to create the neon outline. Using the material editor, it all worked fine, but since the materials don't export well, I was faced with the prospect of UV unwrapping a very intricate model.

I was really out of my comfort zone. No matter where I added the seams, there would be an issue that made it impossible to get the neon outline. I tried exporting a low poly model and baking the outline onto the texture, but I am not very good at colouring in. Eventually, I decided to just brute force the UV unwrapping. I selected every edge on the object and turned it into a seam, which left me with a messy UV map, but it was usable. Within a few minutes, I had the model textured in Substance Painter and exported it to a format that worked for the web.

The final problem I want to talk about is Javascript. I have been using Javascript for the best part of 6 years now, but I have always used it with a Framework - Next.JS has been my go-to tool for building websites recently - All of the samples I found on 8th Wall used Javascript without a framework. It made me realise how easy modern frameworks make your life. Add to this the fact that the documentation for A-Frame is not the best, I really, really struggled. If you read my "Getting Started with AWS" post, you might remember I had a similar problem with knowing where to start with AWS despite years of experience.

This isn't my first rodeo though. I learned AWS and NodeJS on the job as a consultant with quite a lot of time pressure. I had to go back to basics, assuming nothing, and start building mental models from there. In the end, it wasn't too bad. In about a day, I understood how to find information and could manipulate the scene enough to test a solution before going back and refactoring it to be more reusable. This quick iteration really helped accelerate my knowledge.

Conclusion

Recently, I have been spending a lot of time learning about Game Design, immersive technology, and XR. One thing I have learned in my career so far is that constraints really help creativity. Hackathons and game jams are great when learning a new skill because they help you focus and develop your skills. This is what the Sky Effects Challenge has done for me. I have built XR experiences before, but this is the first time I designed one that wasn't a demo for a client. This was a chance to create a complete experience that people could enjoy.

I have also developed a lot of strong beliefs in how XR experiences should work to enhance the world around a user. Immersive XR experiences can't always be passive. They need a way of allowing you to interact with the virtual objects as much as virtual objects need to be able to interact with the world around you. Immersive XR is difficult to achieve when the only consumer device on the market is a phone. The phone acts as a barrier to making an experience immersive. This is why the movement was a core pillar in my design. By having the player physically turn to move the turret, I am breaking down that barrier enough for the player to suspend their disbelief. I would love to hear whether this was successful or not.

I was also super impressed with how well the sky effects worked. I was even more impressed with how fast 8th Wall enhanced the feature. They enabled features that allowed you to create a more seamless divide between the sky and the ground. They also developed a feature that allowed something to move from the sky to the ground, which looks incredible.

People don't want to have to download an app for something that will only last a few minutes, so being able to create quality XR experiences using advanced features like sky segmentation accessible from a QR code is really exciting.

The next stop on my journey will be THREE.js which will help me take this site to the next level. I also want to design another sky effects concept that takes advantage of a full 360-degree playspace. This is a great time to challenge my assumptions and a great way of reinforcing my THREE.js skills.