• images
  • 11:15 am
  • images
  • No Comments.

Universe: hard lessons and directions

The old Python/C++ codebase coupled with the server side Python/C++/Lua codebases (I promise how I did this made sense) has lead to some pretty hard lessons.  For example: while it’s fine to have worker threads running/handling data based on dynamic code allocated inside the VM and handled by whatever is best suited to it (Lua can use tables as indexes, need I say more?), it’s not fine to have the message passing system relaying everything through C++ for the purposes of getting a uniform system.  The issue here is that Lua had to do some weird things to get the message back to the Python master-server and the C++/Python client, upon getting the “translated” message had to do similarly nasty things to get it into the Python from the C (notably the protocol does work in Python, I had reasons to filter/restrict/control in C++, namely features of C++11 and threading to avoid the Global Interpreter Lock).

The real issue here (and that should say a lot) is that the threading and locking of data became problematic when I started to shift the Python code out.  Ignoring how unmaintainable the C++ code became without a higher level language, ignoring how painful networking became, and ignoring how straight up irritating some of the libraries were which were tested (no, seriously, they are bad, use enet if you want networking over UDP), I had a small problem: each actor had to run its own data in an isolated way.  Since I don’t believe in shared memory (it’s just a straight-up no-go with multiprocessing related things if you’re at all concerned with stability in the face of weird issues), it became a message passing system.  As I’ve since noted elsewhere, and discussed many times, this already exists: in Erlang.

What the hell does this mean for the eventual ‘user’? The original alpha-streams allowed users to just download blobs of files (archives), make sure they had the proper runtime support (MSVC2010, or for the undistributed it was MSVC2012), and they could be on their merry way.  With the master server moving to Erlang/OTP (17+), the theory would be that the server running the game would thus need to have Erlang installed.  This isn’t exactly hard, and about the only thing that would likely worry people is having to add it to their path — which of course could be handled by a batch file on Windows, and would already be the case for most Linux/POSIX systems if they used the repositories of their system (for Mac, well, you’re on your own, but that’s always been the Orbsphere way when it comes to that platform; I’ve literally, completely, and transparently maintained for the test user on Mac that if it compiles without errors I ship it for testing, and it’s not that I have no love for the operating system (well, I don’t), it’s that I don’t have a Mac to test it on other than a Mac Mini Server which compiles the code, and before that it was an early Intel desktop which was horrible to use).

By way of clarification it probably helps to understand the parts:

  • Master Server refers not to the “call home DRM-like beast” at the other end, but rather to the server of the metaverse in which the Orbsphere Universe game instance lives.  Remembering the old Universe alpha-streams had parallel universes with initial states of identical entropy, and the aim was to see how far, and how well, parallel planets (growing into system controls, growing into galaxies, etc.) would fare given the parallel realms.  (Later, of course, I added the travel mechanics to the last distributed beta stream so that sufficiently advanced universes could jump dimension to dimension to interact, peacefully or not.  Pre-travel mechanics they could communicate after a point, but only within the fourth dimensional reach of their lunar-equipment (where moons housed the inter-dimensional transfer equipment …));
  • Server infrastructure referred to the local client-homed “processing array” of threads which spoke to the master server through channels of a single socket (at one point it was actually four sockets to overcome some weird latency with chat).  This off-loaded the latency related issues with the (forthcoming at the time) graphical client, as well as stopping things like sound, networking, and other things locking up the frame updates.  In essence the “server” was the logic core, but in its own right only had the power to run a local game in a limited setting and it would never update back to the master server (and even if it did the master server would reject it because it had no idea what was happening).  In short: the server was a logic processor with incredibly limited self-sustaining behaviour to simulate what’s happening based on an entropy seeding system which was designed to allow portability (it was PRNG, not RNG, where it was used);
  • Client infrastructure was in two parts.  Early graphical clients provided vision of what was happening while the actual input/handling was done in a command line window (terminal, shell, whatever; I guess shell).  Scripting here was initially done in a sloppy way, later it was moved to handle Lua input, with the eventual aim of key based inputs (which Python and Lua both, at one point, accepted; this was disabled on the second stream, as some remember, because features grew faster than the input system could handle neatly, and giving people an API to use and pcalling their calls was cleaner than giving them a forever updating list!).  The honourable mention to the Bifrost member who pointed out I could just self-document the API by letting people input “help()” and having it dump it from a text file (thank you Lua debug).

Where now? Most probably to Erlang.  I started porting some of the old ship code to Erlang before remembering I don’t have virtual/abstracts, and so progress stalled while I got my head around what functional programming meant for Orbsphere.  Of course it means cleaner updates (hello hotcode updates mid-session), but it should also allow, to a degree, some fairly nifty tricks for the Master Server with a platform binary present alongside the Erlang (like version regression), depending on how I handle beam distribution.  I’m actually tempted to hook it up to some sort of version control (git, most probably), which will be called out of band (as it’s GPL) to set/get the current build version based on the hash, and use an internal table (or even a webpage or something) to handle the version naming.

Sounds … something … I couldn’t agree more.