Blacknova Realms

📆 Post by TheMightyDude, Tue, February 07, 2023

Development Information Update

Have hit a brick wall today with my ThreadManager code, seems I couldn’t shutdown a tread in the NetworkReadLoopThread when it needs to close the Socket when the remote end drops the connection and where I wanted to not have Threads running that didn’t need to be running.

It seems that you cannot close a Thread within itself due to how its coded.

But decided on while the game is running whether its connected to the server or not, will have all required Threads running, which is fine due to they just wouldn’t be doing anything in them.

Both Server (Console App) and Client (Desktop App while in development) I am able to Connect, Authenticate and Create New Account.

Next step would be to try the Client code within the Unity Engine and see if it works, my guesses would be that it won’t work and will have some errors that will need to be resolved.

But once that’s all resolved and working I “should” have the basic Networking Done and “should” be then able to add new stuff.

Current Server Look (subject to change)

I know it looks like not much has changed, but code wise a lot have changed, lot of redundant code was removed, how the code did stuff is also different.

SRP (Secure Remote Password) Support.

📆 Post by TheMightyDude, Fri, January 27, 2023

Development Security Feature

Just a very quick update this time round…

SRP (Secure Remote Password) is now added into the API for Authentication, so passwords are now no longer sent over the network or even stored in the database.

All that is stored is UserSalt and UserVerifier and that is it.

Health & Development Update…

📆 Post by TheMightyDude, Mon, January 23, 2023

Development Status

Well been taking a me time for a while and I think I am in a good place so have gone back to development.

Took a little while to figure out my code that I had backed up due to losing all my up-to-date code in the datacentre fire, but starting to get back to knowing what I was trying to do.

But just because I am back to developing doesn’t mean it will all be done anytime soon, I am way off due to having to design and write my own API containing Multi-Threaded Networking etc.

But it’s a start at least.

Sort of update...

📆 Post by TheMightyDude, Tue, September 20, 2022

Development Information Status

Hey all,

Bit of a small update I am afraid.

Server so far

This is the server so far, it accepts multiple connections at the same time, queues up network messages etc, and that is it LOL

Sorry for being A.W.O.L

📆 Post by TheMightyDude, Mon, June 14, 2021

Development Information

Well, been having loads of bad low days so not done much as late, but I have played about with WebSockets and while they are on the cool side, BUT sadly they are not easy implemented to support multiple connections which this game will need.

I am not saying no to a browser-based game, but for now I will be concentrating on doing it as a Desktop Game Client using TCP Sockets, at the start it will be just for Windows (probably Windows 10) and then maybe Linux and Mac.

Most CPU’s support hardware AES so the games network packets will be using Symmetric Encryption.

Well, erm, we are sort of getting back online.

📆 Post by TheMightyDude, Mon, April 19, 2021

Development Information Status

Well, most of you already know that we lost our server in a fire that occurred in the datacentre that hosted our rack server along with loads of others, sadly our server was destroyed in that fire.

We have acquired a new server much faster than the one before and have installed all new up to date software etc.

Now comes a very long and slow process setting up each section bit at a time.

This is a huge setback due to not having a full backup of all my new code, but not all is lost I still have some proto code here that I could look into and work from that, but I think a complete new fresh version might be the best bet due to I did change ways of doing things.

I am still on the fence whether to have the game as a browser-based game using Unity WebGL which will have to also use WebSockets or do a desktop-based game also using Unity IL2CPP which can use TCP or UDP, I just need to look at the pro and cons of it and their limitations.

🚧🚧 Development put on hold for now. 🚧🚧

📆 Post by TheMightyDude, Thu, November 12, 2020

Development Information Status

Just a heads up referring to the development that I have been working on, sadly I have put all my development on hold for now.

This is due to I haven’t been in the mood to even work on code, this phase hopefully won’t last long.

Sorry if people were hoping for this to be done soon, but doing a massive multi-player game from scratch on your own trying to keep as many players happy at the same time, that along with health issues all starts to take toll.

Plus, I would like to make a game in which I like playing which BNT sadly isn’t and hasn’t been for many years, this was why I was going to do a recode with some changes, sadly this has had challenges especially with the Networking side which I had to code from scratch to be Multi-Threaded.

Please note I have not given up on this project, just more of a put aside kind of thing for me to come back to at a later date.

Sorry for being A.W.O.L.

📆 Post by TheMightyDude, Wed, October 07, 2020

Yeah, I know I have not posted anything here in a while, I have been ill on and off for the last few weeks, this along with all this Covid that has been going on at the moment, so I have been spending more time chatting with friends and other developers.

I have also been working on some code, not as much as I have been in the past, but some code has changed somewhat, have now got a sort of framework for the server side in place to make it more easier to access, some of that framework is also for client side but just not done much client side as yet.

But I have been looking into Clusters and Instances, which is rather interesting :)

Clusters would allow for more stuff to be run and more concurrent users per game which would be nice, but we won’t need that for a while yet, but it would be really nice if it supported that in case this game becomes popular like it used to several years ago.

Instances is a very cool subject; this could allow for us to just create say a Universe Solar System and then have instances of that allowing for many Solar Systems run on the same server which could handle a set number of players in it at the same time.

Instances could also be used for Battle Zones where player battle with other players, this would take PvP (Players vs Players) away from the main game areas where players just want to play the game without being killed resulting in them loosing everything and then giving up on the game, this has hurt Blacknova Traders a lot in the past and has pushed away possible new players.

I haven’t coded any of the Cluster or Instance code as yet, so this will be fun trying to do some later on when I am ready to do so.

More to come.

Structured Debugging Feature

📆 Post by TheMightyDude, Sat, February 29, 2020

Development Planning

Working on updating my Realms.Debugging.ThreadedDebugLog class to supports different message types, like (ERROR, WARNING, INFORMATION etc) at the moment it only displays text that is sent to it.

This will then also add structuring and colouring to the outputted text to the console, logfiles will only store the structuring and will not contain any colouring in it.

Network Update 8

📆 Post by TheMightyDude, Sat, February 29, 2020

Development Planning Server Client

Well its been a while since I updated the blog, anyhow I really thought I had it when a small glitch caused the test server to crash and core dump, which is not what I wanted it to do.

No matter what or how I did it I was having issues adding TLS into my TCP Connections, most common was if you open a connection to the server using like PuTTY and not type nothing it would just hold that thread on the server dealing with new connections resulting in no more new connections. This was due to how the SslStream does its authentication as Server and Client, the server will just wait a set time until it times out.

So, at this point I think I am just wasting time trying to get this to work as I want it to, so I am just going to use a web request over a secure connection (i.e. HTTPS) and stick with that, loads of games I have checked do it like that, so that must be the better way to do this.

Doing so will allow for me to progress onto structuring the network packets.

🛠 Framework Update 2

📆 Post by TheMightyDude, Thu, November 07, 2019

Development Planning Server

Ok, played about with some code and it now loads up the Dummy Services fine, and I can now unload them (sort of), but it works as I want it.

Going this route, I should be able to have the Auth Service as encrypted with SSL / TLS and the other Services as unencrypted connections.

I can either have all services in separate server applications or all in one server application.

The main thing was to allow for secure connections when needed and not be forced to encrypt everything.

Sure, there will be more connecting and disconnecting but it would be like that if I separated all the Services, so no issue there.

🛠 Framework Update…

📆 Post by THeMightyDude, Tue, November 05, 2019

Development Planning Server Linux

Well after cleaning up some classes it seems that a lot of code wasn’t being used, this was due to moving over to a new better way of doing stuff.

Like before I am using Reflection to load up all the required services and have them hook into the framework.

I have also setup SVN locally where I will be working on the code.

I should now be able to have all services in the same runtime assembly and just have the client connect to the server and tell it that it wants to login, the AuthService will respond with that it needs to enable encryption, they both enable encryption and client authenticates, server then responds with yeah ok along with a connection session id (i.e. UUID type structure) followed by the IP Address and Port of the Lobby System which handles all the list of running games, then the client drops connection.

The Client then connects to the Lobby Service via the supplied address and port (this is not encrypted) and asks for a list of all the game servers running. This connection remains connected for server news, ID Chat (global chat that goes through our network to anyone on your friends list etc, so each player can be on a completely different game server and chat), Friends / Clan members connecting Alerts etc.

The Client then selects what game server it wants to play on and then opens up a second connection for the game server.

The Game Server will handle all the game as normal along with Game Chat (only allows chat to players on that server only.

Anyhow the client sends its connection session to the Game Server and that Game Server then will ask our network for a reference id (UUID formatted structure) to refer to that client every time it connects to that game server.

Game Servers DO NOT store any account information, only we store that, Game Servers will ONLY store game information like (Character, Ship, Universe, Planets etc)

Anyhow more information to come.

I see Reflection :P

📆 Post by TheMightyDude, Tue, September 24, 2019

Development Planning Server

Ok, I have had a few hours spare today so I have been playing about with how the new framework will be done.

The Server Runtime looks up the Different Services in the Runtime Assembly and then Invokes their Initialization Method which then setup its own Handler of Network Messages, starts up its own features (database loading, caching etc) and plugins etc.

Then the Server Runtime continues onto starting up its own Network Service and waits for a new connection.

The Network Connection can be Encrypted (RSA Key Exchange) using TLS1.2+ per Networking Service, so if all services (Auth, Chat and Game) are in a single Server Runtime all Network Traffic will be encrypted, however if all separate Server Runtimes then you can select to use Encryption per Service.

🛠 Working on new Framework...

📆 Post by TheMightyDude, Mon, August 26, 2019

Development Planning Server Linux

I am trying to design the code as a Framework, this will allow me to create multiple Projects (Auth Service, Game Service, Chat Service, Game Client, and Proxy Service) using the same parts of the code.

These Projects like Auth, Game, Chat and Proxy Services might end up as Modules, these Modules can also support Plugins (i.e. sub Modules) that are loaded up at runtime, so these Modules and Plugins can be switchable.

I am also aiming to use .NET 4.5 upwards; I have done test applications using .NET 4.7 and it worked fine on Ubuntu Server.

The reason to using Modules is so that if at any time we stop supporting this game we can easily release the Service Modules not normally supplied for Game Servers like Auth, Chat and Proxy Modules, and with a slight config change will allow an admin to host the game as a whole.

--- More to come later ---

🚧🚧 Bases 🏗🏭

📆 Post by TheMightyDude, Sat, July 20, 2019

Feature

I played some games last week and wondered how I could have some of these featured in this new game.

Features like bases on planets, a base is required to update the areas to allow for more stuff to be placed on.

So, assume we are allowing for players to own planets, they would have a default size of plot on the planet, now if they build a base on the planet it will allow for different structures / buildings etc to be placed, so we could have different types of bases, like Military and Research bases.

Each type of bases will unlock different types of buildings and will have levels to them, each level unlocks more areas to place those types of buildings.

But due to where I am wanting to have this game run as a persistent running game this wouldn’t work as intended the longer it was run.

But there shouldn’t be any issues in having bases on planets for resource harvesting, researching bases, and so on, like the higher the base level the larger the area of land to place stuff, assuming the Planet, Moon, Asteroid etc is large enough to support that base size.

More on this later.

Network Update 7

📆 Post by TheMightyDude, Fri, July 19, 2019

Development Planning Feature

Well not really done much for the last few weeks, had a few Hospital Appointments and also being ill on and off 😷.

While no coding has happened much as late 😢, I have been looking at some stuff, I did look at WebSockets, sadly I would have to use Apache as a Proxy Server and due to Apaches Limitation in its amount of opened connections this wouldn’t be the best route to take.

Sure, I could create our own Proxy Server that might work, but run the risk of security flaws due to not testing it in a secure safe manner.

As for the Client and Server Framework, I “think” I have a good idea on how to do this now so that I won’t have that Singleton / Static issue that I noticed last time.

I just need to spend some time coding a rough version of it and then split the generic code that’s used for the Client and Server and put that into a base class and derive / inherit from that.

I have currently been working with TCP Sockets, but might also add UDP Sockets to the framework.

So, anything that needs to be secure (via SSL/TLS) like Authentication etc will use TCP and the actual Game networking traffic could be UDP Sockets, this will speed up networking traffic due to its connectionless.

But this is just an idea.

Blog Section Update #1

📆 Post by TheMightyDude, Fri, June 28, 2019

Blog Update Information

You all probably already noticed the changes to the blog section, but I cleaned it up a little and added a more better static background image, I have also cleaned up and changed some files so it should load up faster.

The background image use to be a JPG but is now WebP format so it went from about ~2MB down to ~240KB and I couldn’t see any loss.

💬 Oh Hum, I really need to do a Design Document 😞

📆 Post by TheMightyDude, Thu, June 20, 2019

Development Issues Planning

Oh Hum,

It seems that I really need to look ahead when developing stuff.

Ok, I was using Instances (a Singleton) with my code so that exact object was the same one all over my code.

This worked great for the Auth Server which was almost done and also ok for the Client Connections, sadly this was NOT the case when I started to work on the actual Game Server.

The Game Server will allow for thousands of Concurrent Connected Users (ccu) along with creating Client Connections itself to our services so that it can talk to us, some services might even be running multiple servers or multiple clients.

Like the Game Client will open a connection(s) to our Network Services (For Authentication, News, Global Chat, Friends / Social / Communities etc) as well as one for the Game Server that they will be playing on.

And where everything was using Instances (Singletons) it all started to clash with each other (i.e. I was only able to run one single service), so now I need to look into this once again.

This isn’t really a bad thing, I have found some better ways to do some stuff, its just a little setback that’s all.

I am able to allow for Plugins for the Game Servers which get loaded at start-up which would allow for new features like New Commodities, New AI and so on.

We are still here :)

📆 Post by TheMightyDude, Tue, June 11, 2019

Development Information Status

Sorry not done much as late (wow, has it been that long), RL Health issues all stacking up on me, not really Monitored the games we even run.

But on my down time I have been thinking how to do some stuff, I have also been looking into WebGL, so we "might" have a Browser based version along side a desktop version, but I really want to concentrate on the desktop version first. And if there was a demand for it later on, we could look into building a WebGL version.

I have also been looking into my Networking Library and have seen some parts that might become an issue.

Anyhow I just really wanted to say I have not given up just yet LOL

And we are back.

📆 Post by TheMightyDude, Sat, October 13, 2018

Development Planning Server Linux

Been a while since my last development blog, I did post a couple of blogs which was lost when we had a Server Hardware failure where one of our SSD Drives died, sadly we were running these SSD Drives in RAID 0 (Disk Striping) so we lost absolutely everything.

We did use to do backups, sadly the last one was done over a month ago, so anything we did within that time was lost, so all my main new code within that time for Blacknova Realms was lost. I do have different versions here locally, just not an up to date version due to I was editing and compiling code remotely.

Sooooo, I now need to see how old the local versions are and work from that, I know I left some sticky notes here in the change of direction on the code.

Server Console - Screen Buffer

📆 Post by TheMightyDude, Thu, June 14, 2018

Development Planning Feature Server Linux

Well been looking into the server side of things so it will be written for a Linux Server.

I am aiming to have the server application as a console-based application so that the admin can run it via a remote terminal.

So, I really want it to display a title, body, footer and a section where you enter admin commands.

So, I had to create my own Screen Buffer code which allows me to create segments on the screen and display information on just that segment.

Sure, I could have looked for an already created one and hope for it to be free, but then I have to hope that developer maintains it, I have been burnt too may times "ADODB" etc.

Granted its not perfect, it might have some issues, but I haven't seen any yet.

Network Update 6

📆 Post by TheMightyDude, Sat, April 14, 2018

Development Planning Feature

Well the new version of the Network Library is starting to look rather promising so far, I have got it to a stage where it was when I was adding support for different METs (Message Event Type) like ConnectEvent, DisconnectEvent, DataEvent, ServerNoticeEvent etc.

So far, the very basic server network code is about 40KB in size and the client network code is also about 40KB in size.

So, I am currently working on the MessageManager which works out the type of message and sends it off to the required handler and queues it up.

Each handler processes their own queue on its own thread so that it doesn't slow anything game wise down, so all this is done Asynchronously.

Once this Message Manager is complete I will start on the following Stages:

That's it for now.

Network Update 5

📆 Post by TheMightyDude, Thu, March 22, 2018

Development Planning Issues

Well I seem to of hit a brick wall again to do with the Networking code that I have been creating.

It seems that the way that I have designed it makes it a tad bit difficult to do some lookup stuff.

And having to go through loads of code which needs to happen every threaded network loop, this is causing a CPU performance hit which is impacting the number of connected users that the game would be able to support.

So, I have once again stepped back to see what every part needs to do and I think I might have it this time.

I know the Threading side of the code is fine, so it just falls down to the actual networking side that is the issue.

It seems that I might have just over complicated things with my code an that simplifying it might end up being better.

But we will see.

Data Storage [server side]

📆 Post by TheMightyDude, Wed, January 17, 2018

Development Planning

I am sorry I have neglected this blog section for a while, TBH I had forgotten about it due to on and off ill days.

I had been working on Octree and Quadtree stuff last year, but where its going to be like the current BNT for this version there was no need to use those.

I will be using HashSet<...> to store the information which does take a little more memory than actual List<...> but the lookup is supposed to be faster when using HashSet<...> so I will try those.

At the moment the default storage size for a Universe consisting of 2,500 Sectors is currently taking about 36 MB of RAM and 35,000 Sectors takes about 53 MB of RAM.

Out of curiosity I tried 100,000 Sectors an it took about 94 MB of RAM, so not too bad.

I am also still working on the Database side of things, changing stuff here and there.

I will try to add more stuff here.

Planting an Octree [server module]

📆 Post by TheMightyDude, Tue, July 25, 2017

Development Planning

Well I have been looking into Octree's for the last couple of weeks now and I think I am starting to understand it.

So, I will be using Octree's to store all objects that have a 3D location in space like Ships, Stations, Asteroids, Planets, Moons and so on.

Hopefully this will all help to reduce all the database calls etc due to its all loaded in RAM at runtime and we only update the database if anything changes in RAM and which it checks every X mins (15 mins by default).

Apart from that not much has happened.

Network Server Update [small update]

📆 Post by TheMightyDude, Mon, July 03, 2017

Development Planning

Another short update...

Cleaned up some of the Network Server code, removed redundant code.

Added in support for message handlers, so now we can add in different message handlers, like PING, Chat Messages, Movement, Combat and so on.

Also optimized the network code to do stuff when it's really required to do so.

Put in place holders for future stuff.

Well that's it for now, well I did say it was a short update.

Network Update 4

📆 Post by TheMightyDude, Thu, June 15, 2017

Development Planning

Well, due to not touching the code for a while because of RL stuff, most of the code was stopped in mid coding, so wouldn't compile.

So, I decided on stepping back to see what I really wanted the Networking code to do.

I have now got the Client and Server Network Code to both support Multiple Threads, so it can now support loads of Threads for the NetworkLoop, ReadQueue, WriteQueue, so those sections should never bog down the rest of the server's code.

Using my own Threads takes the Networking Side of the code away from the rest, i.e. Unity for the Client side and the Linux Server code.

A lot of the stuff is hardcoded in, so that will need to be changed.

At this point of time I am not sure on what exact amount of CCU (Concurrent Connected Users) this code will be able to handle, but I am sure we won't be reaching it for a while, I am aiming for 1,000 to 2,000 CCU players, but we will see how it goes.

And then look into creating Node Servers, which will be a learning curve for me.

So, so far the Client can connect to the Server, and they can send and receive Network Messages (formatted data) back and forth, this data is meaningless at this stage, but both ends can extract the individual messages and process them, so it is still work in progress.

Back on Track (sort of)

📆 Post by TheMightyDude, Wed, June 14, 2017

Information

Sorry for being a bit A.W.O.L, health issues happened again to-do with the hot weather here at the moment in the UK, as well as several Hospital Appointments going on all over the place.

I am still looking into different ways to re-write Blacknova Traders.

Like I have already said before on the forums, it being a browser based game really does limit what you can add to the game due to it would be a Request and Response type game, so the game wouldn't be responsive enough to add all the features I wanted to.

Now a full-blown TCP or UDP Socketed type game would allow me to add all the features that I wanted plus more and also makes it more future proof.

Now the status of development and the Dev Team, as far as I am aware we haven't stopped work on the new game, whether it will be just a complete recode from scratch or a completely new concept based on BNT, we have yet to discuss this, this is however possibly due to start sometime next year.

Network Update 3

📆 Post by TheMightyDude, Mon, March 20, 2017

Development Planning

Well I have just done some more of the Client and Server Network Code, so we can now send and receive basic hardcoded Network Messages in a non-secure manner.

It's still not perfect, but we can send and receive stuff now :)

I think some of the code may need to be changed to support the Secure SSL / TLS Mode, but I assumed that would be the case.

Apart from that, not much has gone on.

Network Update 2

📆 Post by TheMightyDude, Thu, March 16, 2017

Development Planning

Just a short quick update this week.

I have got about 60% of the Client Network code done so far, the remaining 40% should get done when I complete the last parts of the Server Network code.

As for the last blog "Network Update 1" to-do with the secure side, I now have two options, have it like I have said using HTTPS for just the secure side like logging in, or have a separate service for just the authentication stuff and services for the actual Game Lobby (Server List / Character Select) and the actual Game Server.

I might combine the Game Lobby Service with the Authentication Service for simplicity.

So, to confirm what's "hopefully" going to happen...

1: Create a Secure (SSL / TLS) Service for Authentication and Lobby area.

2: Create a Service for the actual Game.

3: Create a Client that first connects to Authentication Service and if valid login send Lobby Information.

4: If Valid login and Game Service selected in the Lobby Service, connect to selected Game Service.

So it's one step closer :)

Network Update 1

📆 Post by TheMightyDude, Sat, March 11, 2017

Development Planning

Well a bit late adding this blog, been having some bad days, so haven't been in a mood to do coding.

I am still working on the Networking side of the code where I have had to code from scratch a TCP Server Code that "should" handle thousands of connected users using multiple threads.

I have started this code just as a backup in case the NetworkTransport which is built into the Unity Engine has any limitations or any future restrictions like them requiring a subscription to use their networking library.

Plus, in my code I have more control over how it works.

I am currently working on its Message Queuing System which is about halfway done atm.

I am still on the fence whether to use SSL/TLS over TCP Sockets for everything other than just for authentication.

The issue with using SSL/TLS over a TCP Socket is that once you start it you have to keep using it and using SSL/TLS for every network message would impact the server resulting in supporting less connected players.

So, if we want to only use SSL/TLS for authentication we would have to use Secure WebSockets (i.e. HTTPS) for the handling all of the logging in and normal TCP Sockets for the actual game.

Also, using Secure WebSockets it makes it easier to allow for 3rd party servers if I decide to release the server binaries (Linux using Mono) later on, leaving just the Authentication side being run by us.

So, to answer some of your questions that you sent via email, yes, you create your account on our servers and you then use the Game Client to connection to our network where you will / should be presented with a list of servers to play on.

Game Servers will not store any private information like email address, passwords etc, only we store that encrypted.

The Game Servers request for a hash which is created just for that server, so a player can play on different servers and have all different reference in that server's database.

Once this Server Code is complete I will be working on the Client Network Code and once that has completed I would be at a point where I can start adding game stuff in.

Well that's it for this week.

Features

📆 Post by TheMightyDude, Mon, February 27, 2017

Feature Planning

Space to planet surface transitioning
This will allow for the player controlling his ship to fly about (like Elite, No Mans Sky) and then head for a planet and fly through the atmosphere of the planets right down to the planet surface where you can fly and land.

This will be all done at run time, we are still trying to get this to work, its partly done.

Planets
Now here is the killing blow, players will not own planets, end of.

There is nothing stopping players build bases, structures on a planet, but planets are mostly there for resource gathering and some combat missions.

Database Intergration.

📆 Post by TheMightyDude, Mon, February 27, 2017

Development

I thought I would give another update on how things are going.

At the moment things are going rather well now, but I did have some issues a few days ago, when I tried to implement database layer to the server code.

Well as I said in my last development blog I was going to use Unity's LLAPI (Low Level API) Networking which uses UDP Sockets which means fast transferring of network traffic.

So, that was going very well so I thought ok, now to add the database side of things to the server code, now this is where things went screwy, I downloaded MySQL Connector for Mono & .NET which Unity imported fine, now when I tried to use it I got some errors to do with missing references so I resolved that issue to then hit another.

I was almost ready to just give up on everything when I started playing about where I then got MySQL working fine, I thought great, but what other Databases it could use, so at least I would have a choice which one to use if the other proved problematic down the line.

So, after several hours faffing about I can use either MySQL or SQLite.

There are some pros and cons with both of them.

The database is only used for the initial loading of contents into physical memory and the server then only uses the content stored in memory instead of doing loads of queries resulting in a HUGE performance boost.

The server updates the database every X (15 mins by default) mins with the live data it its own Thread so that it doesn't impact the server's performance.

This is how most MMO's do it, so we are on the right track here :)

Long delayed update...

📆 Post by TheMightyDude, Mon, February 20, 2017

Development Issues

Well I started work on my Network Library a couple of weeks ago, that will be used for all the WebRequests which used Threads, Message Queues etc, this seemed to work great.

This was around the time I got an Eye Infection, so there was me trying to use my PC with a one of my eyes completely blood shot and un able to focus onto anything without pain, so this slowed me down somewhat, hence me not being online much.

Anyway, I thought I would add the next stuff to the library which was the background requests that get called every X seconds for any updates that might have happened between the player clicking on stuff.

This is to support features that notify users that haven't clicked on anything for a while, like for example a friend logs in and also when you are being scanned and or attacked, it will pop up a notification window or change to a combat scene when being attacked.

Anyway, I added in this background requests which seemed to work ok, well until I simulated more and more players playing and also running the background requests, I actually managed to kill Apache twice, MySQL wasn't too happy either but I could resolve that with SQL changes.

Which leaves me to the Apache Issue...

I really want these features to be added, sadly if it's going to result in Apache locking up and crashing due to an over load in requests per second, then I am then going to have to find another route to take.

I am already currently using the Unity Engine for the client side application on the PC and later on Linux, Mac and possibly Tablets depending on the screen resolution.

Anyway, since Unity is being used for the game client I thought why not write the server using Unity as well.

So, I started coding another Network Library from scratch, oh what fun that was, Threads not shutting down correctly resulting in me creating a Thread Manager to manage all the used Threads and so on.

I got about a third of the way through it when I thought about looking at the built-in versions of networking that come built into Unity.

So, I checked the easiest version first, had too many restrictions like the maximum amount of ccu (concurrent connected users) players and that was very low, so I checked the new HLAPI (High Level API) was is still being worked on, sadly like the first it also has the same ccu restrictions and to overcome those restrictions I would have to charge a monthly rate to pay for those services.

But due to not really wanting to make the game a subscription based game to cover the costs and also wasn't really too sure on how many people will even play the game to cover the costs, then looked into the LLAPI (Low Level API) which also uses UDP Packets just like the other types that I tried before this one. But this one you have no restrictions as far as I can tell so far, the only restrictions I can see are there will be loads more coding to be done which isn't too bad and the maximum ccu was 65,535 a lot better than the 20 odd that you get on the other two.

Now if I was ever lucky enough to have a game that had that number of connected players I would probably have some income whether it's by micro-transactions or nicely placed ads etc, I would just setup another server to run a new universe.

So here is the gotcha, the only people that would be able to setup a server would need to have root access and own the entire root server, so no cheap GoDaday hosts.

I know this isn't what you really wanted to hear, but I really did try the easy routes first, sadly this doesn't work out how we wanted it, to be honest I am glad I found all this out now then a year or two down the line.

I will try and keep this game as close to as possible as the current BNT, but "some" stuff will be changed, we have to be realistic here, there is no way in hell in you getting millions for fighters in your little ship: P

Well I think this individual blog has gone on long enough this time, so I will be cutting it here.

Like as always, if you have any questions, just email me or post on the forums.

Laters.

TMD

More Serialization

📆 Post by TheMightyDude, Thu, January 26, 2017

Development

Well had to redo all my JSON Serialization code due to compatibility issue, which worked fine on Windows but had issues on Linux, Mac etc.

So, I thought I would try the built-in version, which works, however it requires loads more code to work and wasn't compatible with what I already had coded so far.

So, for the Client to send a request to the server in a JSON formatted request, I have to create a class container which is more work to do, but it works and doesn't require a 3rd party library.

So, like I said before I will be creating a Windows version first and then try working on the other platforms (Linux, Mac, Tablets), sadly Smart Phones will not be supported due to the small screen size.

Serialization

📆 Post by TheMightyDude, Thu, January 12, 2017

Development

Well not done much as late apart from working on JSON Serialization which will be used for all of the Locale (English only for now) and Configuration files, its not perfect but it works.

JSON Serialization is also going to be used to format the information between the Client and Server.

Like I said I haven't really done much since the last blog.

Hopefully more will get done over the next few weeks.

Just a quick update.

📆 Post by TheMightyDude, Mon, December 26, 2016

Development Planning

Hey all.

Just thought I would say Merry Christmas to everyone.

Ok, I have been doing some huge thinking on ways to do this game and have come up with this.

And if we are support loads of players we are going to have to reduce the amount of MySQL Queries, so we will need to do some caching of some sort of our database records.

I looked into Memcache, Memcahed, MySQL Caching and so on, all having limitations with either the amount of data as well as the size of each data, so it seems that caching is out of the question here.

I also looked at MySQL's Memory Engine which by reading it sounded exactly what I wanted, sadly the performance was either the same as the InnoDB Engine or it was slightly slower which I thought was strange, however I may come back to try this again later on.

Now knowing the above I started looking into tmpfs and ramfs for a type of caching and for simplicity I chose tmpfs.

Using tmpfs will allow me to create a filesystem in a chunk of allocated ram, so it will be fast at reading and writing, which is exactly what I wanted.

So basically, every record in the database will be stored as a file in file folder fashion for simplicity,

So, when the server is started it loads all the required records from various tables and stores them individually in the required folders on the tmpfs partition and the game server will use that copy of the data.

There will be a scheduled background task that runs a Script that locks a record (i.e. tmpfs file), reads its content and if it's time to update then [update the database with its content], unlocks record (i.e. tmpfs file) move to next record and so on, this is done every 10 or so mins, so if the server crashes and needs rebooting all that will be lost is 10 or so mins of play.

I tried it last night loading up all 161 ship accounts that I had in the previous game that was run and it was fast, so this might be the right direction.

I was having some thoughts about lookups that return multiple records, like returning all the planet records that are located in sectors 35 to 97, in a SQL Query that would be simple, whereas my way I would have to for loop through folders to get the same result, it is doable, but I just wasn't expecting it that's all.

Well that's it for now, I hope you all have a nice new year.

2 Pass Verification (part 2)

📆 Post by TheMightyDude, Tue, November 22, 2016

Feature Security

Well I had a play with the 2 Pass Verification and it didn't go as planned.

It seems that the code that I got from GitHub is using Google's charts API for the QR Code generation which seems to be deprecated and on top of that the code was spread over loads of files, where as the new code I have found is all stored in a single file and is also easy to follow in the code.

So I looked else where and found some code that can create a QR Code either using a table (YUCK) and each cell is a pixel, or it uses the GD library to generate an image.

I have tried two Authenticators (The Google Authenticator and the Microsoft Authenticator) and both tests are the same; they both display a 6 digit code that changes every 30 seconds.

So I might use that code for the QR Code Generation along with my own TOTP (Time-based One-time Password Algorithm) code.

2 Pass Verification

📆 Post by TheMightyDude, Sun, November 20, 2016

Feature Security

Well I have been thinking about this for a while now, but due to the current security issues I have seen on Sony's PSN where users have had their accounts hacked and Sony refusing to resolve the issue, I have therefore looked into a 2 Pass Verification.

I will be using Google's Authenticator for this feature.

At this point I haven't tested it yet, but I am sure it will be fine and work :P

Added DB Support for blogs

📆 Post by TheMightyDude, Sat, November 12, 2016

Feature

Well I finally got around to added database support for the blogs, so now they are stored in the database.

So this will help edit blogs and support tags etc.

There be code here captain

📆 Post by TheMightyDude, Thu, November 10, 2016

Development Planning

Well I thought it was time to setup a development blog, so that I can show how the progress is getting on. There may be some images, YouTube Videos and so on, but at this current time I am just in the planning stage, basically seeing what works and what doesn't.

At this current stage I am playing about with Smarty Template, Bootstrap 3, JQuery just to see how they work together. They seem to work ok, sadly there is an issue, Smarty Template can only load the page once, basically it loads it the once and if you want to change a huge chunk of its content the page needs to be reloaded.

Now by using Smarty Template we are able to parse information to the template file to only display certain information, like only display the admin section when an admin logs in and so on.