Development Blog v0.20.15
Well I have been looking into Octree's for the last couple of weeks now and I think I am starting to understand it.
So, I will be using Octree's to store all objects that have a 3D location in space like Ships, Stations, Asteroids, Planets, Moons and so on.
Hopefully this will all help to reduce all the database calls etc due to its all loaded in RAM at runtime and we only update the database if anything changes in RAM and which it checks every X mins (15 mins by default).
Apart from that not much has happened.
Another short update...
Cleaned up some of the Network Server code, removed redundant code.
Added in support for message handlers, so now we can add in different message handlers, like PING, Chat Messages, Movement, Combat and so on.
Also optimized the network code to do stuff when it's really required to do so.
Put in place holders for future stuff.
Well that's it for now, well I did say it was a short update.
Well, due to not touching the code for a while because of RL stuff, most of the code was stopped in mid coding, so wouldn't compile.
So, I decided on stepping back to see what I really wanted the Networking code to do.
I have now got the Client and Server Network Code to both support Multiple Threads, so it can now support loads of Threads for the NetworkLoop, ReadQueue, WriteQueue, so those sections should never bog down the rest of the server's code.
Using my own Threads takes the Networking Side of the code away from the rest, i.e. Unity for the Client side and the Linux Server code.
A lot of the stuff is hardcoded in, so that will need to be changed.
At this point of time I am not sure on what exact amount of CCU (Concurrent Connected Users) this code will be able to handle, but I am sure we won't be reaching it for a while, I am aiming for 1,000 to 2,000 CCU players, but we will see how it goes.
And then look into creating Node Servers, which will be a learning curve for me.
So, so far the Client can connect to the Server, and they can send and receive Network Messages (formatted data) back and forth, this data is meaningless at this stage, but both ends can extract the individual messages and process them, so it is still work in progress.
Sorry for being a bit A.W.O.L, health issues happened again to-do with the hot weather here at the moment in the UK, as well as several Hospital Appointments going on all over the place.
I am still looking into different ways to re-write Blacknova Traders.
Like I have already said before on the forums, it being a browser based game really does limit what you can add to the game due to it would be a Request and Response type game, so the game wouldn't be responsive enough to add all the features I wanted to.
Now a full-blown TCP or UDP Socketed type game would allow me to add all the features that I wanted plus more and also makes it more future proof.
Now the status of development and the Dev Team, as far as I am aware we haven't stopped work on the new game, whether it will be just a complete recode from scratch or a completely new concept based on BNT, we have yet to discuss this, this is however possibly due to start sometime next year.
Well I have just done some more of the Client and Server Network Code, so we can now send and receive basic hardcoded Network Messages in a non-secure manner.
It's still not perfect, but we can send and receive stuff now :)
I think some of the code may need to be changed to support the Secure SSL / TLS Mode, but I assumed that would be the case.
Apart from that, not much has gone on.
Just a short quick update this week.
I have got about 60% of the Client Network code done so far, the remaining 40% should get done when I complete the last parts of the Server Network code.
As for the last blog "Network Update 1" to-do with the secure side, I now have two options, have it like I have said using HTTPS for just the secure side like logging in, or have a separate service for just the authentication stuff and services for the actual Game Lobby (Server List / Character Select) and the actual Game Server.
I might combine the Game Lobby Service with the Authentication Service for simplicity.
So, to confirm what's "hopefully" going to happen...
1: Create a Secure (SSL / TLS) Service for Authentication and Lobby area.
2: Create a Service for the actual Game.
3: Create a Client that first connects to Authentication Service and if valid login send Lobby Information.
4: If Valid login and Game Service selected in the Lobby Service, connect to selected Game Service.
So it's one step closer :)
Well a bit late adding this blog, been having some bad days, so haven't been in a mood to do coding.
I am still working on the Networking side of the code where I have had to code from scratch a TCP Server Code that "should" handle thousands of connected users using multiple threads.
I have started this code just as a backup in case the NetworkTransport which is built into the Unity Engine has any limitations or any future restrictions like them requiring a subscription to use their networking library.
Plus, in my code I have more control over how it works.
I am currently working on its Message Queuing System which is about halfway done atm.
I am still on the fence whether to use SSL/TLS over TCP Sockets for everything other than just for authentication.
The issue with using SSL/TLS over a TCP Socket is that once you start it you have to keep using it and using SSL/TLS for every network message would impact the server resulting in supporting less connected players.
So, if we want to only use SSL/TLS for authentication we would have to use Secure WebSockets (i.e. HTTPS) for the handling all of the logging in and normal TCP Sockets for the actual game.
Also, using Secure WebSockets it makes it easier to allow for 3rd party servers if I decide to release the server binaries (Linux using Mono) later on, leaving just the Authentication side being run by us.
So, to answer some of your questions that you sent via email, yes, you create your account on our servers and you then use the Game Client to connection to our network where you will / should be presented with a list of servers to play on.
Game Servers will not store any private information like email address, passwords etc, only we store that encrypted.
The Game Servers request for a hash which is created just for that server, so a player can play on different servers and have all different reference in that server's database.
Once this Server Code is complete I will be working on the Client Network Code and once that has completed I would be at a point where I can start adding game stuff in.
Well that's it for this week.
Space to planet surface transitioning
This will allow for the player controlling his ship to fly about (like Elite, No Mans Sky) and then head for a planet and fly through the atmosphere of the planets right down to the planet surface where you can fly and land.
This will be all done at run time, we are still trying to get this to work, its partly done.
Planets
Now here is the killing blow, players will not own planets, end of.
There is nothing stopping players build bases, structures on a planet, but planets are mostly there for resource gathering and some combat missions.
I thought I would give another update on how things are going.
At the moment things are going rather well now, but I did have some issues a few days ago, when I tried to implement database layer to the server code.
Well as I said in my last development blog I was going to use Unity's LLAPI (Low Level API) Networking which uses UDP Sockets which means fast transferring of network traffic.
So, that was going very well so I thought ok, now to add the database side of things to the server code, now this is where things went screwy, I downloaded MySQL Connector for Mono & .NET which Unity imported fine, now when I tried to use it I got some errors to do with missing references so I resolved that issue to then hit another.
I was almost ready to just give up on everything when I started playing about where I then got MySQL working fine, I thought great, but what other Databases it could use, so at least I would have a choice which one to use if the other proved problematic down the line.
So, after several hours faffing about I can use either MySQL or SQLite.
There are some pros and cons with both of them.
The database is only used for the initial loading of contents into physical memory and the server then only uses the content stored in memory instead of doing loads of queries resulting in a HUGE performance boost.
The server updates the database every X (15 mins by default) mins with the live data it its own Thread so that it doesn't impact the server's performance.
This is how most MMO's do it, so we are on the right track here :)
Well I started work on my Network Library a couple of weeks ago, that will be used for all the WebRequests which used Threads, Message Queues etc, this seemed to work great.
This was around the time I got an Eye Infection, so there was me trying to use my PC with a one of my eyes completely blood shot and un able to focus onto anything without pain, so this slowed me down somewhat, hence me not being online much.
Anyway, I thought I would add the next stuff to the library which was the background requests that get called every X seconds for any updates that might have happened between the player clicking on stuff.
This is to support features that notify users that haven't clicked on anything for a while, like for example a friend logs in and also when you are being scanned and or attacked, it will pop up a notification window or change to a combat scene when being attacked.
Anyway, I added in this background requests which seemed to work ok, well until I simulated more and more players playing and also running the background requests, I actually managed to kill Apache twice, MySQL wasn't too happy either but I could resolve that with SQL changes.
Which leaves me to the Apache Issue...
I really want these features to be added, sadly if it's going to result in Apache locking up and crashing due to an over load in requests per second, then I am then going to have to find another route to take.
I am already currently using the Unity Engine for the client side application on the PC and later on Linux, Mac and possibly Tablets depending on the screen resolution.
Anyway, since Unity is being used for the game client I thought why not write the server using Unity as well.
So, I started coding another Network Library from scratch, oh what fun that was, Threads not shutting down correctly resulting in me creating a Thread Manager to manage all the used Threads and so on.
I got about a third of the way through it when I thought about looking at the built-in versions of networking that come built into Unity.
So, I checked the easiest version first, had too many restrictions like the maximum amount of ccu (concurrent connected users) players and that was very low, so I checked the new HLAPI (High Level API) was is still being worked on, sadly like the first it also has the same ccu restrictions and to overcome those restrictions I would have to charge a monthly rate to pay for those services.
But due to not really wanting to make the game a subscription based game to cover the costs and also wasn't really too sure on how many people will even play the game to cover the costs, then looked into the LLAPI (Low Level API) which also uses UDP Packets just like the other types that I tried before this one. But this one you have no restrictions as far as I can tell so far, the only restrictions I can see are there will be loads more coding to be done which isn't too bad and the maximum ccu was 65,535 a lot better than the 20 odd that you get on the other two.
Now if I was ever lucky enough to have a game that had that number of connected players I would probably have some income whether it's by micro-transactions or nicely placed ads etc, I would just setup another server to run a new universe.
So here is the gotcha, the only people that would be able to setup a server would need to have root access and own the entire root server, so no cheap GoDaday hosts.
I know this isn't what you really wanted to hear, but I really did try the easy routes first, sadly this doesn't work out how we wanted it, to be honest I am glad I found all this out now then a year or two down the line.
I will try and keep this game as close to as possible as the current BNT, but "some" stuff will be changed, we have to be realistic here, there is no way in hell in you getting millions for fighters in your little ship: P
Well I think this individual blog has gone on long enough this time, so I will be cutting it here.
Like as always, if you have any questions, just email me or post on the forums.
Laters.
TMD
Well had to redo all my JSON Serialization code due to compatibility issue, which worked fine on Windows but had issues on Linux, Mac etc.
So, I thought I would try the built-in version, which works, however it requires loads more code to work and wasn't compatible with what I already had coded so far.
So, for the Client to send a request to the server in a JSON formatted request, I have to create a class container which is more work to do, but it works and doesn't require a 3rd party library.
So, like I said before I will be creating a Windows version first and then try working on the other platforms (Linux, Mac, Tablets), sadly Smart Phones will not be supported due to the small screen size.
Well not done much as late apart from working on JSON Serialization which will be used for all of the Locale (English only for now) and Configuration files, its not perfect but it works.
JSON Serialization is also going to be used to format the information between the Client and Server.
Like I said I haven't really done much since the last blog.
Hopefully more will get done over the next few weeks.