WorldSpawn official WIP thread

[Dutch]

Hahaha clever.

The other day, I had a co-worker ask me how I got into the machining department, if I had to have a degree or formal education or what not. I told him I got my degree at the school of hard knocks. Lol. I actually do have a degree, but nothing related at all to machining... more on the lines of livestock science and management. It was a big twist of events that got me into the trade, learned it all on the job. Had some good people around to show me the ropes...

There are generally two impressions of machinists, which I find funny as hell. 1) "What's a machinist? Do you fix cars or something?" and 2) "You're an arrogant bastard."

Who can blame the second reaction with stuff like this floating around





No LinkedIn though, sorry.

[MadGypsy]

@safety goggles...you mean eyelids

About 20 years ago I started saying

Safety Goggles? You mean eyeBALLS? LOL! I never even thought of the eyelids. About the only "safety" gear I wear is gloves and really it has nothing to do with safety. I get sick of picking little splinters out of my fingers and in non-wood situations (like cranking a wrench) I get sick of the bolt finally breaking and me punching some other part with a bare fist. It's no so much about the pain it's more about not tearing up the back of my hand. Oh and a particle mask...I'm not worried about hurting myself in this world but I'm not trying to painfully die over a bunch of years from mesothelioma or something like that. There's pretty much nothing I can do to myself that isn't going to heal but, I'm not trying to get a disease. Lead poisoning would be another example. When dealing with sanding lead paint every square centimeter of my body is protected and that is probably the only time where I am a paranoid safety nut. I've had lead poisoning before and it really sucks. Generally when dealing with lead I also work (if possible) as long as it takes to finish the job in one shot and then I throw away everything I was wearing. I charge a first born child for lead abatement. I'm not crying about throwing away $100 in clothes and shoes. I've charged as much as $7000 just to sand the lead off of basically standard sized new orleans home. That's before even talking about the price of the entire rest of the paint job. Literally $7000 in 3ish days of work. BUT I treat the home like I treat myself. All windows, doors, etc are sealed (tape), entire house is tented, etc, etc. Even the ground is overly covered so I can roll it all up and throw it ALL away. The house tent is literally the only thing that doesn't get tossed. It also never gets cleaned. Cause how? Where is all that shit gonna go? Hose it off? to where? Just bang it out as good as possible onto the ground tarps, roll it up and bag it til the next job.

I know you can buy vacuum grinders but back when I was sanding houses those were pretty much just coming out (with anything resembling quality) and they were crazy expensive. I'm sure the prices have come down a lot since but back then I would have had to make my already ridiculous price even higher. Also, back then you practically had to dedicate the entire bed of your truck to the vacuum. That wasn't even an option.

[MadGypsy]

I'm sure the grand majority of people probably think this project is dead. It isn't and it never was. I started this project in Flash with a "can I get anything to work?" mentality. I eventually moved it to Haxe and made a bunch of things better. I then ported that over to flash and added a shit load of new features and made the code even better. (We have to keep in mind that this is not just a game engine. I made numerous API's that can be used for the game engine but are versatile enough to be used in other application-based projects.) Then came the big port of 60,000 lines of code back to haxe. I have gone over that code with a fucking lice comb. Every last thing has been considered and reconsidered to a point of absolute portability and reusability.

My current focus is writing better "bottom level" scripts. I can give a few examples.

example 1 - Bytes:
In flash (As3) Bytes are called ByteArray and everything you can think of to do to a bytearray is a public method. Like, ba.readUnsignedShort(). In HaXe it is not this simple. There are numerous ways to go but, the most platform independent way is to put Bytes into a BytesInput (reading) or BytesOutput (writing). This means, instead of simply reading or writing to a solo bytearray, an input and output need to be managed. This has been happening all over my content manager as necessary.

I wrote my own byte class that acts much like ByteArray. It does some extra things that ByteArray doesn't do (UInt8Array, auto-trim/pad null) but, they are more or less the same thing. The thing is platform independence. My class has that. Ultimately, instead of managing BytesInput and BytesOutput, all over the place, there is just ByteStream.

example 2 - typedef/structs

Typedefs are great until you need to key the struct using it...cause you can't key any object. This means Reflection is the only way to essentially "key" a field of an object. This is not desirable. Trying to manipulate objects with reflection is a nightmare, especially if a field is optional. Structs (in Haxe) are good if you intend to hard code every possibility of the struct.

ie

Mystruct.field.name = value

That is not desirable. I could write a program that knows what to do with far less specificity. Doing that with Reflection would be this nightmare

if(Reflect.hasField(Mystruct, "field"))
Reflect.setField(Mystruct.field, "name", value))

loops of stuff like that. I need to extend structs to be keyed. This means I need to write some bad-ass abstract. It needs to be platform independent and it can't just be some Reflection manager. No reflection. No "struct-to-map".

---

The cleaner I make this "low level" stuff, the easier it will be to build on. There are other things like the above. Some of it will get corrected by doing the above things.

Anyway, my engine is not a dead project.

[MadGypsy]

I did what I said and wrote an abstract that lets me key structs (Fields<T>).


It enforces whatever type you assign it. SomeType requires a name and an age field. I took the age field out of the struct that I passed and error checking caught it.


This means I don't have to write a bunch of - Reflection.hasField(object, field), field(...), setField(...), etc. Now I need to go upgrade more stuff than I would like to have to.

[MadGypsy]

This low level stuff I have been working on is awesome. Between my new byte classes and my abstract I have removed approx 1200 lines of code (within approx 60k). All of those lines were preliminary lines of code necessary to setup the data in one way or another. My code removes the need for all of the preliminaries. This means the data is much easier to work with and the code is much easier to read.

I actually wrote 3 byte classes. 1 is the main byte manager and isn't intended to be used directly. The other 2 are ByteStream and ByteBlock which both extend the byte manager.

ByteStream allows you to read and/or write bytes with various inline methods like readUnsignedShort(), readByte(), etc. It also automatically handles positioning of the pointer, padding and trimming null. This is very handy. In a read sense I could

ex:
bytes.readString(16)
and all null will be removed from the return value

in contrast I can ex:
bytes.writeString("myString", 16)
and the value will be null padded for me

Unlike ByteStream, ByteBlock works with allocation of expected bytes and allows pos/len writes. It has a unique feature that makes writing lumps easy. Every time .alloc() is called, any bytes that are currently allocated get dumped to the end of an internal output stream. This means you can call alloc(len) in a loop and have each new iteration of the loop fill the current allocation. When the loop is done you simply call commit() (to dump the last allocation to the output stream) and the final output stream will contain all of the bytes, from all allocations, in the order you wrote them.

I want to write one more class for bytes. ByteStreamArray - mostly this would just be Array<ByteStream> but it will have some public methods to merge, sort, etc the streams.

I'm still working on the upgrading of all my sources to the new byte methods and object syntax. I have done a whole lot and I don't know how much further I have to go. I know how many classes are left but, I don't know how much (if any) changes need to be made to those classes.

For anyone that cares, this is what an abstract looks like. Unfortunately, I was unable to forego Reflection but, this is still better than having to write all this Reflect stuff directly in my scripts. This is my first Haxe abstract. I'm going to give it another shot in the future and try to eliminate reflection. Haxe abstracts are super powerful and there is a lot of various things you can do with them. I still have a bit to learn before my abstracts are amazing. Really, at this time, I am just glad I got one to work at all. It would probably make more sense to name this ReflectMap.



Overall this is pretty simple. I create an abstract named Key that expects a currently unknown type <T> and will be considered of type Dynamic at runtime. All typedefs are Dynamic in an underlying sense so this works well.

Key<T>(Dynamic)

The only real key thing here is the constructor. A null value is acceptable for input but not acceptable for the final "this" value. If I let "this" be null then every method would have to check if "this" is null before doing what it currently does. It's easier to simply check null in the constructor and tell "this" to be an empty object instead of null.

The rest is little more than a wrapper for Reflect. It isn't a complete one though. There are reflection methods that are pretty useless to this which I did not include. The simple forEach method is not a part of Reflect but, I felt it was handy for concocting loops.

[MadGypsy]

Here is a little side by side of the difference my little abstract makes.



The left side is obviously using reflection directly and the right is using my abstract. Both scripts do the exact same thing but, my abstract version does it with a lot less work and repetition. Also, my forEach method slims looping down a bit.

I personally believe that recognizing things like these and doing something about them is just as important as writing good code to accomplish whatever your goal is. I could have gone on forever simply accepting that Reflect is the only way to do "it" and, as you can see from above, I would have spent a lot of time writing the same thing over and over. The same thing applies to my byte classes.

I intend to spend more time recognizing places where the language is not very user friendly and doing something about it.

[MadGypsy]

I have an idea but, there are some problems that need to be solved.

I wrote my expression parser specifically to allow stringified math expressions to be used anywhere a number is expected. But...


In haxe typedefs are strong/strict typed so, let's say I had a typedef that explains a year.

typedef Year_t =
{
var months:Int;
var weeks:Int;
var days:Int;
}

But, let's say that for whatever reason I wanted to assign the days as a math expression ("52 * 7 + 1"). The problem here is that expression is a String but days field in the typedef expects Int.

There is one work-around but it is only good for dynamic targets (python, javascript, etc). I could do this:

typedef Year_t =
{
var months:Int;
var weeks:Int;
var days:EitherType<Int, String>;
}

But, as I just said, that is only good on dynamic targets and actually specifically discouraged for anything else. This leaves me with 2 more solutions. I could write my own abstract (yay! more of that shit) or I could simply do it the lazy way

typedef Year_t =
{
var months:Int;
var weeks:Int;
var daysynamic;
}

I'm not sure what to do about that yet and none of that has much to do with my idea . My idea is quite simple. I want to make my Key abstract smarter. I want it to be able to determine on assignation whether a string value should be parsed to a number and auto do it. I have numerous spots in my code with the same 2 lines checking a field and evaluating it to a number, if necessary. I want to completely remove all of those spots and never look back.

I THINK the way to do this is actually to change my expression parser to determine if something is math or numbers, at all. Then I could simply eval every string. Alternately, I could have my Key abstract determine if something is math or numbers and only send strings to eval() if it is. That seems weird to me though. My expression parser should know if something is math or numbers, not my Key abstract.

Yep. Expression parser needs to be made smarter first, then make the Key abstract use it for every string. I like that. Not only do 2 of my "classes" get an upgrade they become useful to one another, as well. My expression parser is entirely built on RegEx. Probably the easiest way to determine if something is math would be to concat all of my RegEx statements and use it for a search of the string. I'm sure it will become more complicated than that but, that's where I intend to start. I can already guess the complication.

"This sentence is so random"

random found - this must be math

"Is it really a sin to lie?"

sin found - must be math

The above illustrates the real problems I need to solve. However my "sin" example would not be a problem cause it definitely isn't a "sin()" to lie and that's what my parser would actually be looking for. I could just send strings through the parser as is and anything that is not valid math will be returned as NaN but, the idea is to catch not-math stuff before trying to parse the crap out of it.

---

When I was about 12 years old my Dad let me drive the car. It was night time and we were on a highway that was poorly lit with nothing but yellow dashes on the road separating oncoming traffic. He pointed in the distance and said "That guy doesn't have his headlights on." He realized that I could not spot this vehicle so he said "Look for the absence of light." I understood that he meant look for the absence of light in this world and I immediately saw the gaping black void that was shaped like a car.

The idea isn't to find everything that is math and hope it's all math. The idea is to find everything that is math and check if anything is left. Find the absence of math.....if there is an absence of math, this isn't math.

I know what to do. I've parsed a whole fucking lot of things. I never thought not-math would be one of them. You know you've been doing this shit too long when you start parsing things that aren't... lol.

[MadGypsy]

I had an inquisitive thought earlier that I wanted to share. Anybody can feel free to chime in.

In the 6 years I've been here I have shared a ridiculous amount of code. My worldspawn threads alone contain a mini-tome of code and some kind of explanation. I was wondering earlier if I have reached a point yet where I have shared so many snippets and explanations that (at least) some of my followers actually understand this stuff without a bunch of explanation. In other words, is anyone so familiar with my syntax that they pretty much just read the code and understand it?

If so, that's kind of funny cause, it's sort of like I taught you to program whether you like it or not.

[MadGypsy]

I did a whole bunch of stuff related to the things in my last few posts. Everything was going smooth and I decided to do a compile to all of the platforms I care about. That's when I realized that my expression parser is compatible with everything except Javascript. I thought I had already tested and found it to be compatible. That is impossible though because, I rely on regular expression look-behinds In 3 spots and apparently Javascripts version of RegEx does not support look-behinds. It was real disheartening when I got an error like "Unsupported RegEx group" (or something really similar to that). The error might as well have said "Good fucking luck, something is wrong." I turned every friggin line into a breakpoint and was able to at least find the line in my original code (ie not the JS transpile) that was breaking javascript. The line had a look-behind in it so, I googled "JS regex look behind"... and immediately learned there is no such thing in JS. Basically, all I'm trying to say is it wasn't a nightmare to track down the problem. I thought it would be though.

As far as look behinds not being supported goes, my work-around is actually going to be the new all-platforms way of doing it. My work-around is actually better than the original code. Initially I was going to use a compiler condition and tell it to only use the workaround if porting to JS but, I realized that it will work on every platform. This means I can ditch look-behinds altogether and that's good. Look-behinds are not very good. If you are slick you can get regex to find some super specific shit with them but, it's slow. It has to find something that matches what you ARE looking for and then start completely over and check if the look-behind is or isn't there (depending on whether you use a positive or negative look-behind). The more look-behinds you do, the more times it has to start over, checking data it has already checked. The "beauty" of look behinds is they don't become part of the matched results. It's like you are saying "I want to find this specific thing but it (must/can't) be preceded with this other thing"

Anyway...ExpressionParser 2.0 is almost done. It will work on all major platforms without compiler conditions.

[MadGypsy]

Well, I think I finally ran the full course of this Key abstract.

It does all of the below things
1) allows string key access on structs/objects
2) auto-converts string values to math expression results where the string qualifies as math.
3) allows a secondary struct/object to be used as default values for any field that does not exist in the target object.
4) toString() method prints struct/object data in it's totality. toString(true) will add the proper tabs and new lines.
5) all procedures are done with a deep scan. It is impossible to hide any data. For instance #2 will find strings no matter how deep they are buried or what they are buried in.

Here is an image that gives a gist of a number of the above listed things. defaultEnt represents values that are hard-coded in the application. inputEnt represents external values that could have come from anywhere except inside the application. receiverEnt takes the external and default object/struct and either performs or allows the things in my above list.

some things to note:

armor does not exist in the inputEnt but, it does exist in defaultEnt. It then exists in the console for the toString(true) call of the receiverEnt. This is an example of the use of having a default ent. Using defaults allows me to make sure that no field is null, empty or undefined.

health in the inputEnt is a math expression but, in the toString(true) call of the receiverEnt it is properly expressed as the expression results. Also, that expression is buried in an object within an object but it was not skipped.

The print of the object in the console has all of the proper tabbing, new lines and commas. With very little work I could write a similar function to toString which properly spits out JSON.



------

Regarding my expression parser:

Below is an image that shows how I eliminated look-behinds. In a sense I am faking a look-behind but, it is more versatile because my fake look-behind can harness specific conditions whereas a regex look-behind either is or isn't.

The code on the left completely replaces the highlighted code on the right. My new code is exceptionally longer but, it handles things with more "confidence" than the original method and it is compatible with all major platforms.



Well, now that all this is done I need to make sure that all the rest of my code is using it and start moving forward again with things that are actually engine related.

[MadGypsy]

one more example:

Earlier in this thread I posted this image which illustrates the entire reason I did all of this. The left was what I was originally stuck doing, and the right is the results of using my new abstract.


Due to some new features I added, the right from above has become even slimmer. This is mainly due to my defaulting feature. Notice there is an entire loop block missing from above right.


In the most basic sense, I am making it much easier to program and work with object/structs in HaXe. It was a lot of fun to work on this. I can't say I have ever spent much time working on how I get to program. All of my programming primarily focuses on what I am trying to build, not how I get to build it. I intend to address more things like this as I encounter them.

[MadGypsy]

@me:Well, I think I finally ran the full course of this Key abstract.

I actually hadn't but, now I think I have. My final features list is this (copy/paste from my comments)

* -Features:
* -add string key access to this
* -default empty/null/undefined fields on this to another object
* -automatically parse stringified math expressions to results
* -accurately print this object
* -internal forEach loop
* -Reflect methods copy(), fields() and delete() moved to public methods of this
* -int, uint, float, bool, dynamic, array, stringmap & intmap are supported
* -stringmap printed as object
* -intmap printed as array

The new stuff is allowing IntMap and StringMap values and properly printing them. There's one more thing I want to do but, I think it's either impossible or waaaay too much of a pain in the ass to worry about it.

I was trying to make this possible

myKey["innerObject"]["someField"]

It was a disaster. Trying to chain key access is something I would really like to have but, I'm thinking I will have to accept that it is not gonna happen BUT, this is possible

myKey["innerObject"].someField

there are problems with this though. What if someField does not exist? Using key syntax a missing field is handled gracefully. Using field syntax it is either going to exist and work or not exist and break. Also field access is too specific. With key syntax I can dump any string in the key and therefore loop through keys. Field access is hard-typed so you would have to access each field one line at a time.

There of course is a work around but it's not very friendly. Basically, just dump the innerObject into a new Key

var myOtherKey:Key<Dynamic> = new Key<Dynamic>(myKey["innerObject"]);
myOtherKey["someField"];

it would be so much nicer to simply do this
myKey["innerObject"]["someField"]

Luckily, there is maybe no spot in my code, so far, where I am dealing with accessing the fields of objects inside of objects.

I actually do know how to solve this situation but, it will require making every value, even ones without fields into a key. That's not such a big deal. What is a big deal is handling the fact that 99% of key methods do not apply to int, uint, float, bool or string. So, I would have to write all this extra code that basically says

if( this is uint,int,float,bool or string )
ignore fields and return this.

there's a problem with that though. The programmer would have to be specifically aware of this awkward condition and treat things that don't have fields like they have a null one. That's ugly and shitty. Bottom line: Keys are only going to work one level deep. Trying it any other way will just turn a bunch of things into nonsense.

-----

look at how tight my final findAndParseExpressions method is. This function keeps calling itself, finding strings and parsing them into expressions, until it bubbles back up to it's original call and terminates. Doing it this way it is literally impossible to hide or bury a string expression. My toString() method is essentially identical to this system. It just performs different procedures within each condition.

[MadGypsy]

Hah! This is crazy. I found a work-around. I remembered seeing code, when I was programming in whatever language PSVita uses, where you can write the same function multiple times and by changing the interface the proper one would fire depending on whichever one had the proper interface signature for how you called it. That probably sounds confusing so here is an example

function doSomething(val:Bool){...do stuff...}
function doSomething(val:String){...do stuff...}

doSomething(true);

in this case the first "doSomething" would be used because it expects a boolean value.

Anyway, I had remembered seeing that and I wondered if haxe could do it. I knew it couldn't use functions with identical names but maybe the @:arrayAccess part could be reused with a different interface value and the proper one would fire depending on the type of value you use. I opined this because @:arrayAccess doesn't care about the name. It only cares about two things. If the interface accepts one arg it's a getter. If it accepts 2 args it's a setter. From there, all setting and getting is handled with keys and the function name is not used. I assumed that meant I could write numerous @:arrayAccess getters/setters with different arg types and it would simply use the right one based on the value type currently being used in the key.

I was right, it totally works that way.



The syntax is a little strange cause it is an array inside of a key but it will work. The hilighted line on the left is utilizing the hilighted line on the right. It's returning true right now just so I know it actually got hit. I will have to expand the function to find the desired value based on the array of field names provided (no sweat at all).

Alternately if someone hated this syntax
myObject[ ["field1", "field2"] ];

they could use desc directly (desc = descendant). It is an @:arrayAccess function but it is still an inline one. Using @:arrayAccess is more of a can do than is now type of thing.
myObject.desc(["field1", "field2"]);

It's not perfect but, it is probably as close as I will get to chaining keys, and it is still better than using fields directly, dumping an inner object into a new key or falling back on Reflect. With this method I retain key access on all descendants.

[enderandrew]

I think C# does something similar with overloading. You can call a function with a different number of passed values, and a different version of the function will run accordingly.

C# in Depth: Overloading

I'm a novice coder, but from this thread it seems like you're still tackling really low level problems when it seemed like way back when you had a fairly playable build of your engine.

What would you say is your overall progress on the engine?

[MadGypsy]

I finished it up. There are a few ways for it to work;

As we can see form line 46, array guts are put directly in the key but, all keys are treated like strings. This is unavoidable. It is mandatory to specifically declare Arrays with mixed types. The only way to use array guts is to use all strings (or ints) BUT my Key abstract knows if something should be a number and uses a number instead.

On line 48 I go a different way and specifically declare an array of mixed values. I then put that array in the key on line 49. According to the inputEnt, props.weapons[1] is a math problem (I only did that to test finding buried expressions). On line 46 it was changed to "rifle". On line 49 we trace that value to the output panel. The console in the flash player illustrates the change was made as well. This means the various ways of making keys are all working.



---

This is currently every possible way to get or set fields within a Key. In the case of get, if a value does not exist it will return null. This allows for stuff like

var val;
if( (val = myObject["field1"]) != null )
{...do stuff with val