Sitephyre & Virtuoso WIP

[MadGypsy]

@MOM - I have no clue what that has to do with this thread, but thank you for sharing

___________________________

SECURITY:

After nailing down the last of my database node system, instead of moving on to stored procedures and triggers (which honestly, I might not even need) I decided to follow the theme that I know nothing and do some in-depth research on security. I read some really long articles, 2 entire PDF's and trolled stack-overflow. This all brought me to the conclusion that nobody is posting anything "real" about site (primarily database, for now) security. I have seen some of the dumbest suggestions I have ever encountered on how to make a site secure.

A lot of people seem to think it's "secure" to store your keys in the same table that you store your users. I drew that out on paper and thought about it a lot. There is nothing secure about that. The whole point of a key is to have a "multiplier" (so to speak) that no one can find or knows, in order to decrypt your hashes. If someone is banging the fuck out of your user table, you would be essentially giving them the key. So, that way is stupid.

I have seen suggestions to do hash inside of hash. This is also retarded on so many levels. For one, hashing inside of a hash the way people were suggesting is doing nothing but creating a potential collision nightmare. For two, it's just ghetto. It's the made up junky way to misuse 2 or more systems in order to convince yourself your hash diarrhea is secure, and it is not.

I also saw suggestions to make up your own encryption and never tell anyone how it works. This is the most laughable suggestion to date. For one, my software is intended to be open source, so there is no way to "not tell anyone". For two, creating your own encryption methods is just a huge waste of time. There are industry standards. There are encryption methods built to follow those standards. Chances are super duper high that "your" encryption method is gonna be really sad and pathetic in comparison.

After reading all this garbage I changed my search criteria. I already knew that I wanted to salt a variable, hash it and then make thousands of passes. Reading all of the above was a waste of time. What I really needed to know is where is a safe place to store your keys.

This led me to a whole freakin lot of either stupid or not possible (for me) type of answers.Everything from storing them in the user table, to having a dedicated server which is completely separate from your domain... and it's only purpose is to validate stuff. I really like that idea, but It's just not realistic for me.

I did however dig up one post that had a realistic idea that I also liked - storing keys as environmental variables in either httpd-conf or .htaccess. This way made sense to me. In the case of httpd-conf the file is completely outside of htdocs and therefore probably un-retrievable. In the case of .htaccess, apache will not serve up .htaccess files, so chances are much slimmer that someone can get to the keys. In both cases it is not absolutely fool-proof but it is definitely way better than sticking keys in the user table (or any other table for that matter).

So that's where I am now. I have my entire database thought out and installed and I have the framework for the root of my security model hashed out. My goal is to have the entire sign - in/out/up process completed tonight. This is both easy and hard. 2 simple forms and a "log-out" button wrapped in my security model will get the job done but, I think I want to also play with some ajax and add a "premonition" layer to the forms. I also want to work with some jquery and get some attractive error messages displaying.

So, just getting some sign-action forms to plug into the database is cake. Making the page know who you are before you even finish typing and displaying errors right after you make them is a little more work. Fortunately, I have already done all of these ideas at one time or another and I am positive there is some starter/example code hanging out on one of my drives. I may even be able to copy/paste much of it as is. Even if I have to write it all from the ground up, I'm not discouraged. I know jquery very well and actually love coding in it so, regardless, I'm just going to keep trucking and moving my project forward.

[Guest]

I thought this thread was still about...

[OMacKnight]

browsing a forum with a controller

why does that seem like a pretty interesting idea to me?

I /think/ I could get around the input issues if designed right. But you'd need dual analogue thumb sticks or something similar

If I finish all my college work by the end of today I'll do a doodle of how I'd do this

________

If I'm making no sense I do apologise, I'm incredibly tired but this idea just popped in my head reading Mark's response.

[MadGypsy]

I thought this thread was still about...

lmao! Now, I know for a fact that you haven't read any of this... It's OK, I know my typing is long as fuck, all the time. However, if you would like to talk about controllers, you could use the thread you thought this thread was

I know another reason you got confused and it's my fault. See, I come up with names for things and I use them for other things... considering the audiences probably wont overlap. So my controller idea was named Radiant Virtuoso Pro but my flash object notation display engine is also named Virtuoso. I may come up with another idea one day that has nothing to do with Quake or web programming and name it Virtuoso as well...lol. I suck at "rules". It makes me wonder if I would have this mentality with say, naming my kids (if I had kids).

"You're big Pete, you're little Pete and you're Girl Pete"

The one thing I don't understand is, didn't you have to go to the proper thread to get the virtuoso controller pic that you posted here in the wrong thread? lol

---

I'm getting better though, when it comes to naming things. When I was in clan 777 I made all kinds of web stuff for our forum and all of it ended in "nator". I had ideas for more stuff and all of it ended in "nator" too. Then I decided that "raider" rhymed enough with "nator" and sounded better so I changed it all to that (ex) Rosternator became Roster Raider

Oh right, and even before that i had my "o-matic" phase...lol. I suck at making names so I don't even bother to give it effort. The possibility that Sitephyre could have ended up Site-o-matic or Sitenator is really super high. However, if I would have went that route, my url would have been something like

http://site_where_I_can_do_stuff_and_meet_people.com

or even worse
http://site%20where%20I%20can%20do%2...t%20people.com

Then I would build the domain-o-nator so you wouldn't have to type all of that.

[Guest]

You've caught me fok

[MadGypsy]

I think I have written and tested every possible way that you can create a database/tables/user(host level) script. ...Maybe I haven't made/tested ev~ery way, but it sure feels like it.

Throughout all of this I encountered tons of problems. At one point I wanted to just blow up my computers and find something else to do with my life. Really, it was that bad.

From SQL obviously (results-wise) not completing while also not throwing any errors, to getting completely different results on every test even though I didn't change any code.

I finally figured out what the most severe problem was. It stumped me forever. I was stuck on there being errors in my SQL and there weren't any, but I refused to accept that there weren't any (because it wasn't working). When I finished reading every character of my SQL for the billionth time it finally hit me.

My SQL is multi queries. My php sends the multi query off and waits for a true/false return before doing (or not doing) the next set of queries. The problem is, SQL returns true after the first query. So my next chunk of sql was running before the first chunk completed. Considering some permissions are table specific, permissions were crashing because the tables didn't exist yet. Other places in my chunks had the same problem.

In short, I was creating an asynchronous instance but treating it like a synchronous one. I have fixed this. Now here's the real kicker. I changed so much code to something else, in an attempt to isolate the problem, that now I have to go back and change it all back to the way I wanted it. Along my original line of thinking, my original way was all problems so I never expected to change it back. Turns out, none of my original code was the problem and I want it back.

Luckily, This shouldn't be a huge deal. I wrote a simple tool that writes all of my original way for me. I just need to point my code to that. So, really, this is not a "rewrite" but a "rewire". However the wire goes through a number of places. Actually, maybe that needs to change... so a mixture of my original way but with a shorter wire.

That's going to take some reconsideration. I will add though, my PHP/SQL knowledge has grown considerably throughout all these problems. I learned all kinds of stuff I never knew and in some cases was introduced to new ways to look at some things.

I also decided to take a good luck at how wordpress scripted their installer. My methods are not all that different. In the most basic sense they are identical. If you compare our sources the actual code is totally dis-similar but the concept is the same.

I decided not to read any more wordpress code. Apparently, (assuming wordpress is a good source) I am already in the pocket on the flow of steps. I don't want to be influenced. It was more like a quick affirmation "Your way is not contrived." and that made me happy cause, I have put some serious work into this.

I'm going to try and finally finish this, today.

P.S> it's cold as a {NOUN} {ADJECTIVE}

[MadGypsy]

Done, I rewired it in like 1 minute. I don't know why I thought I had a long "wire", it was only one spot. It also works great. I just kept clicking install and everything successfully completed, every time. I also went to my database and made sure the tables were there and correct as well as checking that (host level) user permissions were correctly installed. I installed the database like 50 times so I didn't check that every install was correct, but I know it is. I checked like 5 of them and nothing was skipped. The install log has no data in it (it only records errors) it is set to only append, so this means not one recorded error for all 50 installs.

Imma say that's satisfactory. I think I have already had every possible error you can have so, it's actually about damn time that I'm passed that. I'm going to go mess with some css and make my very craigslist looking form into something a little more decorated

[OMacKnight]

grats

[MadGypsy]

Sigh, the more I learn about SQL, the more I realize how much I suck at it. My work is going fine and everything, I'm finding answers to things very quickly. I don't even have an error to worry about atm.

This post is not about problems. This post is about "the rabbit hole". In this sense the rabbit hole is a deep and twistingly burrowed tunnel with never ending SQL information scribbled along the walls. I make things that work well, get all excited, think I'm moving forward with great velocity and then I have to research something...just to find out that I am only like one foot down the rabbit hole.

The thing is, I'm teaching myself all of this stuff (and always have). I have no peers (in this sense). So EVERYTHING is implementing ideas and then figuring out how to fix them. That's the learning process. Go, go, go,.. what's this error,.. learn something.

I try to just flat out read the docs, like study them without any purpose other than to learn. This doesn't work for me. I either get an idea from something I read and shut down studying to "try something" or my brain just goes bzzzzzzzzzzz and nothing I read means anything to me. Especially in the SQL docs. I mean, this is a "description" in the SQL docs

table_references:
    escaped_table_reference [, escaped_table_reference] ...

escaped_table_reference:
    table_reference
  | { OJ table_reference }

table_reference:
    table_factor
  | join_table

table_factor:
    tbl_name [[AS] alias] [index_hint]
  | table_subquery [AS] alias
  | ( table_references )

join_table:
    table_reference [INNER | CROSS] JOIN table_factor [join_condition]
  | table_reference STRAIGHT_JOIN table_factor
  | table_reference STRAIGHT_JOIN table_factor ON conditional_expr
  | table_reference {LEFT|RIGHT} [OUTER] JOIN table_reference join_condition
  | table_reference NATURAL [{LEFT|RIGHT} [OUTER]] JOIN table_factor

join_condition:
    ON conditional_expr
  | USING (column_list)

index_hint:
    USE {INDEX|KEY} [FOR JOIN] (index_list)
  | IGNORE {INDEX|KEY} [FOR JOIN] (index_list)
  | FORCE {INDEX|KEY} [FOR JOIN] (index_list)

index_list:
    index_name [, index_name] ...

Do you understand all that? Well, I do (mostly) but my brain gets sick of parsing all that crap for meaning.

PHP however is the opposite. I could read PHP docs all night like it's a very long magazine or something. That's another rabbit hole but at least it's a rabbit hole that I feel very comfortable in. PHP seems to have an absolutely never-ending slew of built in functions. I come across new ones all the time. Actually, I came across one the other day that annoyed me. I just finished writing a nice little script that read/writes/appends files just to learn about file_get_contents() and file_put_contents(), which is basically a one liner that does exactly what my longer-than-one-line script is doing.

I actually refuse to use those - for being there all along without me knowing about it (lol) take that, file_get/put_contents()! Anyway, I guess it's time to get back to some CROSS JOIN

apparently it's useless but Imma learn about it anyway.


why useless?

...INNER JOIN (table1, table2, table3) //without cross join
...INNER JOIN (table1 CROSS JOIN table2 CROSS JOIN table3) //with cross join

why would I want to substitute typing a comma with typing CROSS JOIN? The above statements are different in no other way. So, CROSS JOIN is useless... or so it currently seems.


**regardless if I script in 100 languages, I am an AS3 programmer. So, I just want to throw it out there that (ex) file_get/put_contents is not something I "should know". I know AS3... I just program in everything else. Actually, I went from AS3 to PHP without a hitch. They are so similar syntactically. It's taken me a couple years though to really dig through the php docs and look beyond the general stuff.

AS3 is based on ECMA standards, PHP has to be as well, I believe javascript also uses ECMA Script standards or some sub standard. So, based on that being the 3 languages that I have the most success in. I think it is safe to say I am an ECMA languages programmer.

I just threw in that last paragraph to maybe spur people to realize/think of something. Maybe you never thought about languages belonging to a family or the fact that learning one language can lead to you automatically "getting" any other language in that family. That being said, I know languages that i don't even know yet.

[MadGypsy]

Hmmm, I was confused to what ECMA standards meant. I thought the standards were based on syntax and hierarchy - which is why the syntax and hierarchy is so similar for certain languages. This is not the case.

The standards have everything to do with function. I can still say I'm an ECMA programmer though cause my most learned languages are based on the standard, however PHP is not an ECMA language. This is probably due to ECMA languages primarily being used for manipulating object on a screen, and of course PHP can't do that in the way I mean. I'm sure there are ways to fake it and those don't count.

Anyway, live and learn...

[Spike]

ecmascript is the language.
javascript (mozilla) is *an* implementation of ecmascript, as is jscript (microsoft) or actionscript (adobe).
ECMA is a standards organisation and does not relate specifically to the web or any standard in particular.

PHP is a different language completely.

and I hate both languages because neither is statically typed.
which is a significant issue with php because it means that you can end up setting php variables via an http request that you should never have had access to... including overriding configuration paths and other stuff...

[MadGypsy]

@Spike - yeah I did research between my last 2 posts and cleared up my misunderstanding. PHP might have some loose wheels but, I still like driving it. It may be my favorite language. I'm actually starting to really like SQL though.

I'm seeing how SQL is not primarily a "GET/SET something BECAUSE OF something" language. There's actually a lot of other stuff you can do and I'm finally at a point where I can start exploring them. I'm still only a couple/few feet in the rabbit hole but, I'm actively following the tunnel

oh ps> @"standard" - That actually wasn't my fault for misunderstanding. I have seen "standard" used to describe the relation of js/as to ecma in many tutorials. Which for some reason reminds me that I know E4x (ecma for xml) and I never knew what that meant til yesterday.

E4x is the preferred method for referring to xml in as3, that's why I know it. There's actually nothing to really know dot syntax parent:child relationships, use @ to get node attributes and treat nodes of the same name like an array. There, now you know e4x too.

<node>
<other_node name="myName" />
<other_node name="myOtherName" />
</node>

return node.other_node[1][email protected] //myOtherName

[MadGypsy]

Tricksy Script (episode 1)


UNIQUE - the unique key allows you to define designated columns as mandatory to contain a unique value. For instance, this is a constraint you would use on something like user names. If a user tries to sign up with a name that already exists it gets rejected on the database level if there is just one other row that has the same value for the designated column. What they don't tell you is:

UNIQUE KEY uk_userKey (key1, key2, key3)

is NOT shorthand for

UNIQUE KEY (key1)
UNIQUE KEY (key2)
UNIQUE KEY (key3)

The former expects the "sum" of all keys to be unique and the latter is specifying that each key has to be unique individually.

That shit just drove me nuts for like 2 hours. It's not in the docs. Actually the docs for UNIQUE tell you nothing at all, no examples. Just 3 paragraphs of info that is useless if you don't already understand how the constraint works.

I actually found the answer in the comments and not even for the UNIQUE key. I applied what someone said about PRIMARY keys to the unique key and coupled it with one little sentence I remembered reading from another thread. Voila' my unique keys are working as expected now.

I noticed this problem when I faked 10 sign-ups simultaneously. There was an error in my script, I fixed it and ran the code again. When I went to my database to see the results. I had 20 users, which would be impossible if my UNIQUE constraints were functioning properly. I automatically knew it was UNIQUE that was messed up but figuring out why was a pain in the neck.

random advice: NEVER use W3CSchools to learn MySQL. The reason is very simple. They try to teach you all the different flavors of SQL in one shot and the results are dumping a bunch of useless script in your queries. They also don't seem to update the information as the flavors of SQL are updated. Their SQL is ancient. If you want to learn MySQL go to dev.mysql.com/doc and make sure you read the comments/replies.

[OMacKnight]

I learned another thing today

(I also learned at college that if my college work is this simple I'll have the active time to be programming (at the expense of me working on my own games, though that's sort of me developing anyway))

[MadGypsy]

Tricksy Script (episode 2)

This is an opposite "tricksy". I was reading about ROLLBACK. You would generally use this by "monitoring" a chain of queries and if one fails you start "rolling back" to where you were before you began the query (recoup).

However, InnoDB engine (which is what I am using) does this automatically. You don't have to write any ROLLBACK scripts with the InnoDB engine.

Since this is a "tricksy" that deals specifically with InnoDB, I will include another "tricksy" related to the engine.

FOREIGN KEYS will ONLY work on InnoDB. A few months ago I couldn't get any of my FOREIGN KEYS to work but the script that included them wasn't throwing any errors. I couldn't figure it out and wound up writing a bunch of manual FOREIGN KEY functionality (php side). I eventually (and at complete random) was enlightened to the problem and easily fixed it. (I was using MyISAM engine, I just "find/replace all" to InnoDB)

FOREIGN KEYS allow you to link the columns of one table to the columns of another, so that if the referencing table has those keys updated or deleted, the table that holds the constraint will either update the appropriate column (automatically) or delete itself.

example

user
{
id:some_value
}

user_mailbox
{
owner_id:some_value
FOREIGN KEY (owner_id) REFERENCES user (id) ON UPDATE CASCADE ON DELETE CASCADE
}

if you changed the value of id on a user table row it gets automatically updated on all corresponding user_mailbox rows with an owner_id of the same value. If you delete that user row, the user_mailbox row(s) that correspond(s) will also be deleted.

InnoDB is bad-ass. There are all kinds of things that make the database do a bunch of work for you. It's a trade-off though cause InnoDB databases are generally more bloated than say MyISAM. However you could mix and match to some degree. You could just make your tables that contain foreign keys InnoDb and make all the rest something else, but you would have to juggle what you are dealing with at any given time and act appropriately. For instance all your non-InnoDB tables would have to be ready to be manually rolled back upon an error.

I'm not sure what would happen if you had to JOIN tables which use different engines. Maybe nothing, maybe it would be a disaster. Personally, I'm fine with just using InnoDB across the board.