Quakespasm on Steriods

[Baker]

sv.datagram is the unreliable broadcast buffer. if its too big for that client (after entities etc), then the entire unreliable buffer is discarded and the game tries to go on without it.

Yeah, like the sounds and cut-off, I've seen that happen in very rowdy games (sometimes a few seconds later the server crashes because of max edicts 640). The network I was on yesterday wouldn't let me post here and it looks like you use per client buffers like I imagine you use in FTE.

[Baker]

The R4 version is even better than I thought. sv_protocol 15 and 666 look like they optionally use extended capability via your extension system but still let a standard 15/666 (like GLQuake or Fitz 0.85) connect but a QSS client would use the NACK system + delta compression.

I also noticed you switched the server from reporting it as ProQuake 3.4 to ProQuake 3.0 to avoid the ProQuake NAT fix via extra port.

/Just random comments.

[Spike]

falling back to standard 15/666 is kinda the point of the extension system. otherwise it would just be another protocol number without fallbacks.
FTE is able to select between 15 or 666 automatically, but I didn't bother with qss at the time. that said, r5 will do makestatic the same way as it does baselines, so other than sounds it should be easy enough to get qss to fall back to 15 too.

proquake 3.4 uses a separate port on the server for gamedata like every other crappy nq engine, but for some bizzare reason it also uses a separate client port for the actual game too. I have no idea why, but I suspect its due to some really crappy NAT/firewall someone had that could only track a single remote port for each local port, and also failed to update that remote port if it connected to a different place after...
in which case, a single server port sidesteps that issue anyway, as well as ensuring the port pair never changes (note that qss accepts the server reporting port 0 as no-change, which the server only doesn't do for compat rather than any other reason).
the only reason qss reports itself as proquake is for the 16bit angles thing with engines that are not using 666 nor protocol extensions, which is not part of vanilla qss.
other than rcon, proquake's tweaks are thus made redundant by qss.

I don't have a list of the features required for each proquake revision. qss doesn't implement proquake's stufftext-hack extensions (so the reported version should probably be lower because of that, but I kinda doubt mods bother to check the version anyway), and its not commonly used for multiplayer so I didn't bother with pings on the scoreboard either. From reading the code, 3.0 seemed to be a reasonable version that worked, but I didn't really test that much, certainly not with public servers.

[Baker]

Possible "dumb" questions:

1) Bots. In the network code, there is a lot of reference to bots. AFAIK, Frikbots and frogbots don't use SV_ConnectClient at all. Is this for some other kind of bot (globot?)

	if (client->netconnection)
		Con_DPrintf ("Client %s connected\n", NET_QSocketGetAddressString(client->netconnection));
	else
		Con_DPrintf ("Bot connected\n");

Update: Looks like this is for DP_SV_BOTCLIENT extension. "like spawn but for client slots (also calls relevant connect/spawn functions), returns world if no clients available"

2) Multicast -- isn't in dpextensions.qc AFAIK. It's all over the place in the FTE code, "#82 void(vector where, float set) multicast (QW_ENGINE) (don't support)". It's a message of some sort like write byte. But beyond that, don't really know what it is useful for. Multicast seems to be for something other than send to everyone and sent to 1.

[Baker]

Wave 2 done. Minimal single port server isolated.

/Suspect its all downhill from here.

However, I anticipate 3 things that will need sorted out.

1. Gamedir. Need to look through R4 and see how game dir is communicated. Game dir needs to be communicated, even if nothing is done with it. Shouldn't be a secret. The server also needs to make that information available externally.

2. What to do about the delta compression and the bobbing. Haven't looked through it hard enough yet.

3. (Can't remember. It'll come to me eventually.)

/Plenty of testing to do. I's to dot, T's to cross, etc.

[Spike]

engine support for bots takes the form of the botclient extension, where bots take up actual client slots, instead of having to use numerous hacks to rewrite the scoreboard and then deal with what happens when a real player joins and consumes a slot that was already in use. kaboom.

multicast is useful in various ways. unlike broadcasts, multicasts can do real filtering like on pvs or client protocols.
it also allows support for unreliable unicasts, which nq never supported, solves an issue with fragmenting reliables if you utilise backbuffers, provides handy end-of-message hints for network translation, and is all-round useful.
if I had my way, MSG_ONE, MSG_BROADCAST, MSG_ALL, would all be removed entirely in favour of multicasts.
dpextensions.qc is significantly lacking. its also annoying to work with. I included a qsextensions.qc in that zip for a reason. go read.

gamedir is communicated as "id1;qw;fte;hipnotic;quoth;warpspasm" for instance. except with the engine-specific gamedirs of id1+qw+fte omitted for brevity. only gamedirs which can be switched on the fly need be listed, which can indeed be awkward.
if you're switching gamedirs on the fly, then you kinda also need to be willing to re-exec configs on the fly (fuck off with those vid_restart commands inside all those config.cfg files, its so fucking annoying and needs to die).
'obviously' this implies you need to be able to properly reset cvars etc to their engine-default so that omissions in one default.cfg file don't screw stuff up.
oh, and poluting a user's gamedir with every single file on the server is rude too (oh look! I just included my own default.cfg file and now the client quits every time it starts up! mwahaha?).
and if a mod adds extra frames to the player.mdl, you'd better hope that mdl is inside a pk3 so it can be downloaded properly. what, you're going to skip the whole downloads part? hurrah! someone who isn't crazy!

bobbing should already be supported...
tbh, I'd leave the deltas till last. or at least till r5, if I ever bother with releasing that.

[MadGypsy]

Off Topic:

@(fuck off with those vid_restart commands inside all those config.cfg files, its so fucking annoying and needs to die).

I don't recall ever seeing you utter a curse word...ever. So you must seriously hate those things.

gamebot, multicast, backbuffer...there, now we're back on topic.

[Baker]

@gypsy

-- FTE/DarkPlaces's engines support some "truly not at all like Quake not even close to reminding you of Quake and maybe not even a first person shooter at all" mods and have to embrace the idea of a full engine restart on gamedir change.

So who knows what kind of nightmares Spike has.

41) Bobbing is ... broke in R4. I imagine I'll discover why on purpose or incidentally.
42) dpextensions, qcextensions ... since my experience with QuakeC is pretty nil. I think once someone told me how to use dpextensions.qc so I could use .alpha and see what it looked like before FitzQuake 0.85 existed. But I definitely believe what you are saying about yours being easier.

43) Mods. If IPv6 actually had a way of determining a local network IP (LAN ip, like 192.x.x.x or 10.x.x.x or 172.x.x.x or whatever the list of those [subnet mask?] ... I'm a bit IPv4 rusty ... haven't though about that in a while) ... I'd be tempted to allow a client to download anything in that specific circumstance. Your local network = why not?

But IPv6 is 9 times out of 10, a royal clusterfarm of murky. Hehe, in 2014 I started adding IPv6 support and then after thinking about the crazy problems (banning, stuff like this, other things) .. I decided to switch to stop and instead something more productive. Sometimes the petty stuff that is impossible in IPv6 just makes me wonder ...

Determining if an IPv6 address is local is nigh impossible, AFAIK. Then again, who is going to have a IPv6 local network, hahahaha.

[Spike]

@mg, I swear quite a lot more than I should. if you save your swears for when people are actually being stupid (or have been stupid), people are more likely to stop and actually fix the shit that they're creating/propagating. swearing is useful, but only if you do it rarely.

@baker
41) yeah, sorry, forgot about that. I fixed it in my local build. I ought to dump out an r5, but meh. any other issues with it yet? :s

42) easier for me, certainly. fteextensions.qc does conflict with some of the builtins in defs.qc, but on the other hand some of those builtins are wrongly defined anyway. qsextensions.qc includes only new stuff which is a bit different from fteextensions.qc, but I included the extra pragmas to avoid all the warnings about unused stuff that is common with dpextensions.qc. I should really put out some allextensions.qc file or something that combines them all without conflicts, but tbh I'm too lazy to fix up dpextensions... as well as figure out which version of it is actually the most recent (afaik its not the one on dp's svn...) either way, for just .alpha its kinda overkill.

43) ipv6 doesn't really do local network IPs, because NATs break end-to-end connectivity, and that's bad. decent routers will still provide firewalls, in which case home lans are likely to just use thier globally routable /64 rather than a specific subnet.
really though, you shouldn't be trusting a local lan any more than a public one. datacenters etc etc.
the only one you can really trust is loopback, your os is meant to block such spoofed connections.
also, ipv6 hosts are meant to always have a link-local address, such packets are not meant to be routable, so there's your local network. just don't trust it.

[Baker]

I've run certain things through a very strong gauntlet.

51) If you get ECONNRESET (connection reset by peer) from a socket, giving them the boot is the right thing to do --- nobody is home. Like I need to tell you that. But I noticed spamming if I clicked "X" to close a connected client, which immediately rudely quits.

52) DropClient (false) is usually the wrong thing to do, I guess --- leaves an immortal solid player ghost sitting around. DropClient (true) gives them the proper dead treatment.

The irony is, I've done testing and fine tuning as I've implemented your code, fixing any issues I comtemplate.

Like I noticed ProQuake clients slightly had trouble interacting with QSS and fixed something small.

So the irony part ... you did a hella lot with a codebase you don't like.

When I look at the implementation, the "really damn close" just causes me to want to "fix it right".

Network code is horrifically difficult to deal with, especially in Quake because its spread all over the damn place and nested behind arrays with members references function pointers.

You must have had quad damage and pent, but they wore off and then you became aware of the suckage.

For me, the reverse is true. Initially, it was tough but each piece of difficult forward progress caused me to need to fix things that were just shy of perfect.

Very incredible code you wrote.

[Spike]

51) ECONNRESET is a microsoftism (and is normally only valid for tcp connections, which are actual connections... hence the name). you can spoof it with stray icmp port-unreachable packets with spoofed source addresses (and even crash(sys_error) vanilla servers with it, or really any program written before win2k started screwing people over with this non-standard error code). if you know the victim's ip+port, you can kick them off the server with a single such packet.
note that quakespasm's net_wins.c avoids spamming in this case.
you can check for that erroneous error result on other systems too, its safe because it'll never happen.

52) huh? SV_DropClient(false) is required if you want to invoke qc (which can also transmit packets, so be aware that this can break the hidden net_landriverlevel global (as can prints, so be VERY careful - globals are evil)). that argument is dubbed 'crash', and if true then it avoids potentially unsafe actions that would just cause it to crash again, which skips networking and skips notifying the qc.
quakeworld has a drop flag on clients instead, which helps solve some recursion issues if nothing else.
I swear those places where its called with true are not my code/fault!
the fact that they were still in there shows how little I bothered to actually test it. hurrah.
/me wonders if fixing it will break any mods due to the 2-second rule not really existing for players. did nq do the whole zombie thing?

I don't personally think my code is as incredible as you claim. certainly the finished result has just as many globals as the original, so its still crap from that perspective.
its not that hard to figure out the main->dgram->syssockets heirachy. The rest is just fixing it to not do stupid stuff. yes the original version was a little overwhelming with separate files for bsd+dos+win+lin+modems+udp+ipx etc, but once you get what's going on, it shouldn't be that hard to figure out where the code responsible for the various issues is.

small note, but I assume that you're aware that changing client_t, qsocket_s, or the qsockaddr struct will cause certain qccx-hack-using mods to break, yeah? and that not changing them will mean you can't use ipv6 and stuff? just checking... seriously, good luck with that...

[MadGypsy]

small note, but I assume that you're aware that changing client_t, qsocket_s, or the qsockaddr struct will cause certain qccx-hack-using mods to break, yeah? and that not changing them will mean you can't use ipv6 and stuff? just checking... seriously, good luck with that...

You said similar things to me when I first started with my project It's terribly mean...lol. I remember one time you said something like

"And of course you've been saving blah blah blah so, doing x, y, z times a, b, c plus 1, 2, 3 is no sweat."

where abcxyz123 all represent shit that I had no clue what you are talking about. and no I didn't save the blah blah blah either...LOL! Oh man, fun times.

I will say that this thread is so far over my head that it makes me want to cry (figuratively). Good thing IDC about putting MP in my engine. It would take me somewhere near the rest of my life to ever get this stuff working.

@Globals - there is not a single one in my entire engine except for the flash ones that I can't do shit about. I've also never really bothered with final classes but I recently read that they can improve performance so I went and finaled everything that could be final. A huge chunk of my engine is completely static as well. I know this doesn't really have anything to do with this thread but it is relevant to some of the things that were said here.

[Baker]

Finding new ways the cause crashes:

1) Quake in general. Type "reconnect" in the console when connected to a server. Crash.

/There was a 2nd one that was QSS specific related to networking, but now I can't reproduce.

[manwithnoname22]

Determining if an IPv6 address is local is nigh impossible, AFAIK. Then again, who is going to have a IPv6 local network, hahahaha.

Last I checked, ipv6 has link-local addresses for that purpose. Those are the addresses with fe80:/10 prefixes. I may be wrong on that, though.

Doesn't ioquake3 (for Quake 3 Arena) do that for its implementation of ipv6?

[Baker]

Last I checked, ipv6 has link-local addresses for that purpose. Those are the addresses with fe80:/10 prefixes. I may be wrong on that, though.

Doesn't ioquake3 (for Quake 3 Arena) do that for its implementation of ipv6?

Sounds like what I always thought. But I'm not Mr. Networking.