Legacy update v2.76-1075-g765ea79

[ryven]

Anyone got the missing simple items demo?

[kujaw]

I actually like it. For the first moment it was a little shock, but then I got used to it. However I agree that maybe the graphics of explosives could be adjusted, cause airstrike is really, really huge, and grenade is ok.

I think it also depends on the goal of etlegacy (or particular game, that we are playing at the moment) - if we would like to use etlegacy for competition playing, then I think user should be able to turn on/off wolfparticles.

However, for public servers and casual players it would be better to have consistency among less and more experienced players, so switching this setting should be disabled. Maybe it should be enabled per server, so only the admins on a server could switch particles on (and no player could switch it off in their config) or the opposite.

[eMMiel]

1 hour ago, ryven said:

Anyone got the missing simple items demo?

Normally i could find you a demo for that. Can i send it to you on discord? 

[eMMiel]

5 minutes ago, kujaw said:

I actually like it. For the first moment it was a little shock, but then I got used to it. However I agree that maybe the graphics of explosives could be adjusted, cause airstrike is really, really huge, and grenade is ok.

I think it also depends on the goal of etlegacy (or particular game, that we are playing at the moment) - if we would like to use etlegacy for competition playing, then I think user should be able to turn on/off wolfparticles.

However, for public servers and casual players it would be better to have consistency among less and more experienced players, so switching this setting should be disabled. Maybe it should be enabled per server, so only the admins on a server could switch particles on (and no player could switch it off in their config) or the opposite.

I totally agree with you. Especialy the point you made about where we see ETL going. Competitively or not. 

The solution you're giving to keep consistency and letting the server admins decide is actually very good!

[MrMuppet]

Rework explosion and particles effects #1450

Particles effect has been turned on by default and the cvar removed (see #1173), in order to get rid of the unfair advantage provided.

The default explosions effect is however too strong, so take measure to diminish its intensity. Something like a semi-transparency would be great.

In addition, the cg_wolfparticles cvar hid many other smaller effects (smoke effects, debris, ..) at the same time. Since these are cosmetic only:
:handled these by the cg_visualEffects cvar too, so only the tamed explosion effects would be mandatory.
:consider allowing shoutcaster to turn the main explosions effects off too

[ryven]

4 hours ago, eMMiel said:

Normally i could find you a demo for that. Can i send it to you on discord? 

ofc my et friend

[Spyhawk]

You can now test the reworked explosions (wip) on the latest build on `/connect etlegacy.com`.

[ryzyk]

I see its smaller and a bit transparent so for me it feels already much better as it doesn't look so out of place and doesn't obscure vision too much anymore.

[MrMuppet]

On 5/23/2020 at 1:48 PM, Spyhawk said:

You can now test the reworked explosions (wip) on the latest build on `/connect etlegacy.com`.

@Spyhawk @ryven @Timothy @kemon @RaFaL @Aranud Is there any other improvements on this build? If so, do you wish us to stress test it?

[Spyhawk]

There are, with other fixes. You can see what's changed in the public changelog.

However, there are also many smaller changes done, and we can't guarantee stability. At this point, I am (personally) not willing to recommend any further build upgrade, because I feel TM (as a whole) doesn't understand the drawback and many here rather have their cake and eat it too. The too high expectation of players generates a lot of stress and frustration that have lately reached an unhealthy level for developers (at least for me, personally).

I'll try to write more about it later tonight or tomorrow, so everyone is eventually on the same page and get a clearer picture of the situation: how it started, what went good, what went much less good, where we're heading right now and what that means for TM, etc.

[MrMuppet]

ok, cheers.

[Spyhawk]

So here we are. The following will most likely be quite long, because it's not easy to give a full picture of where we are now without looking at how we arrived in the current situation.

Disclaimer: this is an attempt to describe the past year more or less accurately, some part of it only involves my own point of view, I am not speaking on behalf of the whole ETL dev team. In any case, don't take it personally, because infuriating people is not the objective here. Giving an explanation on the current situation in a more factual manner is, so TM knows what's the way ahead. Just drop me a PM if you feel to insult someone, I promise I won't even read it.

Let's quickly recap from the beginning.

 

// How ET:Legacy started 

ET was released sometimes in 2003, and various mods flourished thanks to the available SDK. The entire source code was released in 2010, and quite a few projects started to improve the engine. Of all these efforts, only ET:L remains. In these early days, the work was primarily focusing on the engine side, but work started to improve the mod side too.

Because of license restriction and unlike most existing mods, the ET:L mod is built from etmain (silent, nq, nitmod and many others were based of the open etpub code). In other words, the mod was several years behind other popular mods, barebone and buggy and only slowly integrated features over time. It was also not clear what the objective should become in these early days, but it became gradually clear it should stay close to the original game design: no fun feature, nor any special weapons, just plain ET experience - but still aiming at fixing many of its shortcoming.

Development was done by a small number of people. A few others joined the project over time (including myself). Some aren't active anymore. It took quite a few releases over the year to reach a point were the mod was actually enjoyable. Since the mod wasn't popular, we couldn't get lot of feedback. Testing before a new release consisted of 2 or 3 night sessions of testing with a couple of guys we could find to help (7 or 8 people, over a few games). With these limited testing sessions, it's no surprise that the quality of each and every release was subpart. Every release was better than the last though, but still crucially lacking in stability and had very major issues we couldn't detect in time.

Also, the project had long delay between release. There is so many things to fix, so many lacking features, that getting a release out has never been easy: you need to reduce the planned features, cut what's not absolutely needed, and try to fix important bugs or at least workaround them as best as possible. The whole process requires tons of work, time and energy, that involved people feels usually exhausted - at least it always felt like it for me.

And such was the situation for 2.76: about 2 years in development, not seemingly near a state of being "good enough" for release. And basically not any feedback for 2 whole years. We knew the code was bad, we we had no choice other than to release it anyway to get the much needed feedback. We planned it for being a "just acceptable" release and planned a second release 2 or 3 months later to fix the important shortcoming the community of players would give us. It took about 3 months of crazy work to stabilize the code as much as we could, but by January of last year the release was out.

And indeed, 2.76 release went badly. But not for any of the reasons we could have predicted: a small, non optimal fix trying to limit the way admins can pollute players UI indiscriminately was badly received. I would lie if I described the way it went other than "being shit upon after giving some stuff for free" by some of the ET community. I personally managed it okayish, in part because I wasn't actively exposed in community forums, and because I expected 2.76 to be far from perfect. I also had similar experience in the past (in a complete other context), so this wasn't entirely new to me. I was however surprised by the intensity of the backlash. I know some members of the ETL team had understandably a hard time to deal with it.

 

// Continuous feedback

The planned short term release never happened. After some discussion with some input of the community, the planned scope changed, more bigger fixes were included, the protection feature required a complete redesign, etc. In a nutshell, we were back to the usual business of delaying any new release "until it's good enough".

Until then, there were a few servers running Legacy mod, but one in particular started to become regularly populated. We noticed these guys had some issues with a few 2.76 bugs that we had already fixed, so we arranged to provide a fixed version by compiling the latest code we were working on. It kinda happened organically: soon after we found out we had feedback on what worked and what other bugs were found, which helped to provide a new version with fixes and some new features we planned for 2.77. Collaboration with TM started here.

Of course, we followed some internal process to ensure not to send broken builds (fixing a bug sometimes can create 3 new bugs): each dev test the feature or bug he's working on locally, then we upload a build on our own test server. We check everything is seemingly working smoothly before sending the build to TM a few days later, or we work on a new build if it doesn't.

 

// What worked.

(and quite well)

• Continuous feedback. It really helps. We catch bugs as soon as possible, we have early feedback, we can play test our own code live with many humans players. I have no doubt the overall quality of the code is much better than anything we released previously.
• Log access. We had log access simply by asking to get them, and more recently, direct access to logs, allowing us to check them in case we need more info to solve an issue (I'm still tracking an issue in TM logs, but it's happening so rarely that I'll need more time. The good news is that we now know its occurrence is much rarer than initially thought).
• Security. Remember Antum and how TM server suddenly crashed? A close collaboration helped to pinpoint the exact issue so we could fix it once and for all. That was a quite thrilling episode.

 

// What didn't work.

(this is longer than the above, but don't be fooled as these are much more specific than what actually worked)

• No real testing done on our own test server. While devs check changes before approving new builds for TM, nobody else does. Our local independent testing by devs on their machine and testing on our test server isn't much different. Testing is done with 1 or 2 players on our test server, and 40 players on a full TM servers, but nothing is done in-between. The process doesn't scale up, it's all or nothing.
• Sometimes shit happens and bug slip through. Some issues are very specific and can pass all of our checks because of a single variable. One recent example is a bug that happens only with a specific compiler, and it took time to understand why devs couldn't actually reproduce it.
• Some annoying bugs cannot be reproduced locally, or on a server with only a small number of people. Some bugs are map dependent, some other depends on the number of players, or happen only in a very specific situation that will naturally arise more frequently on a full server. That was especially the case last summer/autumn, when each and every build we shipped to TM were seemingly affected by a crash we couldn't reproduce on our end.
• Delay before build deployment. Continuous feedback works well when it is... continuous, but the more the delay between the time we provide the build and the time it is installed on the server, the more it becomes useless. TM admins have their reasons to not put a build right away (often due to admin availability), but sometimes the delay was simply too long, introducing a "bug loop": if it takes several weeks to get feedback on a build we provide, then we can only fix detected issues after these few weeks, and the next build will undoubtedly have new issues introduced during these few weeks without any feedback. For those that don't understand what we mean here: quick update = faster feedback = faster fixes = less bugs overall. At some point the delay was so bad (~6 or 8 weeks at some point last year) that we even internally joked about keeping severe bugs intentionally, just to entice admins to update faster to the next build that would solve it (we never did that obviously, the codebase has already enough of these). It wasn't always as bad of course, but quite a few times we had time to prepare a new build before the old one was even deployed on TM.
• Very specific config setup. Because TM used to level up all players to maximum levels automatically, there was a whole range of features and bugfixes we never had feedback on. This is one reason the Prestige feature was introduced earlier than planned  - beside the fact that vanilla behavior has an overall much better gameplay, because the original game balance has been built around it. I won't enter the detail of the Prestige feature feedback here, but to keep it short it's a good step in the right direction, but still underwhelming.

There are others:

• Errors in TM configs which might have skewed feedback results. Over the past few months we've heard a lot of complaints about how the airstrike/artillery strikes were too numerous, with some demands to give more options to limit them - despite the feature having been reworked exactly to provide better airstrike limitation to server admins. All efforts to identify and reproduce the issue locally failed. It is only when plain access to TM logs/config were granted that some missing parenthesis were identified in the server config, preventing the existing cvars to be correctly taken into account. To this day I have no idea how the rest of TM config worked at all during all this time... Another similar situation was introduced with the prestige feature, with initial settings not set as requested which led to corrupted data we couldn't use to determine how well the feature worked. This lead to an extra database reset which didn't make anyone happy, but which was ultimately necessary.
• Very superficial reports. I certainly don't want to dismiss the effort taken by the players that actually reported issues (thanks to all of you), but I feel the vast majority of bugs have been caught by devs playing on TM themselves. As an example, corpses aren't gibbed by explosions on the current TM version. I've seen that issue by randomly connecting to the server, and there is no way nobody didn't notice it at all for several weeks. Players simply don't bother to report issues.
• Tensions between TM admins and ETL. When interacting in TM forums, I kinda noticed some patterns by "reading between the lines" and felt some people at TM feel we push update to your server just because "we do as we please" and are "playing with code without any consideration for the players". I expressed these concerns in the private forum here in a more detailed way. While I expected a more open discussion about that particular issue, only one single person at TM answered (Bystry, who followed ETL dev quite closely the past year or so). His post is actually very interesting, so I'd suggest whoever can read that forum to have a look. Bystry also made a suggestion to get out of the described "stalemate" by proposing a test server. I actually didn't follow up right away, waiting for more input from TM side, but unfortunately, nobody else ever looked at that post again and these concerns were lost to history. (To answer Bystry suggestion now: test servers never worked in the past, and certainly not on the long term. We tried many times. We have one already, right now. Nobody makes the effort to go to an extra server to do testing). Overall, I am pretty sure more communication between TM admins and ETL would have prevented many unnecessary tensions, but the stalemate remains.

 

// Now

Since then, a few things happened. On discord, it was made clear that ETL1/ETL2 aren't servers to get any testing on new builds, and that the builds should be tested before being sent to TM (paraphrasing a bit, but you get the idea). Yes, the build actually are tested, but it isn't a dichotomy "tested/not tested". It is "tested within the best of our abilities and limited time on a small scale on our test server that not many people play on" vs "full TM server battle-tested".

We can't do more by ourselves, certainly not considering our small size. If TM expects perfectly tested builds and no occasional issues, then there is no point in shipping new builds that will be lacking in one way or another. We try our best, but it's not sufficient. Bugs will happen, and sometimes big ones - for all of the reason mentioned above.

For all of these reasons, I cannot reasonably recommend any further build upgrade. Can't have the latest fixes, because they most probably come with their own set of new issues. Can't have the cake and eat it too.

// It's even worst

The current situation certainly produces lot of stress and frustration on dev side. It's never easy to ship a build knowing we might have missed something, and we usually connect on TM as soon as a new build is set to do debugging as fast as possible - in case something bad happens. But adding to this the need to manage expectation of potentially frustrated people, without a way to talk about concerns openly is something else entirely.

It's even worst when we implement critical features that have been public for months, talked more than once by the team, that we deem necessary to modernize the game, as the reason it wasn't done before doesn't exist on modern computers anymore. That inevitably create outrage because 15-year old habit are modified a bit. Worst, it creates high tensions in the dev team: nobody wants to get shit upon (we have had enough of it already), and standing ground to get stuff done instead of going for a quick revert right away is not easy (recent example: explosion rework).

There is a whole other chapter about the competitive side of the mod, but expending on that alone would certainly require twice as many word as already written above. To keep it short, let's say that everyone agree to make Legacy more appropriate for modern technology, until the point we actually change their habits built on 15-year old habits based on crust code that doesn't make any sense today. Be damned if you dare change anything, you fool! Also, be damned if you don't!

 

// The future

I probably ranted way too much already, so let's go to what you want to know: what happens now?

• ETL Legacy servers have been flourishing lately, and many people around are asking for newer builds.
• While ETL is open source and anybody can build by himself, the process isn't exactly easy. We didn't make it easy on purpose, as we tried to avoid spreading builds that might be buggy. So far, only TM got the build directly provided by us.
• These latest builds have new features but also security fixes, and so far only TM can easily enjoy them.

Thus, it is time to make ETL test builds available to a wider range of servers and players, and also, time to end the exclusive relationship that started a year ago between TM and ETL dev team.

You can read about our plans here.

To quickly recap:

• ETL will provide public snapshot directly on the website, for anyone to be tested
• Players can test the latest ETL client (which was never distributed, we only shared the mod code with TM).
• MacOS players can now play on latest macOS version, as 2.76 client doesn't work currently
• Linux players can play on their latest distro version, as 2.76 client doesn't work anymore
• Server owners can install the latest server and Legacy mod version
• Regular update is highly encouraged (server side), or forced (client side). The stable version will still be available.
• We won't provide the info directly to you (in this forum), nor we will post in each interested community forums, but make it available publicly on our website/discord. Everyone interested can follow and test.
• These build are marked *unstable* by default even if they are more stable than anything before. Don't expect any stellar quality, bug will happen.
• Bug reporting to *us*, on discord or github, is encouraged. We'll not go fish for bug reports in the various community forums. 

Work for setting up these public builds has been done already. There are some non technical circumstances that slowed progress, but we hope to fix these shortly. It will certainly take time to create a servers base giving feedback, but we hope this will serve well for 2.77 release.

Of course, TM is welcome to join in testing these builds.

I'm tired after hours writing the above, so excuse me if some bad wording slip through. Many other aspect weren't touched on, but feel free to ask any question.

Edit: Holy cow, that went even longer than I imagined.

 

// Join us

Also, I almost forgot:

• ETL team is small. Very small. We always need help, the more the better.
• In case you still have too much sanity left, don't hesitate any further, join us! We provide no pay, only extra work after your 7-5 main job, no special status nor any recognition, you'll probably get shit on by the community at some point, and you'll be delighted to deal with the occasional moron demanding a quick release after throwing an insult or two.
• Requirement: probably some coding skills, but artists are welcome too! Being a psycho to handle the pressure might be required, but maybe you are one already without knowing it! (NB: compensation for follow-up therapy sessions not included).
   

Edited June 9, 2020 by Spyhawk
typo

[ZeroDC00L]

This was a fascinating read. 

All I can say is THANK YOU to you and the rest of developers! 

I am definitely grateful for all that you do, and you do it for free. 

 

And if needed include me in the testing server if there will be one

 

Cheers 

[Duro]

If i understand your postt well enough i think it's a good thing that the builds are getting public? Public = wider range of players = more testing = more feedback = more bugg fixing?

Thanks for all the effort anyways :)

[Spyhawk]

Theoretically, yes.

In practice, it will certainly take time to get feedback, but nobody knows how long. Code quality will certainly be lower initially, before increasing again in later snapshots.

[slowtobi]

On 6/9/2020 at 12:06 AM, Spyhawk said:

Thus, it is time to make ETL test builds available to a wider range of servers and players, and also, time to end the exclusive relationship that started a year ago between TM and ETL dev team

Thanks for the story- i think you took the right decision. Regarding bug hunting, i can tell you from my side, that last year when you started to release test builds for the TM Server i thought, wow, gaming might become useful actually to keep this game and the community around it alive. But i never ran over bugs which actually bothered me or were not reported in your redmine or the TM forums (apart of server crashes in autumn last year).  So im just grateful for running a stable linux client. 

You are doing a great job, and ETL Server counts will increase, looking at splatterladder Mod release version, might become more interesting now.

 

[Dmxj]

Hey Spyhawk!

Thanks for the topic above!

First of all i wanne thank you for the enormous effort you guys put into this "Game" update.

Like you said before, the feedback points can be better from our perspective but you made a good decision to make the releases publc. More viewer = More players = More feedback = better feedback = faster bug fixes.

Thank you and see you soon!

Ps: don't forget to add fishes into the "oasis water pumps' 

[Sebast1an]

Fascinating read.

It is a shame some people can't recognize the effort the ETL dev team puts in, without any compensation, on top of their real life obligations. But I guess everytime you are dealing in any form of customer service, some bad apples are always bound to show up to throw some insults.

I feel like players need to get better at dealing with small issues, like a reset. I mean prestiges take some time to gather, as does SR, but in the end neither will really help your gameplay. Inconvenience for sure but really nothing more. Just be happy the server runs smooth and you're getting the frags. From my experience I think bugs that truly affect every day gameplay are rare and in the event of one it is usually promptly reported/fixed.

Thank you for all your work Spyhawk and the rest of the dev team.

[ryzyk]

On 6/9/2020 at 12:06 AM, Spyhawk said:

It's even worst when we implement critical features that have been public for months, talked more than once by the team, that we deem necessary to modernize the game, as the reason it wasn't done before doesn't exist on modern computers anymore. That inevitably create outrage because 15-year old habit are modified a bit. Worst, it creates high tensions in the dev team: nobody wants to get shit upon (we have had enough of it already), and standing ground to get stuff done instead of going for a quick revert right away is not easy (recent example: explosion rework).

I can only speak for myself but as I got back to the game only few months ago I've never seen or heard about this particular change coming. I feel the problem might lay in what actually people want/expect from Legacy. I for sure didn't come back to play ET just to see its heavily modified game that doesn't resemble what I used to play back in the day. I started using Legacy over vanilla client because it was more modern but at the same time the server felt like old game(even more so than the only alive ETpro public servers).

The community is small, there really isn't much servers to play on and you can only play at specific hours, so when people that played the same game for many many years react like that for even the smallest change(and explosion rework is not small in my opinion) is not strange for me because they are here to play the game they have always played. Even though small, it's rather mixed group of people with different view on the game, some more competitive than others, coming from various mods and so you will get mixed feedback.

Anyway thanks for the write up, it was nice to get a view on how the ETLegacy development is going and I hope that someday the game will see a healthy surge of new/old players.

[Ctrz]

On 6/10/2020 at 7:35 PM, Sebast1an said:

Fascinating read.

It is a shame some people can't recognize the effort the ETL dev team puts in, without any compensation, on top of their real life obligations. But I guess everytime you are dealing in any form of customer service, some bad apples are always bound to show up to throw some insults.

I feel like players need to get better at dealing with small issues, like a reset. I mean prestiges take some time to gather, as does SR, but in the end neither will really help your gameplay. Inconvenience for sure but really nothing more. Just be happy the server runs smooth and you're getting the frags. From my experience I think bugs that truly affect every day gameplay are rare and in the event of one it is usually promptly reported/fixed.

Thank you for all your work Spyhawk and the rest of the dev team.

I agree. There is a thin line between blatant accustations and beeing everything except appreciative for their work AND feedback. I still havent filed in the issue for my pc FPS dropping from steady 125 to 76 while I can do for example CS:GO high settings. But to be honest - I have also not looked through all the options as I've been using 95% vanilla configuration for the longest time. I'll try to get @m00f to help me with conf and see how it goes and if the issue is with my equipment or something else.

I know there are tensions but they will remain if things are not talked through and discussed how to proceed. In almost every free solution comes a breaking point where devs and community see things differently and its often overlooked resulting one side giving up and leaving.

We surely dont want that and for sure we need to talk more whats going on and try to see the challenges from all perspectives. 

C#

[m00f]

On 6/14/2020 at 11:32 PM, Ctrz said:

I'll try to get @m00f to help me with conf

So much pressure I'm scared!

[Ctrz]

1 hour ago, m00f said:

So much pressure I'm scared!

pff. Never heard Austrian people beeing scared of challenge..  

[Oldman]

@Spyhawk

Super interesting read, thanks for that. And also kudos to you and the ETL team - I would not have dared to install ET again a few months ago, until I accidentally discovered your work, which, since then, has provided me with many(!) hours of gaming-fun on the Muppet server! Love it...

Quote

the vast majority of bugs have been caught by devs playing on TM themselves. (...) Players simply don't bother to report issues.

I think that's natural, as

1. you look very differently at the game compared to regular players that most probably have no clue of 3d-game-development; I mean you've aten the red pill (admit it!), Neo! I haven't noticed the explosions-dont-gib bug. But at the pace, the game gets with 30 players and everything levelled up, I'd guess things like that are easily overseen; my attention is at heads popping up, not at bugs.    
2. 99 out of 100 players don't even know that you request bug reports, and how and where you request them? For someone who has experience in software develpment, it might come natural to visit a site like github and provide a reproducible example with a demo. I don't even have a demo of most of the games I play.
3. "Thomas Bayes" - you think of Skill Rank, I think of an underground gabba dj in rotterdam

Quote

The too high expectation of players generates a lot of stress and frustration that have lately reached an unhealthy level for developers (at least for me, personally).  

There are always nagging people. Goldrush is to frequent. Fueldump is too white. Proning is too lame. Don't overrate it, but look at the many silent folks that play ETL with great passion and appreciate everything behind the development of it. Unfortunately, appreciation for work is often a silent thing. But I hope it's still on your radar.

Thanks again and best,
Oldman
 

Edited June 17, 2020 by Oldman

[Sebast1an]

Are the constant lagspikes that have lasted for weeks now under investigation? Didn't see a post about them here so figured I'd make sure they are being dealt with.

If you're not familiar with the issue, it seems like every time someone connects to the server there's a lagspike. On my lagometer it usually last 2-5 seconds but it gets annoying as the server is quite popular and people connect all the time. Took me a long while to figure out it wasn't my internet but the server causing it. I think kyuss pointed it out that it's everytime someone joins the server and never on any other time.

[Eagle_cz]

This 'can be' connected with our redirector (iptables prerouting for players connecting via 27960/27964). Will check perf. tables for that...

Maybe try to connect to the origin port 27999 and let me know if u have a same problem. Thx