View Issue Details

IDProjectCategoryView StatusLast Update
0004511unrealircdpublic2016-01-13 11:07
ReportervNode1 Assigned Tosyzop  
PrioritynormalSeveritymajorReproducibilitysometimes
Status closedResolutionno change required 
Summary0004511: Endless server mode fight - may be Janus related
Description-
[11:08] -irc.vnode.ovh- *** Unknown channel mode +& from server irc.FknBish.com!
-
[11:08] -irc.vnode.ovh- *** Unknown channel mode +& from server lynx.vnode.ovh!
-
[11:08] -irc.vnode.ovh- *** Unknown channel mode +& from server irc.FknBish.com!
-
[11:08] -irc.vnode.ovh- *** Unknown channel mode +& from server lynx.vnode.ovh!
-
[11:08] -irc.vnode.ovh- *** Unknown channel mode +& from server irc.FknBish.com!
-
[11:08] -irc.vnode.ovh- *** Unknown channel mode +& from server lynx.vnode.ovh!
-
and just keeps flooding until you squit
Steps To Reproduceit's random.
3rd party modulesNone Currently

Activities

vNode1

2015-12-27 07:13

reporter   ~0018950

[11:08] * Joins: Turner ([email protected])
[11:08] * Joins: LouLou ([email protected])
[11:08] * wiki.gigirc.com sets mode: +aovv Turner Turner Turner LouLou
[11:08] * triv.irc-wiki.org sets mode: +l 51
[11:08] * Bootsy/triv sets mode: +v therock247uk[bnc]
[11:08] * twisted.vnode.ovh sets mode: -l
[11:08] * wiki.gigirc.com sets mode: +l 51
[11:08] * twisted.vnode.ovh sets mode: -l
[11:08] * wiki.gigirc.com sets mode: +l 51
[11:08] * twisted.vnode.ovh sets mode: -l
[11:08] * wiki.gigirc.com sets mode: +l 51
[11:08] * irc.FknBish.com sets mode: -l
[11:08] * wiki.gigirc.com sets mode: +l 51
[11:08] * twisted.vnode.ovh sets mode: -l

This happened right before the "attack"

syzop

2015-12-27 10:20

administrator   ~0018951

I see. Could you tell me the linking topology and unrealircd versions at that time?
Eg: wiki (3.2.x) -- twisted (4,'.x) -- ...

syzop

2015-12-27 10:41

administrator   ~0018952

Nvm, connected - I cheated, to find out :D

irc.FknBish.com (4.0.0)
`-irc.vnode.ovh (4.0.0)
  |-rock.gigirc.com (3.2.10.4)
  | |-wiki.gigirc.com (??)
  | | `-triv.irc-wiki.org (??)
  | `-dubstep.gigirc.com (3.2.10.4)
  |-twisted.vnode.ovh (4.0.0)
  `-lynx.vnode.ovh (4.0.0)

so irc.FknBish.com and lynx.vnode.ovh were both on 4.x and all those servers and the ones directly between them were 4.x?

syzop

2015-12-27 10:42

administrator   ~0018953

Then regarding your 2nd post, the +l / -l flood:
What is wiki.gigirc.com? And triv.irc-wiki.org? One of them isn't unreal, right? So some sort of services? Which? Something self-coded?

Also what modifications / 3rd party modules are in use? For example I see a nick with a slash (/) in it :)

vNode1

2015-12-28 02:16

reporter   ~0018957

They are Janus, but the error does not always happen rock is linked to a 4.0

everything links to irc.vnode.ovh not the others, I dunno why it maps like this

dboyz

2015-12-28 02:59

reporter   ~0018958

I might be wrong but this (https://bugs.unrealircd.org/view.php?id=4511#c18950) looks like a desync.

"&" is indeed an unknown char.

dboyz

2016-01-01 04:39

reporter   ~0019002

Additional information, we'll see if this is useful:

<kevin> I mean, it only happens when a server gets linked but it does not happen every round

syzop

2016-01-01 17:58

administrator   ~0019004

dboyz: same network as reporter?

dboyz

2016-01-02 02:23

reporter   ~0019006

Yup, I believe the reporter is kevin. Anyway I think I found the cause to this, please bear with me.

Unrealircd uses 'mode bounce' (i.e. modes are removed) for burst handling (i.e. initial s2s linking) [1]. I might be wrong here but I believe that 'mode bouncing' is handled during SJOIN using do_mode() [2] and char '&' is used to denote 'mode bounce'.

The paracount_for_chanmode_from_server() function added recently to check for unknown channel modes however does not handle the '&' char [3]. This function is called in do_mode().

I'm not in the position to tell if 'mode bouncing' is working or broken though.

[1] - http://www.stack.nl/~jilles/irc/mode-handling.txt
[2] - https://github.com/unrealircd/unrealircd/blob/unreal40/src/modules/m_mode.c#L429
[3] - https://github.com/unrealircd/unrealircd/blob/unreal40/src/modules/m_mode.c#L1342

syzop

2016-01-02 09:52

administrator   ~0019007

Right, so it's a bounce and.. just assuming for a moment that it's properly sent (at the beginning of the string) then the notice is unnecessary. The setting in that actual function (set_mode) is not broken, though, it's just the notice.

So I can and should fix the notice BUT.. the underlying problem of the reporter is a flood, it means a server fight is going on. Fixing the notice would only hide that problem, like: the ircds would still consume a lot of bandwidth in the background sending these messages back and forth but you wouldn't see it while it's going on... only when a split happens like with sendq exceeded but that might not even happen.

Anyway, so it's bounce related. Could be an an issue there like a server reacting on a bounce when it shouldn't. I wish we could get to see the server to server traffic when this issue happens. (OR that I could reproduce it ;p)
If the servers were linked the exact same way as in my /MAP then both servers fighting are UnrealIRCd 4. But perhaps it wasn't using that exact linking topology, that would be one way to explain the randomness of when it occurs vs when it does not occur. Can't tell. Could also be some other factor.

dboyz

2016-01-02 11:07

reporter   ~0019008

The mode bouncing was just an initial guess. Since kevin mentioned that it only occurs during initial server linking.

syzop

2016-01-02 11:08

administrator   ~0019009

vNode1: could you edit the src/modules/m_mode.c file at two places:

1) Around line 533 there's this code:
sendto_server(cptr, 0, 0, ":%s MODE %s %s%s %s%lu",

All you need to do is change the %s%lu to %s %lu (so add a space between it)

2) Around line 1384 there is:
    sendto_realops("Unknown channel mode %c%c from server %s!",
        (what == MODE_ADD) ? '+' : '-',
        mode,
        acptr->name);

Add below that:
    sendto_realops("Backupbuf was: '%s'", backupbuf);

After that you recompile, on the command line: make && make install

Then /REHASH the IRCd. There is NO need for a restart as all this code is in a module.

The 1st change fixes a bug - but not sure if it really applies to you to be honest.
The 2nd change will make it so ircops receive the full mode command when your flood issue occurs. This will aid debugging.

For the debugging stuff just changing irc.vnode.ovh is enough (if that's always the server that prints out the notices).
For the 'fix' you would need it on all hubs.. but I'd say if you are lazy then just only change irc.vnode.ovh for now ;)

Thanks in advance!

syzop

2016-01-03 15:41

administrator   ~0019012

Will look into it again once we got new information from reporter (see previous comment) :). May take a few days, no problem.

dboyz

2016-01-04 05:28

reporter   ~0019013

We've got bad news.

[08:27] <+Kevin> Good, we've delinked the other server so its not a problem
[08:27] <+Kevin> I'll probably also be switching software anyways
[08:28] <+Kevin> I don't really want to debug software I work 10+hours a day
[08:28] <+Kevin> debugging irc software is the last thing I want to do after a long day of work lol.
[08:29] <DBoyz> Lol
[08:29] <DBoyz> Quite true

Perhaps we should consider releasing products that won't drive people away. :/

dboyz

2016-01-04 06:02

reporter   ~0019015

Oh also, they have delinked janus from their network. We are effectively unable to get their assistance.

syzop

2016-01-04 10:58

administrator   ~0019017

At this point in time this is the only IRC network we know of that is experiencing this issue, that's why I asked to change two lines of code to get more information. It's not possible to acquire such information otherwise so this is rather crucial for us. As this was only my second request (or third, but I answered the first one myself) I don't think this is unreasonable from my side. Of course it's entirely up to the reporter to do this or not. It's a free world and we all have our own responsibilities and choices to make.

I can perfectly understand the reporter being angry when software is causing issues for him. What I don't find professional or reasonable on the other hand are statements by dboyz like "Perhaps we should consider releasing products that won't drive people away".

Anyway, regarding the future of this bug report: I hear - 2nd hand - that this network is no longer interested in solving this and thus won't add the necessary debugging lines. Is this really true? Perhaps the reporter, vNode1, can respond himself.

dboyz

2016-01-04 11:41

reporter   ~0019020

I take back my words.

I tried my best to relay messages to the reporter and help out at much as I could.

vNode1

2016-01-04 17:04

reporter   ~0019021

If time permits tonight, I will add it to the main server, I can't see how this will help as the janus link has ended. both parties got frustrated before there was a "possible" solution. I said this to dboyz, and I'll say this to you:
[09:36] <+Kevin> I've always used unreal lol
[09:36] <+Kevin> I will probably be trying inspircd out i think, no offence to unreal I just dont want to wait for stable-stable releases :P
[09:36] <+Kevin> syzop is very smart.
[09:36] <+Kevin> I'm sure it'll get figured out soon enough otherwise.
[09:37] <@DBoyz> I know
[09:37] <@DBoyz> But for people like you who can't wait

syzop

2016-01-04 17:42

administrator   ~0019023

Ok, thank you.

My first thought was also that it may have something to do with Janus, because issues have been reported before and it would also explain why you are experiencing this server fight issue while others don't. But.. I don't want to point fingers without evidence.

I'll update the title of the bug a bit.

blank

2016-01-04 19:41

reporter   ~0019024

Last edited: 2016-01-04 20:17

Test release candidates in future, that's what they're there for (especially when running custom modules, software etc. linked to UnrealIRCd).

vNode1

2016-01-05 05:17

reporter   ~0019025

blank although I appreciate your undesired feed back, it's a bit tougher than that to just drop a test server, the circumstances are NEVER the same. We did do testing on unreal 4 privately but the same situations never came up. thank you: )

syzop

2016-01-13 11:07

administrator   ~0019055

I think this either a bug in Janus (a piece of software which is no longer maintained and known to cause trouble) or related to a MODE bug I fixed earlier.

I'm closing this bug.

Issue History

Date Modified Username Field Change
2015-12-27 07:11 vNode1 New Issue
2015-12-27 07:13 vNode1 Note Added: 0018950
2015-12-27 10:20 syzop Note Added: 0018951
2015-12-27 10:41 syzop Note Added: 0018952
2015-12-27 10:42 syzop Note Added: 0018953
2015-12-28 02:16 vNode1 Note Added: 0018957
2015-12-28 02:59 dboyz Note Added: 0018958
2016-01-01 04:39 dboyz Note Added: 0019002
2016-01-01 17:58 syzop Note Added: 0019004
2016-01-01 17:58 syzop Priority high => normal
2016-01-02 02:23 dboyz Note Added: 0019006
2016-01-02 09:52 syzop Note Added: 0019007
2016-01-02 11:07 dboyz Note Added: 0019008
2016-01-02 11:08 syzop Note Added: 0019009
2016-01-03 15:41 syzop Note Added: 0019012
2016-01-03 15:41 syzop Assigned To => syzop
2016-01-03 15:41 syzop Status new => feedback
2016-01-04 05:28 dboyz Note Added: 0019013
2016-01-04 06:02 dboyz Note Added: 0019015
2016-01-04 10:58 syzop Note Added: 0019017
2016-01-04 11:41 dboyz Note Added: 0019020
2016-01-04 17:04 vNode1 Note Added: 0019021
2016-01-04 17:42 syzop Note Added: 0019023
2016-01-04 17:43 syzop Summary This happens ocasionally on a link from a server => Endless server mode fight - may be Janus related
2016-01-04 19:41 blank Note Added: 0019024
2016-01-04 20:17 blank Note Edited: 0019024
2016-01-05 05:17 vNode1 Note Added: 0019025
2016-01-13 11:07 syzop Note Added: 0019055
2016-01-13 11:07 syzop Status feedback => closed
2016-01-13 11:07 syzop Resolution open => no change required