|
|
| Author |
Message |
braindigitalis
Idler
Joined: 22 Sep 2003
Posts: 443
Location: IRC
|
 Posted: May 10, 2005 2:49am Post subject: |
 |
|
| olene wrote: | | All regex evaluations have a timeout of 200 ms, and invalid or complicated regular expressions won't even set. |
200ms is a very very long time.
My own personal opinion would be to only allow regexp to the trusted -- e.g. admins/opers 
If it takes 200ms max to execute a regexp, thats 5 of them equal to 1 second, 5 complex regexps can cripple a channel for 1 second, for one join. I think what is being referred to is not the amount of time taken to *compile* the regexp but to match it, as they use recursive features to create the match and each of those {1,1000} will recurse itself 1000 times in an attempt to create a match.
In case youre interested, my IRCd uses PCRE (in a module, for serverside message filtering like unreal's spamfilter) -- unreal itself in case you werent already aware uses TRE which isnt as 'top heavy' but at the same time is lacking some options (like backreferences) |
|
| Back to top |
|
 |
nenolod
Idler
Joined: 23 Jan 2004
Posts: 334
Location: A box!
|
| Posted: May 10, 2005 7:02am Post subject: |
|
|
| EviL_SmUrF wrote: | anywho, good luck with your ircd olene. however, you may want to consider making the multi-thread feature an option that can be turned on or off. many shell hosts won't like it, just like they don't like CR  |
No, many shell hosts won't like it because it's written in the CLR, and thus resource-intensive. Also, you obviously forget about several other .net ircds (some from olene, some not), which all have (to date) suffered from all sorts of design flaws. After all, we wouldn't want such an ircd to happen again, now would we? |
|
| Back to top |
|
|
nenolod
Idler
Joined: 23 Jan 2004
Posts: 334
Location: A box!
|
| Posted: May 10, 2005 8:50am Post subject: Re: roflircd |
|
|
| olene wrote: | | How is the ability to interface with any protocol, text or binary, breaking networks? I'd like to see you link unreal with hybrid, or bahamut with ultimate. You can't do it. With rofl, you can link to theoretically anything that uses TCP. |
Sure you -can-. http://cvs.shadowircd.net:81/viewcvs/viewcvs.cgi/bridge
However, it is a really stupid idea because it will cause a desync. |
|
| Back to top |
|
|
codemastr
Idler
Joined: 05 Feb 2004
Posts: 353
|
| Posted: May 10, 2005 10:05am Post subject: |
|
|
| olene wrote: | | Incidentally, codemastr, i tried both of those regex's and both completed in under 1 ms so i couldn't even measure them with my benchmark module. You must have them wrong. I'll look online for some more complex ones. |
It depends on what the input text is, as I mentioned, those weren't tailored for user@host input. In any case, 200ms is quite a lot. Again, consider 500 bots joining a channel. 200ms * 500 = 100000ms = 10 seconds of the server being frozen. Make those bots all join 10 channels and now you're at 10 * 200ms * 500 = 1 minute 40 seconds of freeze time.
Also, how do you define a "complex" regex? Sometimes a regex can look very simple and still be very inefficient. All it takes is someone to specifically craft an inefficient regex. As I'm aware, .NET uses a backtracking matcher. Therefore, I know that using a lot of repetition operators, on top of back references (which leads to some nice NP-completeness) will make the regex very inefficient. At the very least, I'd make sure to disable backreferences. Backrefs are very nice for matching drones, however, they are the most inefficient part of regex. |
|
| Back to top |
|
|
codemastr
Idler
Joined: 05 Feb 2004
Posts: 353
|
| Posted: May 10, 2005 10:27am Post subject: |
|
|
| Quote: | | TRE which isnt as 'top heavy' but at the same time is lacking some options (like backreferences) |
TRE supports back references (it's required by POSIX). The only very useful feature it does not currently support is lookahead/behind assertions, but that's on the TODO. |
|
| Back to top |
|
|
olene
Newbie
Joined: 31 Jul 2004
Posts: 61
Location: olene on DALnet
|
| Posted: May 10, 2005 8:04pm Post subject: |
|
|
Yeah. After i did some elaborate testing, i decided to make the timeout 4 ms.
Actually, the multithreading is optional via compile time define USE_ASYNCHRONOUS_PARSING and by setting LOOP_PULSE to 0 |
|
| Back to top |
|
|
olene
Newbie
Joined: 31 Jul 2004
Posts: 61
Location: olene on DALnet
|
| Posted: May 10, 2005 8:05pm Post subject: |
|
|
| Also, i found a speed increase by setting the compile flag on the regex's, which seems to make any subsequent tests (other than the first test) on a regex be <1 ms regardless of how complex it is. |
|
| Back to top |
|
|
codemastr
Idler
Joined: 05 Feb 2004
Posts: 353
|
| Posted: May 10, 2005 8:54pm Post subject: |
|
|
| olene wrote: | | Also, i found a speed increase by setting the compile flag on the regex's, which seems to make any subsequent tests (other than the first test) on a regex be <1 ms regardless of how complex it is. |
Once again, you need to think bigger. How many bans can be set on the channel, lets say 30. So I set 30 regex based bans. First of all, how you define "complex" may be different than what I define it as. You can have a regex that is 4kb long but is still efficient, and you can have a regex that is 10 characters long and is very inefficient. But lets ignore that and just go with 1ms. 30 bans * 1ms = 30ms per join. Flood with 500 clones and that is 30 bans * 1ms * 500 joins = 15 seconds of freeze. I still don't think that's very good... |
|
| Back to top |
|
|
PingBad
Guru
Joined: 05 Feb 2005
Posts: 2064
Location: New Zealand
|
| Posted: May 10, 2005 8:58pm Post subject: |
|
|
| so, how would you go about making it more efficient, codemastr? Many a time have you said that its inefficient, but what's your solution? |
|
| Back to top |
|
|
nenolod
Idler
Joined: 23 Jan 2004
Posts: 334
Location: A box!
|
| Posted: May 10, 2005 9:29pm Post subject: |
|
|
| PingBad wrote: | | so, how would you go about making it more efficient, codemastr? Many a time have you said that its inefficient, but what's your solution? |
There is no viable method to handle such advanced matching at this time. Regex is the best thing we have, and it isn't fast enough to be used in this manner. |
|
| Back to top |
|
|
olene
Newbie
Joined: 31 Jul 2004
Posts: 61
Location: olene on DALnet
|
| Posted: May 10, 2005 11:22pm Post subject: |
|
|
another benefit of multithreading... the only people that will 'freeze' are the people doing the joins.
Last edited by olene on May 10, 2005 11:40pm; edited 1 time in total |
|
| Back to top |
|
|
zeke
Idler
Joined: 04 Oct 2003
Posts: 320
|
| Posted: May 10, 2005 11:27pm Post subject: |
|
|
http://searchirc.com/forum-rules.php
| Quote: | Attack ideas, not people:
This is okay: "Your idea won't work."
This is not okay: "You are an idiot."
This also includes trolling (defined as posting specifically for the purpose of provoking a negative response from someone)
|
|
|
| Back to top |
|
|
olene
Newbie
Joined: 31 Jul 2004
Posts: 61
Location: olene on DALnet
|
| Posted: May 10, 2005 11:40pm Post subject: |
|
|
| What are you talking about? I think you have the wrong thread. |
|
| Back to top |
|
|
magpie
Idler
Joined: 18 Jan 2004
Posts: 454
Location: Essex, UK
|
| Posted: May 11, 2005 5:43am Post subject: |
|
|
| Personally I'm not even sure there's any point having regex bans in the ircd. Most users won't understand them and for large numbers of users they'll place significant loads on the server (which has already been addressed by people in this thread). For users that want them, why not just let them run their own bot that can perform regex matching itself? |
|
| Back to top |
|
|
PingBad
Guru
Joined: 05 Feb 2005
Posts: 2064
Location: New Zealand
|
| Posted: May 11, 2005 5:47am Post subject: |
|
|
| magpie wrote: | | Personally I'm not even sure there's any point having regex bans in the ircd. Most users won't understand them and for large numbers of users they'll place significant loads on the server (which has already been addressed by people in this thread). For users that want them, why not just let them run their own bot that can perform regex matching itself? |
having an external regex processor (as it were) would certainly lighten the load on the IRCd (as well as the server its run on). |
|
| Back to top |
|
|
|
Register
Log in
