SearchIRC updates

Jason · Posted: Jul 16, 2003 10:37pm Post subject: SearchIRC updates

- Tons of channels are being submitted to the Chat Directory -- we're up to around 450 channels now.

- Channels that are in the directory, will be displayed as such in the search results. ie; if a channel is in the IRC : Bot category, it will show as:

Directory: IRC > Bots

Example: look at the #3 channel on this search: http://searchirc.com/irc-darkbot-1

Jason · Posted: Jul 18, 2003 8:48pm Post subject:

I'm really excited about the new changes made today. Here's the rundown:

- Channel updates in the search results are now near instant. Before today's changes, it would take a very long time to add a 5 meg list of channels/topics to the database. Now it takes just a few seconds. What this means is - Less load on the server, less load on the database (used to have 3 concurrent processes trying to keep up with the load), and more up to date channel listings. This was a problem I was wrestling with for several weeks now. Glad to have finally taken care of it.

- Logging into your MyChannel will no longer log you out of the discussion boards. heh.

- 489 channels are now in the Chat Directory

- There are 5,583 motd's cached. Total size: 9.1 megs worth :)

- Network database now stores 989 networks.

- Topics using IRC color codes to make text white, will now be made silver instead. The problem I was having with this is -- some channels made their text white with no background color, leaving users on SearchIRC unable to see any text. I had initially just set it to ignore the white text color code, but that proved to be a problem when users used black background with white text...

- The http://searchirc.com/top5.php page now has network names in bold. Makes it a lot easier to read.

- Brand new channels that are seen for the very first time are now marked as "Brand New!"

- All graphs should start scrolling towards the right, instead of the left. I think this is more user friendly.

- The huge search box on the front page has been made a lot better looking. It's now enclosed within a rounded blue shape :)

- Front page "boxes" located on the right side of the page now have a cool drop shadow, giving them a 3-D look.

- Fixed a little bug that made going back to the first page on a search mess up and bring you to the front page of the site.

- Fixed the spelling error of 'offical' to official on about 3 pages. duh :)

Jason · Posted: Jul 19, 2003 1:23am Post subject: Reboot Alert

I will be rebooting SearchIRC tonight at 2:30am EST.

Reboot successful. Server was offline for ~3 minutes. Sorry for the inconvince.

Jason · Posted: Jul 20, 2003 12:31pm Post subject:

In the MOTD page for a server, now displayed is the server's "realname" field. This is the info displayed in /links.

From this example: http://searchirc.com/motd/EFnet/irc.aol.com

Displaying MOTD for irc.aol.com : AOL Efnet Client Server

^-- Realname field marked in red

Also the SearchIRC logo has been upgraded (Thanks Asmo!). The new logo has been rendered in a newer version of lightwave and so the graphic is much sharper and edges are no longer "blocky"

Jason · Posted: Jul 25, 2003 8:52pm Post subject: June 25th

SearchIRC has 3 major forms of robots,

1) Connects and stays online at all times (used on the major networks)

2) Connects every hour (used on medium networks)

3) Connects every 3 hours (used on the smaller networks)

I've updated the #3 robot to connect every hour. This means over 700 of the smaller networks that used to get a robot every 3 hours will get it every hour. The load doesn't seem to be much more than previous, so I'm going to keep it this way pending any unexpected problems.

This should help keep the smaller network stats more current, and the channel listings fresh.

Horizon · Posted: Jul 25, 2003 8:59pm Post subject:

Your awesome Jason! Still lovin' the site.

moonman · Lurker Joined: 15 Jul 2003 Posts: 212

aww i saw my robot an hour apart!

[8:40:36pm]

[9:40:25pm]

but he was JUST a bit early!

cute little fella.

Jason · Posted: Jul 25, 2003 9:19pm Post subject:

Two things:

1) The robot will be beaten for not being on time.

2) I'm shocked that anyone reads the updates page.

moonman · Lurker Joined: 15 Jul 2003 Posts: 212

no its ok you should be a little early thats the best way to make a good impression thats what my mom told me anyway

U · Eleet Joined: 18 Jun 2003 Posts: 521 Location: IRC

The beatings will continue till morale improves?

iigor · Newbie Joined: 18 Jul 2003 Posts: 86 Location: Toronto, CA

I mentioned one update... :oops: the smily turns red.. or it always had? hmmm... *off to think... anyway.. OFCOURSE we read the update page

Jason · Posted: Jul 29, 2003 4:29pm Post subject:

JFYI, my bandwidth has doubled since setting the robots to connect to the 700+ "smaller" networks every hour, instead of every 3 hours.

7/28/03 51.69 gigs
7/27/03 49.43 gigs
7/26/03 47.39 gigs

Because of the horrible amount of bandwidth that is being eaten up, I'm dropping back the connect interval to 90 minutes. Will keep everyone posted on how that turns out.

Jason · Posted: Jul 29, 2003 7:54pm Post subject:

ugh. I updated crontab with * */3 * * * instead of 0 */3 * * *, so it started loading robots every minute.... each robot checking it's own connection for other running robots for that network...

you can just imagine.

load average: 944.44, 546.34, 983.55

fixed now though :P

U · Eleet Joined: 18 Jun 2003 Posts: 521 Location: IRC

Jason, just a suggestion, and this comes from me running a 600-700 user network:

I don't know what your levels are currently as to defining size for a network, but I feel that my network for example is perfectly fine with a once every 3 hour sampling.

I would reserve the increased sample rates to networks with more users, say above 2000.

I wouldn't be offended by this in the least, and since my network is pretty static when it comes to channels, I think a 3 hour refresh rate is more than enough for anyone seeking anything on it.

But thats just my opinion Smile

Jason · Posted: Jul 29, 2003 10:03pm Post subject:

U, I agree -- the way SearchIRC functions, it leans towards established channels much more than small channels that are recently created. Doing things like caching channels allows for IRC failures such as netsplits, and network down times while giving a pretty accurate listing of actual IRC channels that exist. Because of the way channels get cached, indexing a network at a slower pace (ie; every 3 hours) doesn't negatively effect the search results, as the only thing that will be lost are the small transient channels that may appear (then disappear) during the times the robot isn't doing /list. Fact is, any established channel, will be here now, in 3 hours, in 12 hours, it will be here almost every single time a /list is done. Sometimes a channel will +s itself, or due to a netsplit, etc, may not currently appear.... regardless, my point is: SearchIRC lists the real channels that exist on IRC, and the info displayed is accurate regardless of the list interval.

I've been approached by admins of networks who want to know why their network's /luser info is outdated. It tricky to balance the two - even though I know SearchIRC does not list inaccurate channel listings, displaying /lusers with user counts that do not closely match a networks info gives the appearance of the info being outdated. The only way to mirror the /luser info closely, is more frequent connects.

On the other hand, I think most users aren't going to be watching the little user count on a network info page.