Message trimming, thread pruning, and other issues

[Lee Enfield-Burke]

Folks,

We've been starting to incur considerable extra charges for overuse of disk space at our servers.

There are a number of reasons for this kind of thing. One is the size of our message archives. When archiving became available, we went for it with pleasure. But our stats show that very very few of our users -- less than one percent -- are bothering to use the feature.

With this in mind, I am going to start "pruning out" our oldest messages, those more than three years old. If there is something more than three years old that you particularly want to keep, I urge you to find the message thread in question and save it to your own computer, as we will no longer be able to keep it on the servers. Pruning will start on Monday, January 8th, so if there's anything you want to keep, please move quickly.

(For members who're concerned: I will do everything I can to avoid pruning the venerable "Topic of Great Randomness".)

Also: old members and new alike, please remember the limits on signature lengths. The old limit of six lines ought to be plenty, but the sigs too take up a surprising amount of message space / server space, and if we have to change the rule to require shorter sigs, we will.

Thanks.

[Nathan]

Lee: Am I correct in reading your announcement as implying that every post made prior to December 31st, 2003 will be culled, even if it is part of a thread that contains more recent messages? I ask because there are any number of long-lasting threads that were started prior to the cut-off date that I would be interested in saving in their entirety, if this is the case.

While it would seemingly make no sense to lop off the earliest messages from which a thread developed while still leaving its later messages, it seems prudent to enquire whether you intend to do so, given your mention of avoiding pruning the "Topic of Great Randomness." (However, that topic, given its avowedly free-form nature, would suffer far, far less from the removal of its earlier messages—indeed, it has been pruned before, if I remember correctly—and this may be why you mentioned the possibility.) It is the fact that you mentioned removing "messages" and not "threads" that initially gave me pause.

I am greatly sorrowed by the fact that new members, for the most part, have not been searching through the archives and uncovering all the old threads, some of which contain, in my humble (and not unbiased, given that I contributed to several of them) estimation, some of the most probing analysis of the YW universe seen on these forums (and certain of these have already been lost during previous culling or forum glitches). (I always thought that it would be a good idea to prepare a list of fascinating older topics, organized by subject, to be presented to new members along with the FAQs, but I never even got around to suggesting this, much less doing any work on creating such a list.)

Given that you are, unfortunately, suffering from storage problems, I would suggest that you cull messages up to a more recent date in the "Chatter II" forums, rather than culling threads in those devoted to the books. This is, of course, not to say that all of these threads—much less all of their messages—consist of deep analysis, or are even on-topic, just that they are, as a whole, far more germane to the forums' purpose and are the result of much more thought than those in "Chatter II." For that matter, there are certain threads in "Chatter II" that are DD-related and might be well served by being moved elsewhere. (I would suggest pruning the "Topic of Great Randomness" as well, except that it appears to be held sacrosanct by many forum members.) However, I can see why you would want to hold to the plan given above, as it requires no individual value judgements, and it thus much easier to implement, not to mention being the one you have already announced.

Thanks,
Nathan

Edited to correct a few typos in the final paragraph.

[bluesalamanders]

I agree with Nathan.

I admit it hasn't really occurred to me to look at the old forum topics much until recently - I'm used to blogs, mostly livejournal, where it's rather uncommon to look at old posts. And it is slightly taboo, even here, to comment on something more than a few days, at most a few weeks, old. But now that I've started looking, I've found some topics that it will be a shame to lose.

And also plenty that it won't, of course.

As Nathan said, you may not want to change the plan now that it's been announced, and the current plan requires no value judgements. It just seems like many of the old topics - as well as many newer than 3 years old - could be deleted for the information being out of date, for the thread not really having much content, and so on, and that would not only allow the meatier posts to remain but would bring them closer to the top.

I also wouldn't particularly miss old TOGR posts, but do understand that many people don't feel the same way. (Though if I'm reading the stats right, it appears that it makes up nearly a third of the total posts? Would it really be so bad to start with a clean slate with that, to sacrifice it to save DD money?)

I realize that judging topics on a case-by-case basis would take a lot more time than just deleting anything that's too old, but in the end it would save a lot of good information, and, if newer than 3-year-old posts were culled as well, it might allow at least as much to be deleted.

Anna

[Gryphon]

I completely agree with Nathan. Short of any pruning of the Topic of Great Randomness. I am violently opposed to weeding out parts of the Topic of Great Randomness, because, while the YW books topics are full of the most insightful and rich discussions, the ToGR is where so many members leave their mark, even the ones who don't visit here anymore. We've even had some new members who came here and read _all_ of it, from start to finish. It's full of a lot of memories.

Please, do not ever delete the Topic of Great Randomness, even for the sake of a "clean slate." It has _everything_ in it. I don't think a lot of members realize what it means to many of the oldbies, and to some of the new members too, who write their heart out in it. A lot of the time, it's fluff, but it also has its other side, where members open up and talk about the things closest to their hearts. It's made a lasting impression on many of us.

[Sean L.]

[bluesalamanders]

Sean -

That sounds like a good idea to me and I would certainly be willing to help, presuming the Admin team agrees.

Anna

[Nathan]

I too think that Sean's idea is a good one, at least for future episodes of culling, even if it cannot be implemented for this round. In fact, one could easily combine his list of messages to be saved with my suggestion of a list of forum 'greatest hits.' Thus, one could not only point new members (and perhaps even some old members) towards some threads that either contain fascinating discussion plumbing great depths of the YW universe, or provide answers to various DD-related questions, but also simultaneously provide a rationale for those threads' not being deleted. Note that a hugely important subset of these topics, of course, are those in which DD has either weighed in herself on some YW issue, or in which her answer to a question in chat is relayed back to the members at large. While the latter are really only critical for the times/years when the DD chat logs were not archived (and, by the bye, I should mention that the logs of the very first two chats are still available using the Wayback Machine), they are still quite important for letting members know that those questions have been addressed in chats, instead of having to hunt around blindly in the chat logs. Similar logic applies to the answers and tidbits that various members have dragged up from USENET and Out of Ambit.

Those threads where the tidbit that's being reported is more important than the discussion could simply be summarized somewhere, perhaps eventually on the putative YW.net wiki (mentioned here, with updated information here), if it ever becomes operational. Quill's thread (here) enquiring (rather obliquely, due to an incomplete memory) about the image that inspired "Uptown Local" is an example (the best I can come up with off the top of my head) of such a thread (though this one is not slated for deletion in the current round of culling): All that is really of importance is the fact that the image was also featured as the cover art for Nancy Willard's Firebrat, and, while in case there are others who are convinced, as Quill and I were when DD originally posted the image, that they had seen it elsewhere. However, this would be even more work-intensive, and, like the list of threads to keep/recommend, is probably best undertaken as a labour of love by various forum members.

It would even be possible to combine such a campaign with the 'threads nominated for deletion' topic: If someone summarized the important points from an old and abandoned thread that contained little, if any, interesting discussion, but answered some important question, in, say, (at least at the moment, since the YW.net wiki is not yet instated) a new thread specifically devoted to such summaries, then that member could nominate the summarized thread for deletion (and anyone who felt that the board would be best served by retaining the thread in its entirety could then make arguments to the contrary).

However, by this point we're getting into what amounts to a wholesale overhaul of board procedures, which I'm not sure even the majority of members would want—I'm not even sure that I necessarily want it, as I am quite troubled by what I foresee as almost inevitable outcomes. (Some simple examples of such outcomes include members disagreeing over how to best summarize a thread, or even having the members who participated heavily in a thread take umbrage at that particular one being singled out for deletion. Some of the first problem could be ameliorated by asking the original author, if she or he still frequents the forums, to write the summary, or, failing that, allowing for composite summaries, or even more than one summary. The second problem could be mostly solved by making sure to credit the members who originally asked and answered questions, though thread deletion will still diminish postcount, for those members who worry about such things. But these solutions are far from perfect, and there are any number of other troubling aspects to all of this.)

Pace the above doubts, I still think Sean's basic suggestion to be a very good one and would be more than happy to assist with these projects, though I'm not sure how much time I would be able to commit.

A few thoughts about the 'suggested threads' list: In addition to the obvious inclusions of a link to the thread and a listing of its primary subject(s), perhaps given as keywords, any suggestion should include an 'advertisement' for the thread, indicating why members should read it, if they have not already (and perhaps even why those who have should revisit it, to see if they have any new thoughts—possibly in light of more recent books—on the issues it raises). This could take the form of a pr&eacture;cis/abstract of the discussion (including questions raised and answered), and should include particular mention of posts by DD or other definitive answers to questions. Less essential, but still useful, could be such things as a measure of topic length, and possibly even some sort of highly subjective rating. (And while Kathy's suggestion [here] that new members search for a topic before creating a new post on it is good—though she didn't mention there that she and Peter Murray, as Seniors, have the power to unarchive topics—this list would have the added advantage of pointing people towards topics that they might not have even thought to consider, but might wish to explore, and could even have some new insight into after thinking about them.)

Speaking of threads that are obsolete, while I didn't think of this when I wrote my previous post, there are any number of these in the "Forum News" section, where it might be a good idea to automatically delete threads that are more than a year or so old (perhaps even just a few months), unless they are somehow marked for retention. (I have no idea how easy this would be to do, however.) The FAQs, various long-standing admonitions to the members and any important forum/chat 'fixes' or 'how-tos' are some of the only threads I can think of that would need such treatment, and the latter two should probably be incorporated into the FAQs (as I believe many are already).

Just for the record, I have never felt that replying to old messages was the least bit frowned upon, provided one was offering something of substance—certainly I have never felt the slightest compunction about doing so, even, I believe, to threads that had languished without any new comments for over half a year, and had been archived. And while I had not participated in any forums or suchlike prior to these, and picking up on unspoken niceties is far from being one of my strong suits, I think that the Seniors' willingness to unarchive topics upon request really indicates that, while members may rarely do so (probably because they, for the most part, do not use the archives), resurrecting old topics is far from verboten, and may even be welcomed, in that it can bring some of the old, fascinating topics to the attention of new members.

[Sean L.]

[marina salem]

Oh no, Sean...no objections. Really, you think of everything (said in slightly sarcastic tone of voice that isn't intended to give offense). I'll help you, if you want, but it'll take me a bit longer to do a forum because of my intricate, messy schedule and nosy parents who always ask what I'm doing when I get on my laptop.

Just tell me what to do...to quote, "Lead on, Macduff."

[Kathy Li]

Just a short topical diversion, but I would like to take this opportunity to float the suggestion that everybody take a few minutes to sift through older threads and see for themselves what's worth looking at and what's now meaningless noise a year or two on, just to get an idea of what to aim for when posting in the here-and-now.

I remember back in the Cro-Magnon days of the internet (when I posted on USENET), someone sent me an email encouraging me to post a lot more, because everything I posted was worth reading. And I had to tell him that he'd gotten it backwards.It wasn't that everything I posted was worth reading, it was that I only posted when I had something worthwhile to say...

[alla]

I'm for the pruning of some topics, but there's this nagging attachment to the TOGR, even if it is taking up nearly 1/3 of all posts.

On the topic of costs, is there some way that we could donate to the upkeep of the boards for those of us who might be so inclined? If there are any others. (I know that I would be).

Alla

[RhyanonSorenson]

I like Sean's idea.

I have an addition to that though. Why not have a start date and an end date to the culling? Between the two dates, we could all go through and use that report button. In the report, we can give our reason why that topic should be deleted. If multiple people report it, say more than ten or twenty or so, it would be deleted. The report feature basically says "We hate this thread's/posts guts." (Not really, but it does say we don't care about this thread enough to have it saved, go ahead and hit delete on it.)

[Sean L.]

Well apparently, that didn't work.

[bluesalamanders]

Yes, I am disappointed about that this wasn't even acknowledged.

Blue~

[Kathy Li]

I have to applaud the effort, but I can also see the admins' pov on this: it's a matter of time, convenience, and fairness. Applying a straight-across-the-board rule like "it's this old and nobody's read it for two years..." is a lot fairer than say a possibly prejudicial personal judgement. Who's to say you couldn't have a weird vendetta where someone wants to eliminate everything by one specific poster they hate?

And then there's the time involved in individually picking and pulling things out, vs. letting a machine handle it by a specified, consistent rule.

Those of you who are relatively new to the board probably don't realize that a culling like this has happened already. You will not find my 200+ line post on "Origami on the Internet" links, nor my fictional bug report on the gamma version of the Manual software. They are gone into the aether. The space here is finite. This is one of the reasons why you have to use it wisely.

As for why we can't chip in--well, that's probably twofold. 1) passing the hat can't be relied upon to pony up a certain amount every month, and 2) whoever pays the bills has control. This is not a democracy, folks. It's DD's space and what she says goes, and personally I'm fine with that. I'm grateful she's bothered to establish and maintain this board. It's not an easy thing to do, nor a small committment. And if you don't like how this place is run, then set up your own board and make it good enough for us to want to visit.

Ideally, we should be doing this kind of culling ourselves, all the time, every time we post. It is our responsibility to see that the message space is used wisely.