Wishlist: "Knowledge Base" feature

[floid]

Short summary: A formal "knowledge base" feature would bridge the gap between "Bugs," "Answers," and external project documentation that never gets written.

My main experience with Launchpad is through Ubuntu and Ubuntu-related projects, but I think this is a general suggestion, so I'll place it here for comment. Ubuntu is the "flagship" Launchpad project, so please consider all references to it in that context. "Issue" as used here encompasses all possibilities of "bug," "question," "quirk," or "feature" that could possibly require documentation.

A lot of criticism directed at Ubuntu relates to the quality of documentation and community support. Other projects are also not immune to the phenomenon of documentation lagging reality (or never getting written at all). In the FOSS "bazaar," it is often difficult to find an authoritative answer to even the most common questions. In fact, the ability to locate documentation can be observed to degrade based on the prevalence of an "issue," as puzzled users are doomed to propagate discussion and theories that will then be indexed for web search.

I propose that a "Knowledge Base" feature in Launchpad would help remedy this situation by allowing projects to maintain and improve their 'institutional memory,' and also ease the process of producing effective documentation.

First, to explain what I am talking about, let me define the properties (and responsibilities) of a "Knowledge Base" article:
* Is a single written article that conveys true and accurate information;
* Makes every effort to ensure the information is consistent with documentation / "best practices";
* Identifies and acknowledges any inconsistencies with, or deficiencies in documentation;
* Identifies which versions of a project the information is known to apply to;
* Identifies which versions of a project the information may (but has not been formally confirmed) apply to;
* Is subject to Wiki-like revision by anyone on Launchpad;
* Is subject to editorial review by project subject-matter experts, with reviewed versions favored.

This is in contrast to a "bug" in Launchpad terms, which is intended to produce discussion leading to modification or correction of software, or an "Answer," where a specific, individual question is handled (with exactly as much detail and expertise as emerges in the discussion).

As it stands, Ubuntu and other Launchpad projects are attempting to achieve this goal through external wikis, but there are some problems. Foremost, the people most familiar with the bugs are active on Launchpad, but may not have access to wikis that have been locked down against spam and vandalism. Next, wikis being wikis, determining how current and trustworthy a given article is can become a complicated game of review that each individual reader must undertake.

If, instead, bug participants were invited to contribute to a KB article when their bug is closed, and others are free to author an entry when they feel they can speak with authority, the available documentation would increase dramatically and duplication of effort could be largely avoided. These articles could then be reviewed as the designated project personnel can get to them, and either approved, corrected, or discarded.

Metadata could also be included as to the reviewer's opinion of the quality/satisfactory nature of the article, and possibly also its subjective "importance." That data, combined with basic demographics like the number of page views / voted popularity of the article, could be useful in countless ways, from encouraging further revision to flagging the knowledge as important to anyone writing external documentation.

Here is a contrived "user story" to illustrate the potential usefulness:

1. Bob is struggling to add a user to his Ubuntu system and, in desperation, posts to a forum.

2. Alice replies to Bob, explaining `vipw` and attaching a Bash script she wrote to edit the desired login image into gdm.conf.

3. Bob, impressed with this arcane knowledge, logs into Launchpad and creates KB entry #321 for the Ubuntu project, "Adding a user." The system forces him to identify the version he knows it applies to (8.10) and lets him separately specify which he thinks it may apply to (all versions past and present).

4. Eve, a designated reviewer, stumbles upon the article a few weeks later and rewrites it to document the GNOME user management facilities. The article gets marked as reviewed and trustworthy, as far as these things go.

5. Trent, a more advanced user, stumbles on Eve's revision and is chagrined that there's now no mention of command-line user management. He drafts up KB #322, "Adding a user from the command-line," and edits #321, preserving it but adding the cross-reference. Regular users still see Eve's edit when conducting a basic search, with a "There is a newer, but unreviewed version of this article." link at the top. When viewing Trent's version, they're warned "This article has not been reviewed. Click here to see the last reviewed version."

6. Eve gets around to reviewing Trent's modification, and approves it. KB #321 is now as useful as it can be (at least until someone adds new xrefs for the Kubuntu and Xubuntu users!).

7. Months later, the project is preparing to cut a new release. Zoe, another designated reviewer, runs a search for entries marked as applying, or maybe applying, to it. There are 5,000 results, so she triages them by refining the search by popularity and component. #321 is a very popular article, and the user management GUI changed quite a bit for this hypothetical release, so she edits #321 one last time and creates a new article for 9.04. By the time the release is out, the top 500 topics have been brought up to date, and users are already submitting articles documenting the other new features of the release. [The KB articles on "hardware known to work with 9.04" and "hardware known to have problems with 9.04" are quite popular, as well.]

...

I think that makes the case. For smaller projects, developers might choose to write about features as they introduce them, preserving that information in the KB instead of (or together with) putting it on a weblog or project page where it might expire or become inaccessible.

[Diogo Matsubara (matsubara)]

Launchpad Answers have a FAQ feature <https://help.launchpad.net/Answers/OfferingHelp#Handling frequently asked questions>. Would that work for your use case?

[floid (jkanowitz)]

Hm, I've been noisy about this, thanks for your consideration. :)

I have a gut, "social engineer's" feeling that a 'strict' KB has a couple of subtle? advantages:

* Familiarity: People are familiar with the level of detail and completeness required in [something that aspires to be] formal documentation. I have a feeling "Answers" and "FAQs" inspire people to be concise, answering the specific question as directly as possible [without writing, for instance, an "Other Considerations" section] and moving on.

* Completeness: Per the above, a good KB entry will pull all relevant information into the KB, inspire people to reinterpret documentation (or discover/acknowledge gaps in documentation), and reduce the number of documents/tabs/... a human has to peruse to "get a grip" on some particular topic, be it general ("Frequently Asked?") or specific.

* Collaboration: Looking at an entry created in the Launchpad FAQ system -- I picked https://answers.launchpad.net/soyuz/+faq/167 -- it appears the system is one-way; when someone [limited to people registered as an Answer contact?] finds a good answer, they can nominate it up to being a FAQ, but there is no in-band [on the page] mechanism for other people to improve on that answer; out-of-band I assume they can jump into the original Answers thread or chase down the author. The model I'm proposing, basically a wiki with blessed checkpoints, makes it easier for more people to pile on and potentially improve the entry.

* Accessibility: In the 'social' sense, related to collaboration and familiarity -- as simply a Person Using Ubuntu (and other Launchpad projects) at the moment, I know I'm hesitant to start smashing links in Launchpad unless I really feel strongly about accomplishing something; who knows what kind of "noise" certain actions will make? Will I end up bothering or wasting the time of people in more active roles (and so making a negative contribution on the whole)? I haven't checked myself with a stopwatch, but it feels like there is maybe a two or three minute window per-page-view to get a casual Launchpad visitor who just showed up to check a bug or try to solve a problem -- as opposed to a "developer" working in the system every day -- to decide to drop everything and "participate." From my perspective, the 'Nominate for FAQ' link (and much else on Launchpad!) has a "Someone Else's Problem" field around it unless I'm deeply involved and absolutely convinced something deserves to be in someone's FAQ, while I know [in general web-browsing experience] that I'll quite often end up making a wiki edit or writing a comment because those are familiar actions with familiar and understandable side-effects that can grab me within that 2 or 3 minute page-view window. Once that hurdle's leapt, most people think nothing of spending 30 minutes or more trying to help out with a comment or edit itself.

The FAQ system does look good for what it is, and a good start on an approach to the overall documentation question. But even "infrequently-asked" questions deserve good writeups and community editing if the people infrequently asking them care enough to dedicate their time. [Of course, som...

[floid (jkanowitz)]

Beating a dead (or already-acknowledged) horse here, but someone else's Slashdot comment - happens to be on a thread about a certain famous Ubuntu backer's commitment to cleanroom usability testing, yay! - echoes this issue and maybe explains it more succinctly: http://tech.slashdot.org/comments.pl?sid=1381337&cid=29535709

Fair-use quote:

8< - - - -
Re: STFU needs to be heard. (Score:5, Insightful)
by jc42 (318812) on Thursday September 24, @09:12PM (#29535709) Homepage Journal

> I always get asked, "How did you get good with computers?" To which I reply, "I was just able to read."

Well, the computer industry is slowly learning how to deal with people like you. More and more, they are implementing the "no documentation at all" standard. In the near future, it won't matter that you know how to read, because there will be no document anywhere for anything.

Actually, for Microsoft and Apple stuff, they're pretty much there now. Most of their new stuff has no written documentation at all. Their one remaining problem is that there are online forums where people actually write about such things, and google can quickly find them for you. But MS and Apple are working on ways of confounding that approach.

So soon you'll have no choice but to ask around to find out how to do something. If you do this via email or IM, your message will be hidden from others, so they won't be able to read the results.

I just wasted a number of hours trying to help a friend figure out how to deal with an incomprehensible Vista error message that makes logins totally fail. There are several thousand questions about the specific message online, and it appears that several hundred people have managed to fix it. But so far, none of the discussions we've found actually say what they did to fix it. So we've apparently hit a brick wall, despite all the bandwidth taken up by discussions of this particular problem. This illustrates how the MS community is learning to hobble those who can read, and ensure that there is nothing useful online on the topic.

Lessee; do I need a ;-) here?
8< - - - -

Back to me, restating my thoughts:

Big difference (in my mind) between a publicly-editable "KB" and Questions/Forums:

* a "KB" enforces deduplication in the same manner as a bug tracker, avoiding the above-described signal-to-noise problem.
(That comment happens to describe a Microsoft issue; Microsoft actually has a KB, like many projects/companies, and it can actually be useful if the KB editors have identified and worked out a solution for a given issue.)

Big difference(s) between my idea of a "KB" and a Wiki:

* More metadata, encouragement or requirement for "applies-to-version"/"applies-to-package" tags means better searchability and truthiness;

* Single-sign-on through Launchpad, so everyone subscribed to whine about bugs can also document workarounds, best-practices, improve on others' documentation, etc (erm.. separate wish, but in the meantime, maybe Launchpad should offer OpenID and the Ubuntu Wikis and Forums should accept it? Unless it already does now, been a while since I checked!)

* "Last approved version" vs. "Most recent version" per-topic is a more workabl...

[floid (jkanowitz)]

Thanks to all for reading and triaging.

I happened to catch Eddy Boxerman of Hemisphere Games restating the problem succinctly @
http://www.hemispheregames.com/2010/05/18/porting-osmos-to-linux-a-post-mortem-part-23/

Fair-use quote:
8<
"Didn’t Love: Lack of Documentation and Consensus

"In the Linux world there is so much choice and non-standardization, and it’s really hard to find out about things. Documentation is sorely lacking, and it’s hard to find solutions because when a question is asked, people don’t agree on the answer. Forum threads aren’t a good substitute for proper documentation, because forum threads can quickly become historical and fall out of date, meaning folks looking for answers spend a lot of time chasing down false-leads and asking themselves “Is this forum thread relevant to me? Is it really what I’m looking for?” In the Linux world, it seems to take a great deal of detective work and reverse engineering to get things done; the plethora of choice means that the newcomer is never certain about their choices — and there will always be someone who disagrees with you (often vocally)."
8<

I'm still keen on the 'rolling replace' approach [browsing and searching defaults to the 'authoritative'/edited entries with the option to browse ahead and collaborate on the 'pending' changes] to address some of the authority/questionable-advice issues... together with 'demanding' formal tagging of what version(s) of a project the article was trying to be authoritative about in the first place!

[I'm quite aware that there's often one authority in the 'bazaar,' but people tend to wish the distributor can offer a 'canonical' answer for a given issue. My point - which I've surely driven into the ground by now - is that selection, maintenance, and revision of the 'official' documentation could be automated to the same extent as a VCS/Bugzilla/Launchpad-dot-net.]

This would also help deal with the 'printed documentation is always doomed to be 2 releases behind' side-effect of the 6-month release cycle, which can be toxic to new users.

[floid (jkanowitz)]

Whups, should read- "I'm quite aware that there's often *no* one authority.."

[Matthew Paul Thomas (mpt)]

This is probably mutually exclusive with bug 240067, "Launchpad projects need wikis". A wiki and a knowledge base aren't the same thing, but it would be overkill to have both.

[floid (jkanowitz)]

Wow, people are still reading this! (Because they wish they could close it?)

I have to respectfully disagree to the extent that a sufficiently open Wiki is a free-for-all, and a sufficiently locked-down Wiki isn't actually so much a Wiki anymore - but still lacks some useful side-effects of "thinking in the KB pattern", particularly the way entry numbers (as with LP bug numbers) provide a quick sort as to which information is potentially most relevant/current.

As an example of some common trouble with Wikis:

https://help.ubuntu.com/community/UpgradeNotes is linked from the actual "Get Ubuntu" page on Ubuntu.com and happily defines "Long Time Support". It also contains a melange of instructions for numerous different versions... but actually isn't altogether horrible, just the fastest thing I could find that points to the "slippery Wiki slope."

The first search result for "compatible hardware" in that Wiki's search is this link:
https://help.ubuntu.com/8.04/installation-guide/amd64/hardware-supported.html
...yes, that link. To information that would be relevant to 8.04, if it weren't a 404.

The first apparently-relevant result hasn't been updated since 2011 (and in this case the stylesheet used makes it very, very difficult to recognize that the information may be stale when you're actually on the page, since it's in fine print. At the very bottom.)
https://help.ubuntu.com/community/RecommendedHardware

It's the dangling-reference/abandoned-resource problem that bothers me with Wikis the most ( sample discussion of the issue in hypertext systems in general here: http://www.ra.ethz.ch/CDstore/www5/www362/overview.htm ) ... in large projects, like an OS distribution, you end up with supposedly general topics competing... I'm not saying "RecommendedHardware" and "SupportedHardware" and "SuggestedHardware" actually all exist as articles in the Ubuntu Wiki, but it doesn't take much browsing to run into similar problems.

There's nothing really "magic" about the KB approach except that encouraging applies-to: version tagging, and resolving forward and back references (superceded by:, etc), along with the tendency toward serial numbering, makes locating actual relevant and correct information much easier.

And from a use-case perspective, I'd say a "KB" maybe fills the same niche as the "Answers" feature... but that again tends to be too focused on "resolving" / closing "issues" than actually improving documentation quality as a whole. (And when someone internal to the project discovers a bug that slipped into a release that needs a workaround... do they 'Ask' a question against the project for others to find an 'Answer' to? Unfortunately I do suspect producing quality documentation for free has competed - unconsciously - with the mission of a lot of FOSS companies to 'sell support'. Not ascribing actual malice, but it's just a conflict of interest that helps ensure the situation stays somewhat dire - why allocate resources to _reducing_ the need for the one revenue-generating product?)

Anyhey, just food for thought and I do think both Wikis (preferably more open than closed) and more formalized "KB"s both have a place - Wikis are sort...