Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Wiki inventor Ward Cunningham develops federated wiki (wired.com)
148 points by dctoedt on July 4, 2012 | hide | past | favorite | 70 comments


I think this could lead to a lot of interesting possibilities but I'll focus on one.

Lets say you're on Wikipedia and for example want to know about the assassination of J.F.K, the Wiki article contains most of the stock facts and so on, but lets say you want to know about the really crazy and out there conspiracy theories of Hoover and aliens and so on, a federated system would allow people to set their own page up and you can continue through to it. The sort of content that sends most Wikipedia moderators into catatonic seizures indexed and available.

There's plenty of other uses out there, I just think it's an interesting point that it'll allow anyone with enough time on their hands to create an archive of what they think is relevant as a counterpoint to what someone else thinks is relevant. Don't like the standard descriptor? Normally you'd have to fit a wiki mod, fork it off and have it as a counter point.


Lets say you're on Wikipedia and . . . you want to know about the really crazy

I am a Wikipedian. I edit sporadically these days, after being really active when I first started editing. Before I began editing, I read whole books about the history of Wikipedia and its internal processes. Because of the anarchy of how Wikipedia started out, with the typical Wikipedian having NIL research and editing experience (not even the research and editing experience implied by getting an undergraduate degree at a research university), you can find plenty of crazy on Wikipedia already.

http://en.wikipedia.org/wiki/Wikipedia:Lamest_edit_wars

And of course blatantly false assertions about the assassination of President John F. Kennedy have already stood on Wikipedia for a long time.

http://www.nytimes.com/2005/12/04/weekinreview/04seelye.html...

http://www.usatoday.com/tech/news/2005-12-11-wikipedia-apolo...

http://en.wikipedia.org/wiki/Wikipedia_biography_controversy...

The Deletionpedia experiment,

http://deletionpedia.dbatley.com/w/index.php?title=Main_Page

which seems to have fizzled out, was an archive of pages deleted from Wikipedia, in an attempt to make volunteer Wikipedia editors ("Wikipedians") more accountable for deleting articles. The result of the Deletionpedia experiment was to show that most content that is deleted from Wikipedia well deserves to be deleted, and probably doesn't serve any reader in any useful way anywhere on the Internet.

The Internet is already flooded with counterpoints. It's bad enough that in important subjects (for example, medical research) most of the best published literature is not available on the Internet (it is still behind publisher paywalls), while a lot of scam artists flood the Internet with websites that exist only to sell something that cheats consumers.

I do look forward to seeing how Ward Cunningham's experiment develops. MeatballWiki introduced some thoughtful discussion of online communities to the broader online world,

http://meatballwiki.org/wiki/OnlineCommunity

and that was enabled by the innovation of wiki sites, that any user could edit a webpage already posted on the site. There is a place for wikis, and some wikis are very helpful,

http://www.artofproblemsolving.com/Wiki/index.php/Main_Page

but the last thing the world needs is more content forks for crazy points of view that get edited off of Wikipedia.


I don't think any of what you say particularly supports your final point. The internet is flooded in counterpoints, but few of them are on Wikipedia, or in Wikis. And none of them are as visible as Wikipedia's take on things. I think people are increasingly relying on Wikipedia to provide The Truth(tm) -- mostly because it's gotten so damn good at delivering it, the JFK examples notwithstanding.

Counterpoints seem to survive in ideological communities that have their own closed world view and their own knowledge databases which are considered to be definitive.


I've always found you can learn a surprising amount about a subject by reading it's wikipedia 'Talk' page.


> The sort of content that sends most Wikipedia moderators into catatonic seizures indexed and available.

The reason it is anathema to the Wikipedia concept is because Wikipedia wants its name on good content, not Alex Jones JOOZ-DID-IT (Oh, and Aliens-Big-Pharma-Bilderberger-USPS-Did-9/11 Too) bullshit, unless such bullshit is clearly marked as coming from a specific minority viewpoint.

I think that's a great reason to keep the crap on the current version of Wikipedia to a minimum: The world is confusing, especially when there are people like Alex Jones out there trying to make it more confusing to sell more videos. Having something where you have a good chance of getting the Sane People Version of controversial (or only-controversial-because-of-the-crazies) topics is invaluable if you want somewhere to start. Especially if it's a topic that books aren't really written about, like specific health scams.

So the Federated Wiki universe will need some way to mark specific forks of articles as being from a given group of people, so people aren't mislead into thinking that the sane editors approve of the version of reality Alex Jones is selling the gullible.


Exactly, the mods are there to keep the crazy out but occasionally you need to reference some of the more outlandish things for various reasons and letting people for things off into tinfoilhatswiki.net would be good for the mods sanity and useful for people who need to reference the over the top stuff. Not sure how the link direction would work, but I'm sure there's a way.


I agree that some way to remove links to forks that are spam or that you find reprehensible is a good idea, but the whole point of forking instead of editing is that the forks aren't approved.

As I said elsewhere, I (nor the people I quoted) meant to represent this as an alternative to Wikipedia itself. I'd see it more as some that a wiki farm like Wikia could use to allow editors to put more of their own perspective into an article (that's my opinion, not the Smallest Federated Wiki developers' opinion).


> the forks aren't approved.

I get this. My whole point is that it needs to be very obvious where a fork comes from and what else the people responsible for it are responsible for.


... and then the spammers have a field day: on every page, a list of 10,000 forks, re-titled with names like "Luis Vitton CHEAP$!$" and "U 2 can aFf0rd Rolex!"

Interesting idea when all players are playing by the same rules and with the same intent. Not as appealing when the most active are there simply to generate noise.


This has been a problem for every Internet service everywhere ever. That hasn't stopped email from being useful, forums from being useful, traditional wikis (weird to say "traditional wikis", now that there's a new kind of wiki) from being useful, websites from being useful, etc.

The spam problem always has to be solved. That said, very active wikis have shown themselves to be more resistant to spammers than most online services. Because anyone can remove the spam, there's a pretty high cost to keeping the spam flowing, particularly if there are sufficient impediments to automation in place.


Of course every non-moderated service has spam, that was kinda the point of my post... Maybe the part you missed was that they introduced a new way of spamming via forking pages... I don't think I ever made any allusions to something not happening because of it. Thanks for the pep talk though, I'll keep it in mind as I do my nightly task of helping clean up spam on the sites where I moderate.


Spamming via forking seems, to me, to be very similar to trackback spam on blogs. But, yes, it will probably require a bit of novelty in terms of how one fights it.


It seems odd to me to focus on Wikipedia as the comparison point, but then again I was on C2 back-in-the-day and have always seen the quintessential wiki as a superior version of a forum rather than Wikipedia. If we are comparing to currently-active technology, I would argue that the federated wiki concept has the potential to bring tumblr or del.icio.us style contributions to a different, longer-form audience.

I do worry that the federated concept will make authorship too important. The lack of permanent credit on C2 was part of what made it valuable: people more often wrote things to contribute value when they didn't have the motivation of scoring points, gaining karma or making a name for themselves.


TFP: http://wardcunningham.github.com/

in the spirit of slashdot: the fucking project



A little off topic, but a friend of mine had a dinner conversation with Ward Cunningham and refactoring expert Ralph Johnson a few months ago and posted it on youtube: http://www.youtube.com/watch?v=jqGYoKvekik The video could have used some editing (shorter!) but has some interesting parts.


"To run The Simplest Federated Wiki, you’ll need your own web server, which Cunningham thinks is an important part of the project."

Sounded great till that point. He's putting his own nerd centric bias into it. If I'm a history guy, not a tech guy, and I want to "fork", asking me to run my own "server" or whatever is a non-starter.


Smart money says if there's demand for it people will be able to buy custom instances with it for a variable amount of money.


FTA: To make Federated Wiki easier to adopt, there’s a one-click installer to deploy a server to Amazon Web Services.


Ah I missed that, saved the full thing for later.


Here's the one-click installer if you're interested: https://stackhammer.cloudsmith.com/tools/quickdeploy.html#fe...


Agreed. Github is popular even though git itself is obtuse.


Exactly, you just need to reduce the friction level to a certain point. Saying it requires it's own web server sounds like a chore, but Mac OS X comes with a web server built in. You could in theory run your own local Wiki. Or someone clever could sell cheap instances powered off Amazon/Google/Heroku/cheap VPS host with a custom web frontend that does the heavy lifting.

TL;DR good companies make life easy for people who want it without hassle.


Since we have free accounts on app engine, heroku etc... and even gdrive or dropbox and so on... wont it be easy for someone to write a simple web app or ipad app or something like that allows the masses to run their own wiki's without being technically sophisticated. To an end user the process could be as easy as install app from chrome webstore... specify some minimal data... and you have an app engine account hosting your tiny wiki. If tomorrow you choose to move it heroku, it would as simple as a button click and registering on heroku.


I had the exact opposite reaction to that. "But is it too nerdy to catch on? To run The Simplest Federated Wiki, you’ll need your own web server" How is running a web server nerdy? Does this person not realize that wikipedia is also run on a web server, or that most people will be reading this very article online? The Web is one of the biggest things going on right now and certainly one of the fastest-growing. It concerns me that people aren't even taught the basics of running their own servers. How are you supposed to participate in the Web if you don't even have a web server?


"Does this person not realize that wikipedia is also run on a web server."

The difference is that you don't have to run your own server to edit or otherwise participate in Wikipedia. You don't need your own server to have a blog on Tumblr or WordPress.com.

And since Simplest Federated Wiki requires either Ruby/Sinatra or Node.js/Express, you need more than just a shared hosting account to run it, which raises the barrier to entry (though the one-click installer, or PaaS providers, help lower it again).

It may eventually be a non-issue if someone builds a "wordpress.com for federated wikis."


Not sure if you're serious?

It's like saying I should have to know how to lay asphalt to drive on the highway.


Most people should at least know how to drive a car. Well I guess you could use public transportation and still get a large benefit out of the public highway system. But having your own car lets you do things public transport never will. And it would be ridiculous for someone to claim that learning to drive is too nerdy, even if it is pretty complicated and has a lot of arbitrary rules you have to go to classes to learn.


Most people should at least know how to drive a car.

Why? Sure, it lets you do some things that almost nothing else does, but you can usually work around them, and it imposes a great cost of both time and money. Why should one spend them?

And likewise, learning how to run a web server on the public Internet (as opposed to a toy server on your LAN) requires a huge amount of time and knowledge, in terms of learning how to use the software, secure it, keep it spam-free, keeping up with the updates and security vulnerabilities, configuring DNS, etc. And at the end of the day, it can still be taken down by the hosting/VPS provider or ISP, just like your Wordpress.com blog.

So, is it really worth it? I doubt it.


Driving a car is like using a browser. It's not the infrastructure. Servers are infrastructure.


Also: wikipedia offers its entire content available as a tarball. Fork and run with it as you wish. While not "federated" it's pretty damn open.


The text, anyway, I don't think a dump including the images has been available for quite some time. And that's not just due to the size -- though that's also a concern, I imagine -- but mostly due to copyright concerns. Text submitted to Wikipedia is for the most part under a copyleft license, images often are not.


Holy smokes, how big is it?


Depends on which language you'd like: http://en.wikipedia.org/wiki/Wikipedia:Database_download


8 GB bzip2-compressed, for the English Wikipedia, if you download the dump that contains only the current revisions of article-space pages. Much bigger if you want the one with every historical revision.


Thanks. Important to mention that is text only.


small enough to have offline on your phone, as several apps do.


The text of a large percentage of the current version of English Wikipedia, without discussion pages, is that small.


As I see it, one of the biggest benefits of traditional wikis' linear history was that it fostered quick back-and-forth collaboration between users. Unless there's a strong mechanism for discovering changes made by other users, a forking model seems like it'd rapidly result in a bunch of different (and potentially non-mergeable) versions of any page.


Could this be layered on top of the current Wikipedia through browser plugins? And how would discoverability work in the context of 10,000 current versions of a wiki page? It's hard enough when there's 30 different forks on github.

edit: federated search, federated social? How would the system prevent search gaming and social gaming?


It's a bit too early to worry about 10k forks for a single page, isn't it? The typical use cases -- certainly the early ones, when it's not widespread -- might involve numbers of forks that aren't too different from GitHub.


Much better headline than the one actually on Wired, "Wiki Inventor Sticks a Fork in His Baby."


If the problem is having the knowledge hosted in one place only, controlled by "one" person only, the solution should be to decentralize, not make everyone able to copy it everywhere. Plus, this solution seems to add more work to the contributors, since you'll need to review each fork/edit and choose which to merge or not.

On the other hand, a wiki working like a P2P network, where everyone hosts the same version and edits are automatically propagated accros the web would solve the "problem" without adding more job on contributors.


Instead of plain EC2 installer t needs a Heroku Installe. Then anyone can have it running for essentially free in no time


see "In praise of cooperation without coordination: Clay Shirky at TEDGlobal 2012"

"The result is Git, distributed version control, which Shirky is here to explain to us. It’s a distributed workflow that brings chaos back into the system. Yet there’s a beautiful innovation to ensure that said chaos doesn’t promptly override said system: a signature that creates a unique identifier for every single exchange. This enables cooperation without coordination. “I tell you this not because it is great that open-source programmers now have a tool that supports their philosophical way of working,” Shirky says. “I tell you all of this because of what it means for how communities come together.”"

http://blog.ted.com/2012/06/29/in-praise-of-cooperation-with...


So where is the link to the actual project ?



With always-on, always-connected devices everywhere, we are not that far from every device being a web-server by default, and every device being a router for web traffic.

Data storage, and even computation may be centralized in the cloud... but why not distributed distribution?


Ye olde, but still worth watching: Van Jacobson's Google Talk from 2006, "A New Way to look at Networking": http://www.youtube.com/watch?v=gqGEMQveoqg


Holy mother of misleading titles.


When I talked with Ward in 2005, he mentioned his vision of a Wiki had a continuum of pages where he said you could take a page and browse the different "directions" people had edited it to.


That sounds interesting, but useless. I only want to see the most 'correct' version. It's almost as if saying when you use an app you can also use many of the apps versions, including its beta or alpha, regardless of how incorrect or useless they are with the current version.

I also think that information needs to be organized and many of his ideas make the collection or the organization of information almost impossible.

But nevertheless very interesting.


Did you notice that most articles on Wikipedia, although mostly 'correct', are very dry? And this is because they try to be as 'correct' as possible. But it brings to a point where some (most??) users can't use it comfortably anymore. Think children, who are trying to find some basic information about the subject they are interested in. My 8yo recently has gone through the 'dinosaurs' stage. Drawings, books, stories, films. You think she found this fascinating: http://en.wikipedia.org/wiki/Dinosaur ?? Nope. Here comes in the federated version, with 'alternative' of the story. Tailored for the 4-10yo audience. Is it less 'correct'? Perhaps. But useful nonetheless.


Wikipedia already has alternate versions of articles such as https://simple.wikipedia.org/wiki/Dinosaur

Something similar could be done for various reading ages.


The article describes this as being like a fork. For example, choosing between OpenOffice and LibreOffice, or ffmpeg and libav. Both are the current version, but with different ideas and implementation.

If two groups are editing a book about Python (for example) on a wiki, they might have different ideas about how a particular concept should be explained. They fork that, and both versions can exist in parallel, even exchanging ideas when suitable.


I hear what you're saying about only wanting the most "correct" version, and no doubt that's how most people feel when looking for information, but about some things you have an ability and perhaps an obligation to contribute to deciding what is the most correct version...

This suggests to me that Ward is onto something here, but also that you may have a point about it's limitations in its current form. I hope the hypothes.is project (funded by kickstarter, http://hypothes.is) succeeds, and I wonder if some degree of integration between the two is possible so that reputation and correctness rating across wiki-nodes (or any such federated system) can naturally "bubble up" to top visibility with some citation/indication of the bubbling via the hypothes.is client, regardless of which specific fork of the wiki you happen to be looking at... I'd love to discuss this possibility further with Ward, hypothes.is, or anyone else interested since they'll both be open source projects any of us can try to make this happen in :-)


Well, you're only getting the correct version in terms of Wikipedia's processes. This relates to both a horizontal axis of what narratives (for lack of a better abstraction) are being represented and a vertical axis relating to the depth to which a narrative is mentioned, if at all.

Both of these are, famously, super disputed on Wikipedia itself: people fight about the contents of articles about controversial issues (→ NPOV), and people fight about the extent to which certain things are covered (→ notability).

I think there's room for multiple, equally "correct" versions of an article on both of these dimensions; but particularly regarding the notability criteria.


Who decides which version is the most correct? Read the same Wikipedia article on history in different languages and you'll find out that there's no such thing as the 'correct version'.


Didn't I see something from Dave Winer doing similar thing for Twitter, personal stream?


"To make Federated Wiki easier to adopt, there’s a one-click installer to deploy a server to Amazon Web Services." <- oh yeah, super-federated.


What's not federated about that? And obviously you don't have to run it on EC2, that's just the easiest way at the moment.


webring?


I don't see how this is anti-Wikipedia. But I suppose conflict sells, so linkbait (to be clear, linkbait on the part of Wired, not the submitter) titles like these are inevitable.

Also, for a lot of people, 'wiki' is simply short for 'Wikipedia'. That's unlikely to change at this point. I don't know what Ward can do to make a Federated Wiki (great name, BTW) achieve any kind of success with that kind of barrier.


I'm the author of the article, and I can see why people got confused. I don't think Max Ogden was implying that it's a bad thing that you have to give up your view point on Wikipedia - and Ward Cunngingham definitely didn't imply that in my conversation with him. I regret that readers thought that. I don't think either Ogden or Cunningham are expecting this to replace Wikipedia, but they are hoping that it will be useful for other wikis.


Forking does not lead to spooning. This sounds like it will quickly become an impenetrable rat's nest. Better to fight it out on the one true source, Wikipedia.


The Internet is already made of "rat's nests" like these. DNS records, BGP route tables, cross-domain embedding, all of these are pulling together information from lots of inconsistent sources to render a page in your browser, and it works pretty well.


Google is a rats nest but you can eventually find what you want, I go to Wikipedia first.


The problem Wikipedia is having at the moment is that some people have taken over sections of it as their own "turf". And there will always be a holy war between the completionists and the deletionists, etc. With a federated wiki, you get to pick what kind of info you want to see, and who gets to curate it.


If deletionists had been running the show when Wikipedia got started, it would have failed.


At last you can fork the truth and make your own version of it. I imagine this will be a hit with governments everywhere.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: