Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at
Visualizing Complexity

The comprehensibility of a system seems to be inversely proportional to that system's complexity.  Visualizations often help make the data within such systems more attainable.  The following encompasses an incredible collation of visualization tools for making the complex more visually manageable:  Their applicability to visualizing social and neural networks is obvious, but they may also serve to inspire other approaches to expressing data to the end user.


Posted 11-09-2006 12:01 PM by Billy McCafferty
Filed under:



Michal Grzegorzewski wrote re: Visualizing Complexity
on 11-09-2006 5:19 PM

heh, you gonna bite that $ 1 mln :)

After your last post regarding statistical methods i begun my own crusade ;)

so - good luck to you! ;)

Michal Grzegorzewski wrote re: Visualizing Complexity
on 11-09-2006 5:46 PM

btw - don't you think, that Spearman's rank correlation coefficient would be more appropriate for this task? I believe that the only way to get that 10% is to use some not well known metaheuristic :)

Billy McCafferty wrote re: Visualizing Complexity
on 11-09-2006 6:59 PM

I couldn't agree more that something unique will need to be employed!  My only question concerning Spearman's rank is that user-ratings are already put into "ranks" before the calculation is applied - so I'm not sure what benefit Spearman's would provide over Pearson's.  But I haven't applied Spearman's yet, so I could be completely out of line here! ;)

Here's what I'm currently using and/or developing:  Pearson correlation coefficient to get an approximate prediction; genetic algorithms to find near-optimal Pearson correlation coefficient variables; cluster-based approaches to Pearson; Bayesian networks to predict ratings based on each user's personal history with respect to actors, directors, categories, etc; and looking at visual graphing techniques to spur new ideas for other non-linear approaches.  And if I get really ambitious, I'll apply neural networks to help hone the cluster-based approach (

I'm assuming the 4.9% gain is about as much as can be theoretically attained using pure linear approaches.  But then again, the leaders may already be using non-linear approaches as well.

The biggest hindrance right now is brute-force calculation generation for the Pearson baseline.  I've suckered a couple people at work to let me run the process on their machines as well!  Good luck and keep me posted...I'll also look more into the Spearman approach as well.

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Google Reader or Homepage Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of
Red-Gate Tools For SQL and .NET


SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
NHibernate Profiler
Balsamiq Mockups
JetBrains - ReSharper
Web Sequence Diagrams
Ducksboard<-- NEW Friend!


Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers


Community Server (Commercial Edition)